Research‎ > ‎


Evolutionary Testing of Stateful Systems: a Holistic Approach

posted Feb 23, 2011, 6:05 AM by Matteo Miraz

Testing should be one of the key activities of every software development process. However it requires up to half of the software development effort when it is properly done. One of the main problems is the generation of "smart" tests to probe the system, which is both difficult and time-consuming. The research community has been proposing several ways to automate the generation of these tests; among them, the search-based techniques recently achieved significant results.

This doctoral dissertation presents TestFul, our evolutionary testing approach for stateful systems; it is tailored to work on object-oriented systems. It uses a holistic approach to make the state of object evolve, to enable all the features the class provides, and to generate the shortest test with the utmost coverage for the class under test. We employ several complementary coverage criteria to drive the evolutionary search. We aim to generate tests with high fault detection effectiveness. To this end, we consider the system from complementary perspectives and we combine white-box analysis techniques with black-box ones. The evolutionary search is completed with a local one, and we establish a synergic cooperation between them. The evolutionary search concentrates on evolving the state of objects, while the local search detects the functionality not yet exercised, and directly targets them.

All the proposal were subject to an extensive empirical validation. We devised a benchmark composed of independent benchmarks for tests, public libraries, and third party studies. As comparison, we consider both search-based, symbolic, and traditional (i.e., manually generated by human being) approaches. The achieved results were encouraging: TestFul efficiently generate tests for complex classes and outperforms the other approaches. The proposals presented in this dissertation open new interesting research directions. On one side, one can continue refining the search strategy, by considering more advanced search techniques and by leveraging more advanced coverage criteria. On the other side, one can adapt the approach to work either at a coarse-grained level -and focus on the integration testing- or on other kind of stateful systems (e.g., components or services).

The slides used during my defence are available here.

A Component-oriented Metamodel for the Modernization of Software Applications

posted Feb 11, 2011, 12:07 PM by Matteo Miraz

The modernization of a software system is a complex and expensive task and requires a deep understanding of the existing system. The capability of re-factoring a complex application into some high-level views is mandatory to elicit its structure and start localize possible changes. The high number of different implementation technologies imposes a model-based, neutral approach to reconstruct the structure and hide unnecessary details. OMG supports this view and proposes KDM (Knowledge Discovery Metamodel) as means to describe software systems in detail, but unfortunately KDM supports a component-oriented decomposition of the system of interest only partially.

To bypass this limitation, the paper proposes the COMO (Component-Oriented MOdernization) metamodel to extend KDM, by borrowing recurring concepts from component-based solutions and software architectures, and to support a proper componentization of the system we want to modernize. The paper presents the main elements of the COMO metamodel and exemplifies them on a simple case study.

Improving Evolutionary Testing by Means of Efficiency Enhancement Techniques.

posted Sep 13, 2010, 10:38 AM by Matteo Miraz   [ updated Sep 13, 2010, 2:40 PM ]

TestFul is a novel evolutionary testing approach for object-oriented programs with complex internal states. In our preliminary experiments, it already outperformed some of the well-known search-based testing approaches. In this paper we show how TestFul can be further improved by leveraging three efficiency enhancement techniques: seeding, hybridization, and fitness inheritance. We considered four extensions of TestFul: three using each enhancement separately, and one using all of them at the same time. We used these new versions of TestFul to generate tests for six Java classes taken from the literature, public software libraries, and third party benchmarks. We compared the performance of the original TestFul against these new versions. Our results show that each enhancement technique results in a significant speed-up and, even more interesting, the highest improvement is achieved when all the enhancements are combined together.

The slide used in the presentation are available here.

TestFul: Automatic Unit-Test Generation for Java Classes.

posted Sep 13, 2010, 10:37 AM by Matteo Miraz   [ updated Sep 13, 2010, 2:40 PM ]

This paper presents TestFul, an Eclipse plugin for the generation of tests for Java classes. It is based on the idea of search-based testing, working both at class and method level. The former puts objects in useful states, used by the latter to exercise the uncovered parts of the class.

TestFul: an Evolutionary Test Approach for Java.

posted Sep 13, 2010, 10:35 AM by Matteo Miraz   [ updated Sep 13, 2010, 2:40 PM ]

This paper presents TestFul, an evolutionary testing approach for Java classes that works both at class and method level. TestFul exploits a multi-objective evolutionary algorithm to identify the “best” tests. The paper introduces the main elements of TestFul. It also compares TestFul against well-known search-based solutions using a set of classes taken from literature, known software libraries, and independent testing benchmarks. The comparison considers statement and branch coverage, size of generated tests, and generation time. On considered classes, TestFul generates better tests than other search-based solutions, and achieves higher structural coverages with tests small enough to be usable.

TestFul: using a Hybrid Evolutionary Algorithm for Testing Stateful Systems.

posted Sep 13, 2010, 10:34 AM by Matteo Miraz   [ updated Sep 13, 2010, 2:40 PM ]

This paper introduces TestFul, a framework for testing stateful systems and focuses on object-oriented software. TestFul employs a hybrid multi-objective evolutionary algorithm, to explore the space of feasible tests efficiently, and novel quality metrics, based on both def-use pairs and behavioral coverage, to judge the quality of tests.

We compare our framework against random testing by considering the level of coverage, the size of generated tests, and the time required to generate the tests. Our preliminary results show the validity of the approach: TestFul outperforms random testing in most of the cases.

A Flexible and Semantic-aware Publication Infrastructure for Web Services.

posted Sep 13, 2010, 10:33 AM by Matteo Miraz   [ updated Sep 13, 2010, 2:41 PM ]

This paper presents an innovative approach for the publication and discovery of Web services. The proposal is based on two previous works: DIRE (DIstributed REgistry), for the user-centered distributed replication of service-related information, and URBE (UDDI Registry By Example), for the semantic-aware match making between requests and available services. The integrated view also exploits USQL (Unified Service Query Language) to provide users with higher level and homogeneous means to interact with the different registries. The proposal improves background technology in different ways: we use and integrate USQL as a high-level language to state service requests, widen user notifications based on URBE semantic matching, and apply URBE match making to all the facets with which services can be described in DIRE. All these new concepts are demonstrated on a simple example scenario.

A Distributed Approach for the Federation of Heterogeneous Registries.

posted Sep 13, 2010, 10:30 AM by Matteo Miraz   [ updated Sep 13, 2010, 2:41 PM ]

Registries play a key role in service-oriented applications.

Originally, they were neutral players between service providers and clients. The UDDI Business Registry (UBR) was meant to foster these concepts and provide a common reference for companies interested in Web services. The more Web services were used, the more companies started create their own “local” registries: more efficient discovery processes, better control over the quality of published information, and also more sophisticated publication policies motivated the creation of private repositories.

The number and heterogeneity of the different registries -besides the decision to close the UBR- are pushing for new and sophisticated means to make different registries cooperate. This paper proposes DIRE (DIstributed REgistry), a novel approach based on a publish and subscribe (P/S) infrastructure to federate different heterogeneous registries and make them exchange information about published services. The paper discusses the main motivations for the P/S-based infrastructure, proposes an integrated service model, introduces the main components of the framework, and exemplifies them on a simple case study.

Hybrid Service Oriented Architectures: A Case Study in the Automotive Domain.

posted Sep 13, 2010, 10:29 AM by Matteo Miraz   [ updated Sep 13, 2010, 2:41 PM ]

Vehicles are becoming complex software systems with many components and services that need to be coordinated. Service oriented architectures can be used in this domain to support intra-vehicle, inter-vehicles, and vehicle-environment services. Such architectures can be deployed on different platforms, using different communication and coordination paradigms. We argue that practical solutions should be hybrid: they should integrate and support interoperability of different paradigms. We demonstrate the concept by integrating Jini, the service-oriented technology we used within the vehicle, and JXTA, the peer to peer infrastructure we
used to support interaction with the environment through a gateway service, called J2J. Initial experience with J2J is illustrated.

Refactoring object oriented software: cross-cutting concerns identification and isolation.

posted Sep 13, 2010, 10:28 AM by Matteo Miraz   [ updated Sep 13, 2010, 2:42 PM ]

The capability of dividing a big problem in smaller, independent and more tractable units is crucial for the success of the project. Object oriented decomposition uses to work well to manage decomposition in a software project. Altough there are some concerns that overcome the object oriented software decomposition schema. These concerns are called cross-cutting concerns.

Being able to identify and isolate cross-cutting concerns, present in an object oriented software, can improve the software readability and maintainability. The identification of these concerns is called aspect mining. The mined crosscutting concerns can be implemented as isolated compilation units, called aspects. Aspect oriented languages were born as object oriented extension to implement isolation of concerns.

In this work we present the problem of cross-cutting concerns and their identification and isolation in an existing object oriented software.

1-10 of 12