2019/07/19 by Giovanni Grano, Christoph Laaber, Annibale Panichella +1 · 1 voice · 29 citations
Computer Science · #Algorithm #Code coverage #Computer science #Distributed computing #Fault coverage #Heap (data structure) #Machine learning #Operating system #Overhead (engineering) #Regression analysis #Regression testing #Reliability engineering #Software #Software Engineering Research #Software Reliability and Analysis Research #Software Testing and Debugging Techniques #Software development #Test case #Test strategy #Test suite #Unit testing #cs.NE #cs.SE
paper · pdf · doi:10.1109/tse.2019.2946773
published in IEEE Transactions on Software Engineering 47(11), 2332-2347 (IEEE Computer Society) · 16 pages, 3 figures, accepted for IEEE Transaction on Software Engineering
arxiv published 2019/07/19 · openalex publication_date 2019/10/11 · openalex created_date 2019/10/18 · arxiv created 2019/11/08 · arxiv updated 2019/11/11 · openalex updated_date 2026/08/05
Automated test case generation is an effective technique to yield high-coverage test suites. While the majority of research effort has been devoted to satisfying coverage criteria, a recent trend emerged towards optimizing other non-coverage aspects. In this regard, runtime and memory usage are two essential dimensions: less expensive tests reduce the resource demands for the generation process and later regression testing phases. This study shows thatperformance-awaretest case generation requires solving two main challenges: providing a good approximation of resource usage with minimal overhead and avoiding detrimental effects on both final coverage and fault detection effectiveness. To tackle these challenges, we conceived a set of performance proxies—inspired by previous work on performance testing— that provide a reasonable estimation of the test execution costs (i.e., runtime and memory usage). Thus, we propose an adaptive strategy, called aDynaMOSA, which leverages these proxies by extending DynaMOSA, a state-of-the-art evolutionary algorithm in unit testing. Our empirical study —involving 110 non-trivial Java classes—reveals that our adaptive approach generates test suite with statistically significant improvements in runtime (−25 percent) and heap memory consumption (−15 percent) compared to DynaMOSA. Additionally, aDynaMOSA has comparable results to DynaMOSA over seven different coverage criteria and similar fault detection effectiveness. Our empirical investigation also highlights that the usage of performance proxies (i.e., without the adaptiveness) is not sufficient to generate more performant test cases without compromising the overall coverage.