2017/06/06 by Peter Charbachi, Linus Eklund, Charbachi, Peter +5 · 1 voice
Computer Science · #Software Engineering Research #Software Reliability and Analysis Research #Software Testing and Debugging Techniques #cs.SE
paper · pdf · doi:10.48550/arxiv.1706.01636
openalex publication_date 2017/06/06 · openalex created_date 2021/05/24 · openalex updated_date 2026/07/28
Testing is an important activity in engineering of industrial software. For\nsuch software, testing is usually performed manually by handcrafting test\nsuites based on specific design techniques and domain-specific experience. To\nsupport developers in testing, different approaches for producing good test\nsuites have been proposed. In the last couple of years combinatorial testing\nhas been explored with the goal of automatically combining the input values of\nthe software based on a certain strategy. Pairwise testing is a combinatorial\ntechnique used to generate test suites by varying the values of each pair of\ninput parameters to a system until all possible combinations of those\nparameters are created. There is some evidence suggesting that these kinds of\ntechniques are efficient and relatively good at detecting software faults.\nUnfortunately, there is little experimental evidence on the comparison of these\ncombinatorial testing techniques with, what is perceived as, rigorous manually\nhandcrafted testing. In this study we compare pairwise test suites with test\nsuites created manually by engineers for 45 industrial programs. The test\nsuites were evaluated in terms of fault detection, code coverage and number of\ntests. The results of this study show that pairwise testing, while useful for\nachieving high code coverage and fault detection for the majority of the\nprograms, is almost as effective in terms of fault detection as manual testing.\nThe results also suggest that pairwise testing is just as good as manual\ntesting at fault detection for 64% of the programs.\n