2018/10/25 by Dag I. K. Sjøberg, Sjøberg, Dag I. K., Gunnar R. Bergersen +1
Computer Science · #FOS: Computer and information sciences #Open Source Software Innovations #Software Engineering (cs.SE) #Software Engineering Research #Software Engineering Techniques and Practices #cs.SE
paper · pdf · doi:10.48550/arxiv.1810.10791
openalex publication_date 2018/10/25 · arxiv created 2018/11/28 · arxiv updated 2018/11/30 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
In a recent article, Falessi et al. (2017) call for a deeper understanding of the pros and cons of using students and professionals in experiments. The authors state: we have observed too many times that our papers were rejected because we used students as subjects. Good experiments with students are certainly a valuable asset in the body of research in software engineering. Papers should thus not be rejected solely on the ground that the subjects are students. However, the distribution in skill is different for students and professionals. Since previous studies have shown that skill may have a moderating effect on the treatment of participants, we are concerned that studies involving developers with only low to medium skill (i.e., students) may result in wrong inferences about which technology, method or tool is better in the software industry. We therefore provide suggestions for how experiments with students can be improved and also comment on some of the alleged drawbacks of using professionals that Falessi et al. point out.