Evaluating the Impact of Scaffolding and Visualizations for Mutation Testing Exercises in Software Engineering Education
Software testing is an important approach to increase code quality, and thus part of software engineering education. Mutation testing is an effective testing technique with increasing adoption in industry, measuring and improving how well a test suite can detect small changes to a program under test. This paper aims to study the use of mutation testing tools in an educational setting and understand students’ conceptual and technical challenges in applying mutation testing. It reports on two case studies of incorporating mutation testing into software engineering curricula.
The Scaffolding Study explores the impact of using different mutation testing tools directly or indirectly via a uniform interface provided by an educational infrastructure. We observe that scaffolding (indirect tool use) reduced technical difficulties and improved the consistency and quality of student submissions.
The Visualization Study explores the impact of different forms of output of a mutation testing tool. Specifically, it assesses to what extent visualizations support students in the tasks of reasoning about mutants and writing detecting tests for them. We observe that visualizations improved the average mutation scores, with lower-performing students seeing a boost in particular.
We further explore challenges around automatic assessment of mutation-testing assignments. For example, we observe that even with assignment scaffolding, between 18% and 21% of student submissions required manual modifications to successfully execute.
Tue 24 JunDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
14:00 - 15:30 | |||
14:00 20mTalk | Layered Learning: Teaching CI/CD and Software Testing through Storytelling Software Engineering Education | ||
14:20 20mTalk | Sojourner under Sabotage: A Serious Testing and Debugging Game Software Engineering Education Philipp Straubinger University of Passau, Tim Greller University of Passau, Gordon Fraser University of Passau | ||
14:40 20mTalk | Evaluating the Impact of Scaffolding and Visualizations for Mutation Testing Exercises in Software Engineering Education Software Engineering Education Hannah Potter University of Washington, Ana Paiva INESC TEC, Faculty of Engineering, University of Porto, Domenico Amalfitano University of Naples Federico II, Anna Rita Fasolino Federico II University of Naples, Porfirio Tramontana University of Naples Federico II, René Just University of Washington | ||
15:00 20mTalk | A Comparative Study on ChatGPT and Checklist as Support Tools for Unit Testing Education Software Engineering Education Zihan Fang Vanderbilt University, Jiliang Li Vanderbilt University, Anda Liang Vanderbilt University, Gina Bai Vanderbilt University, Yu Huang Vanderbilt University | ||
15:20 10mTalk | Creating a Student-Friendly PaaS Platform: Experiences with Tsuru in Software Engineering Education Software Engineering Education Robert Chatley Imperial College London, Jason Bailey Imperial College London, Ivan Procaccini Imperial College London, Zaki Amin Imperial College London, Estibaliz Fraca Imperial College London |
Sirius is located just behind the registration desk.
Facing the registration desk, its entrance is on the right.