ICST 2023
Sun 16 - Thu 20 April 2023 Dublin, Ireland
Sun 16 Apr 2023 11:30 - 12:00 at Grand canal 4 - Session 1

Racket is a functional programming language designed for programming language design and implementation, and is the basis for programming education at many colleges and universities. Recent research results have shown that mutation analysis can be an effective substitute for manual grading of student test cases. In order to evaluate its efficacy in our college’s introductory programming courses, we created a prototype mutation analysis tool for Racket. We describe the design and implementation of the tool and perform a feasibility study using two assignments from an intro CS course where student test suite thoroughness was evaluated by hand by human graders. In our results, we find a moderate correlation between mutation score and hand-grading test suite quality score and conduct a qualitative analysis to identify situations where mutation score and hand-grading score do not correlate. We find that, compared to hand-grading, mutation analysis may require more stringent adherence to the interface specified in an assignment as well as more precisely specified assignments. On the other hand, inter-reviewer reliability is a known challenge of hand-grading, and we observe several instances where hand-graders may have assigned the wrong score. Given the relatively cheap cost to providing mutation analysis feedback to students (compared to hand-grading feedback), mutation analysis still provides the opportunity to provide faster, more frequent, feedback to learners, enabling them to improve their testing practices further. Future work will study the effectiveness of various mutation operators in racket and perform larger-scale evaluations.

Sun 16 Apr

Displayed time zone: Dublin change

11:00 - 12:30
11:00
30m
Talk
Analysis of mutation operators for FSM testing
Mutation
Danial Nikbin Carleton University, Yvan Labiche
11:30
30m
Talk
A Tool for Mutation Analysis in Racket
Mutation
Bambi Zhuang Northeastern University, James Perretta Northeastern University, Arjun Guha Northeastern University and Roblox Research, Jonathan Bell Northeastern University
12:00
30m
Talk
The Inversive Relationship Between Bugs and Patches: An Empirical Study
Mutation
Jinhan Kim KAIST, Jongchan Park KAIST, Shin Yoo KAIST
Pre-print