End-to-End Automation of Feedback on Student Assembly Programs
(Experience Paper.) We developed a set of tools designed to provide rapid feedback to students as they learn to write programs in assembly language (LC-3, a RISC-like educational instruction set architecture). At the heart of the system is an extended version of KLEE, KLC3, that enables us to both identify issues and perform equivalence checking between student code and a gold (correct) version of each assignment. Feedback begins when students edit their code using a VSCode extension that leverages static analysis to perform a variety of correctness and style checks, encouraging students to improve their code quality. Each time a student commits code to their Git repository, our system triggers. Using KLC3 (KLEE), the student code is executed along with the gold version, and issues and behavioral differences are delivered back to students through their Git repository as a human-readable report, test cases, and scripts. A queueing system allows students to monitor progress, but the response is generally available within minutes. We also extended the LC-3 simulation tools to support reverse debugging, making the process of finding complex bugs much more tractable for students, and used Emscripten to develop a browser-based interface for use in testing and debugging. Finally, our tools maintain an individual regression test suite for each student and require a submission to pass all previous tests before re-evaluation in KLC3, thus avoiding encouraging programming-by-guesswork. We deployed the system to provide feedback for the assembly programming assignments in a class of over 100 students in Fall 2020. Students wrote a median of around 700 lines of assembly for these assignments, making heavy use of our tools to understand and eliminate their bugs. Anonymous student feedback on the tools was uniformly positive. Since that semester, we have continued to refine and expand our tools’ analysis capabilities and performance, and plan to deploy the system again in the near future (the class is offered every Fall).
Tue 16 NovDisplayed time zone: Hobart change
11:00 - 12:00 | AutomationResearch Papers / Tool Demonstrations / Journal-first Papers at Kangaroo Chair(s): Eunsuk Kang Carnegie Mellon University | ||
11:00 20mTalk | Automating User Notice Generation for Smart Contract Functions Research Papers Xing Hu Zhejiang University, Zhipeng Gao Monash University, Xin Xia Huawei Software Engineering Application Technology Lab, David Lo Singapore Management University, Xiaohu Yang Zhejiang University | ||
11:20 20mTalk | End-to-End Automation of Feedback on Student Assembly Programs Research Papers Zikai Liu ETH Zurich, Tingkai Liu UIUC, Qi Li Purdue University, Wenqing Luo UIUC, Steven S. Lumetta UIUC | ||
11:40 10mTalk | An automated model-based approach to repair test suites of evolving web applications Journal-first Papers Javaria Imtiaz National University of Computer and Emerging Sciences, Islamabad, Muhammad Zohaib Iqbal National University of Computer and Emerging Sciences, Muhammad Uzair Khan National University of Computer and Emerging Sciences | ||
11:50 5mTalk | BeAFix: An Automated Repair Tool for Faulty Alloy Models Tool Demonstrations Simón Gutiérrez Brida University of Rio Cuarto and CONICET, Argentina, Germán Regis Universidad Nacional de Río Cuarto, Guolong Zheng University of Nebraska Lincoln, Hamid Bagheri University of Nebraska-Lincoln, ThanhVu Nguyen George Mason University, Nazareno Aguirre University of Rio Cuarto and CONICET, Argentina, Marcelo F. Frias Dept. of Software Engineering Instituto Tecnológico de Buenos Aires | ||
11:55 5mTalk | FLACK: Localizing Faults in Alloy Models Tool Demonstrations Guolong Zheng University of Nebraska Lincoln, ThanhVu Nguyen George Mason University, Simón Gutiérrez Brida University of Rio Cuarto and CONICET, Argentina, Germán Regis Universidad Nacional de Río Cuarto, Marcelo F. Frias Dept. of Software Engineering Instituto Tecnológico de Buenos Aires, Nazareno Aguirre University of Rio Cuarto and CONICET, Argentina, Hamid Bagheri University of Nebraska-Lincoln |