Immediate Feedback for Students to Solve Notebook Reproducibility Problems in the ClassroomShort paper
Jupyter notebooks are very popular, with nearly 10 million notebooks available on GitHub. They have gained popularity especially in educational settings. In France, it is one of the tools used by teachers in post-secondary classes to teach computer science. When students finish their work, they send their notebooks to the teacher for grading. However, it is likely that the teacher will not be able to reproduce the results contained in the notebooks. Indeed, students rely on the non-linearity of notebooks to write and execute their cells in an arbitrary order. Conversely, teachers are not aware of this implicit execution order and expect to reproduce the results by running the cells linearly from top to bottom. These two modes of execution conflict, making it difficult for teachers to evaluate the work of their students. In this paper, we present a new tool, the Notebook Reproducibility assistant, which provides immediate visual feedback to students on the reproducibility of their notebooks directly within Jupyter. The tool notifies the student as soon as the output of a cell is no longer reproducible when executing linearly from top to bottom, allowing them to solve the problem immediately. To evaluate the benefits of our tool, we perform a controlled study with 37 students, followed by a focus group. We show that the tool significantly improves the reproducibility of notebooks without interfering with the students’ work.
preprint (main.pdf) | 272KiB |
Tue 12 OctDisplayed time zone: Central Time (US & Canada) change
13:50 - 14:30 | Understanding and Shortening the Feedback Loop in ProgrammingResearch Papers Chair(s): Sandeep Kuttal The University of Tulsa | ||
13:50 10mPaper | Understanding the Challenges and Needs of Programmers Writing Web Automation ScriptsFull paper Research Papers | ||
14:00 10mPaper | Edit-Run Behavior in Programming and DebuggingFull paper Research Papers | ||
14:10 10mShort-paper | Immediate Feedback for Students to Solve Notebook Reproducibility Problems in the ClassroomShort paper Research Papers Christophe Casseau Univ. Bordeaux, Bordeaux INP, CNRS, LaBRI, UMR5800, Jean-Rémy Falleri Univ. Bordeaux, Bordeaux INP, CNRS, LaBRI. Institut Universitaire de France., Xavier Blanc Univ. Bordeaux, Bordeaux INP, CNRS, LaBRI, UMR5800, Thomas Degueule CNRS, LaBRI File Attached | ||
14:20 10mShort-paper | Shortening Feedback Loops in a Live Game Development EnvironmentShort paper Research Papers Tom Beckmann Hasso Plattner Institute, Eva Krebs Hasso Plattner Institute, University of Potsdam, Patrick Rein Hasso Plattner Institute, Stefan Ramson Hasso Plattner Institute, Germany, Robert Hirschfeld Hasso Plattner Institute (HPI), University of Potsdam, Germany |