Write a Blog >>

Jupyter notebooks are very popular, with nearly 10 million notebooks available on GitHub. They have gained popularity especially in educational settings. In France, it is one of the tools used by teachers in post-secondary classes to teach computer science. When students finish their work, they send their notebooks to the teacher for grading. However, it is likely that the teacher will not be able to reproduce the results contained in the notebooks. Indeed, students rely on the non-linearity of notebooks to write and execute their cells in an arbitrary order. Conversely, teachers are not aware of this implicit execution order and expect to reproduce the results by running the cells linearly from top to bottom. These two modes of execution conflict, making it difficult for teachers to evaluate the work of their students. In this paper, we present a new tool, the Notebook Reproducibility assistant, which provides immediate visual feedback to students on the reproducibility of their notebooks directly within Jupyter. The tool notifies the student as soon as the output of a cell is no longer reproducible when executing linearly from top to bottom, allowing them to solve the problem immediately. To evaluate the benefits of our tool, we perform a controlled study with 37 students, followed by a focus group. We show that the tool significantly improves the reproducibility of notebooks without interfering with the students’ work.

preprint (main.pdf)272KiB

Tue 12 Oct

Displayed time zone: Central Time (US & Canada) change

13:50 - 14:30
Understanding and Shortening the Feedback Loop in ProgrammingResearch Papers
Chair(s): Sandeep Kuttal The University of Tulsa
13:50
10m
Paper
Understanding the Challenges and Needs of Programmers Writing Web Automation ScriptsFull paper
Research Papers
Rebecca Krosnick University of Michigan, Steve Oney University of Michigan
14:00
10m
Paper
Edit-Run Behavior in Programming and DebuggingFull paper
Research Papers
Abdulaziz Alaboudi George Mason University, USA, Thomas LaToza George Mason University
14:10
10m
Short-paper
Immediate Feedback for Students to Solve Notebook Reproducibility Problems in the ClassroomShort paper
Research Papers
Christophe Casseau Univ. Bordeaux, Bordeaux INP, CNRS, LaBRI, UMR5800, Jean-Rémy Falleri Univ. Bordeaux, Bordeaux INP, CNRS, LaBRI. Institut Universitaire de France., Xavier Blanc Univ. Bordeaux, Bordeaux INP, CNRS, LaBRI, UMR5800, Thomas Degueule CNRS, LaBRI
File Attached
14:20
10m
Short-paper
Shortening Feedback Loops in a Live Game Development EnvironmentShort paper
Research Papers
Tom Beckmann Hasso Plattner Institute, Eva Krebs Hasso Plattner Institute, University of Potsdam, Patrick Rein Hasso Plattner Institute, Stefan Ramson Hasso Plattner Institute, Germany, Robert Hirschfeld Hasso Plattner Institute (HPI), University of Potsdam, Germany