Jupyter notebooks—documents that contain live code, equations, visualizations, and narrative text—now are among the most popular means to compute, present, discuss, and disseminate scientific findings. In principle, Jupyter notebooks should easily allow to reproduce and extend scientific computations and their findings; but in practice, this is not the case. The individual code cells in Jupyter notebooks can be executed in any order, with identifier usages preceding their definitions and results preceding their computations. In a sample of 936 published notebooks that would be executable in principle, we found that 73% of them would not be reproducible with straightforward approaches, requiring humans to infer (and often guess) the order in which the authors created the cells.
In this paper, we present an approach to
- automatically satisfy dependencies between code cells to reconstruct possible execution orders of the cells; and
- instrument code cells to mitigate the impact of non-reproducible statements (i.e., random functions) in Jupyter notebooks.
Our Osiris prototype takes a notebook as input and outputs the possible execution schemes that reproduce the exact notebook results. In our sample, Osiris was able to reconstruct such schemes for 82.23% of all executable notebooks, which has more than three times better than the state-of-the-art; the resulting reordered code is valid program code and thus available for further testing and analysis.
Tue 22 SepDisplayed time zone: (UTC) Coordinated Universal Time change
08:00 - 09:00 | Software Analysis (1)Research Papers at Kangaroo Chair(s): Michael Pradel University of Stuttgart, Germany | ||
08:00 20mTalk | Pending Constraints in Symbolic Execution for Better Exploration and Seeding Research Papers Timotej Kapus Imperial College London, UK, Frank Busse Imperial College London, Cristian Cadar Imperial College London, UK Pre-print Media Attached | ||
08:20 20mTalk | Broadening Horizons of Multilingual Static Analysis: Semantic Summary Extraction from C Code for JNI Program Analysis Research Papers | ||
08:40 20mTalk | Assessing and Restoring Reproducibility of Jupyter Notebooks Research Papers Jiawei Wang , Tzu-yang Kuo The Hong Kong University of Science and Technology, Li Li Monash University, Australia, Andreas Zeller CISPA, Germany |