EASE 2022
Mon 13 - Wed 15 June 2022 Göteborg, Sweden

The tremendous success of Deep Learning (DL) has significantly boosted the number of open-sourced DL frameworks hosted on GitHub. Among others, performance and accuracy bugs are critical factors that affect the reputation of these DL frameworks, therefore understanding the practice of discovering and investigating them for DL is important. In this paper, we conduct an exploratory study on the nature of reporting performance and accuracy bugs bugs for DL frameworks, aiming to improve our knowledge on this topic. Our study covers 10 most popular open-sourced DL frameworks on GitHub (e.g., TensorFlow, Keras, and PyTorch), based on which we sample 664 representative performance and accuracy bugs bug reports out of a total population of 22,522. Through systematic analysis of these samples, our key findings are: (1) low speed is the primary reason that a performance bug related report is submitted but we see no consistent pattern for accuracy related ones; (2) most of the reports are about issues encountered in the training stage; (3) only a small proportion of the reports provide insufficient information to investigate; (4) the majority of the performance and accuracy bugs bug reports (from 69% to 100%) are not related to the actual bug or regarded as unclassified; (5) around 50% of the performance and accuracy bug reports, which indeed reveal bugs, are not resolved by direct patches. Deriving from the above, we discuss a set of actionable implications to the researchers, maintainers, and report submitters on this subject. To promote open science, the labeled dataset has been made publicly available at https://zenodo.org/record/6371676.

Wed 15 Jun

Displayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

14:30 - 16:00
Research Track Virtual 2 Session 2Research at Wednesday Track 2 session 2 (Tesla and online)
Chair(s): Aleksander Fabijan Microsoft, Rebekka Wohlrab Carnegie Mellon University
14:30
30m
Talk
Do Developers Modify Dead Methods during the Maintenance of Java Desktop Applications?
Research
Pietro Cassieri University of Basilicata, Simone Romano University of Salerno, Giuseppe Scanniello University of Basilicata, Genoveffa Tortora , Danilo Caivano University of Bari
15:00
30m
Talk
Black-box Test Case Selection by Relating Code Changes with Previously Fixed Defects
Research
Tutku Çıngıl Ozyegin University, Hasan Sozer Ozyegin University
15:30
30m
Full-paper
On Reporting Performance and Accuracy Bugs for Deep Learning Frameworks: An Exploratory Study from GitHub
Research
Guoming Long Loughborough University, Tao Chen Loughborough University
Pre-print

Information for Participants
Wed 15 Jun 2022 14:30 - 16:00 at Wednesday Track 2 session 2 (Tesla and online) - Research Track Virtual 2 Session 2 Chair(s): Aleksander Fabijan, Rebekka Wohlrab
Info for room Wednesday Track 2 session 2 (Tesla and online):

Link to join: https://eu01web.zoom.us/j/61841934182?pwd=ZkpFVFhpTnZKcXhNbU4ydlUxRWl1Zz09