Write a Blog >>
MSR 2022
Mon 23 - Tue 24 May 2022
co-located with ICSE 2022

Quantitative studies on the border between Mining Software Repository (MSR) and Empirical Software Engineering (ESE) apply data analysis methods, like regression modeling, statistic tests or correlation analysis, to commits or pulls to better understand the software development process. Such studies assure the validity of the reported results by following a sound methodology. However, with increasing complexity, parts of the methodology can still go wrong. This may result in MSR/ESE studies with undetected threats to validity. In this paper, we propose to systematically protect against threats by operationalizing their treatment using simulations. A simulation substitutes observed and unobserved data, related to an MSR/ESE scenario, with synthetic data, carefully defined according to plausible assumptions on the scenario. Within a simulation, unobserved data becomes transparent, which is the key difference to a real study, necessary to detect threats to an analysis methodology. Running an analysis methodology on synthetic data may detect basic technical bugs and misinterpretations, but it also improves the trust in the methodology. The contribution of a simulation is to operationalize testing the impact of important assumptions. Assumptions still need to be rated for plausibility. We evaluate simulation-based testing by operationalizing undetected threats in the context of four published MSR/ESE studies. We recommend that future research uses such more systematic treatment of threats, as a contribution against the reproducibility crisis.

Wed 18 May

Displayed time zone: Eastern Time (US & Canada) change

05:00 - 05:50
Session 3: Introspection, Vision, and Human Aspects Technical Papers / Data and Tool Showcase Track / Industry Track / Registered Reports at MSR Main room - odd hours
Chair(s): Alexander Serebrenik Eindhoven University of Technology, Sebastian Baltes SAP SE & University of Adelaide
05:00
4m
Short-paper
Geographic Diversity in Public Code Contributions
Technical Papers
Davide Rossi University of Bologna, Stefano Zacchiroli Télécom Paris, Polytechnic Institute of Paris
Pre-print Media Attached
05:04
7m
Talk
Operationalizing Threats to MSR Studies by Simulation-Based TestingDistinguished Paper Award
Technical Papers
Johannes Härtel University of Koblenz-Landau, Germany, Ralf Laemmel Facebook London
Pre-print Media Attached
05:11
4m
Talk
The General Index of Software Engineering Papers
Data and Tool Showcase Track
Zeinab Abou Khalil Inria, Stefano Zacchiroli Télécom Paris, Polytechnic Institute of Paris
DOI Pre-print
05:15
7m
Talk
Challenges and Future Research Direction for Microtask Programming in Industry
Industry Track
Masanari Kondo Kyushu University, Shinobu Saito NTT, IIMURA Yukako NTT, Eunjong Choi Kyoto Institute of Technology, Osamu Mizuno Kyoto Institute of Technology, Yasutaka Kamei Kyushu University, Naoyasu Ubayashi Kyushu University
DOI Pre-print Media Attached
05:22
7m
Talk
Starting the InnerSource Journey: Key Goals and Metrics to Measure Collaboration
Industry Track
Daniel Izquierdo-Cortazar Bitergia, Jesús Alonso-Gutiérrez Santander Bank, Alberto Pérez García-Plaza Bitergia, Gregorio Robles Universidad Rey Juan Carlos, Jesus M. Gonzalez-Barahona Universidad Rey Juan Carlos
Pre-print Media Attached
05:29
4m
Talk
Investigating the Impact of Forgetting in Software Development
Registered Reports
Utku Unal METU, Eray Tüzün Bilkent University, Tamer Gezici Bilkent University, Ausaf Ahmed Farooqui Bilkent University
Pre-print
05:33
17m
Live Q&A
Discussions and Q&A
Technical Papers

Tue 24 May

Displayed time zone: Eastern Time (US & Canada) change

11:00 - 12:15
Blended Technical Session 4 (Introspection, Vision, and Human Aspects)Technical Papers / Registered Reports / Data and Tool Showcase Track at Room 315+316
Chair(s): Ayushi Rastogi University of Groningen, The Netherlands
11:00
15m
Talk
Challenges in Migrating Imperative Deep Learning Programs to Graph Execution: An Empirical Study
Technical Papers
Tatiana Castro Vélez City University of New York (CUNY) Graduate Center, Raffi Khatchadourian City University of New York (CUNY) Hunter College, Mehdi Bagherzadeh Oakland University, Anita Raja City University of New York (CUNY) Hunter College
Pre-print Media Attached
11:15
15m
Talk
Operationalizing Threats to MSR Studies by Simulation-Based TestingDistinguished Paper Award
Technical Papers
Johannes Härtel University of Koblenz-Landau, Germany, Ralf Laemmel Facebook London
Pre-print Media Attached
11:30
8m
Short-paper
Geographic Diversity in Public Code Contributions
Technical Papers
Davide Rossi University of Bologna, Stefano Zacchiroli Télécom Paris, Polytechnic Institute of Paris
Pre-print Media Attached
11:38
8m
Talk
The General Index of Software Engineering Papers
Data and Tool Showcase Track
Zeinab Abou Khalil Inria, Stefano Zacchiroli Télécom Paris, Polytechnic Institute of Paris
DOI Pre-print
11:46
8m
Talk
Investigating the Impact of Forgetting in Software Development
Registered Reports
Utku Unal METU, Eray Tüzün Bilkent University, Tamer Gezici Bilkent University, Ausaf Ahmed Farooqui Bilkent University
Pre-print
11:54
21m
Live Q&A
Discussions and Q&A
Technical Papers


Information for Participants