Replicability Study: Corpora For Understanding Simulink Models & Projects
Abstract—Background: Empirical studies on widely used model-based development tools such as MATLAB/Simulink are limited despite the tools’ importance in various industries.
Aims: The aim of this paper is to investigate the reproducibility of previous empirical studies that used Simulink model corpora and to evaluate the generalizability of their results to a newer and larger corpus, including a comparison with proprietary models.
Method: The study reviews methodologies and data sources employed in prior Simulink model studies and replicates the previous analysis using SLNET. In addition, we propose a heuristic for determining code-generating Simulink models and assess the open-source models’ similarity to proprietary models.
Results: Our analysis of SLNET confirms and contradicts earlier findings and highlights its potential as a valuable resource for model-based development research. We found that opensource Simulink models follow good modeling practices and contain models comparable in size and properties to proprietary models. We also collected and distribute 208 git repositories with over 9k commits, facilitating studies on model evolution.
Conclusions: The replication study offers actionable insights and lessons learned from the reproduction process, including valuable information on the generalizability of research findings based on earlier open-source corpora to the newer and larger SLNET corpus. The study sheds light on noteworthy attributes of SLNET, which is self-contained and redistributable.
Presentation Slide (Replicability ESEM-2.pptx) | 4.24MiB |
Fri 27 OctDisplayed time zone: Central Time (US & Canada) change
13:30 - 15:00 | 6B - Code quality and beyondESEM Journal-First Papers / ESEM Technical Papers at Rhythms 3 Chair(s): Valentina Lenarduzzi University of Oulu | ||
13:30 20mFull-paper | Replicability Study: Corpora For Understanding Simulink Models & Projects ESEM Technical Papers Sohil Lal Shrestha The University of Texas at Arlington, Shafiul Azam Chowdhury University of Texas at Arlington, Christoph Csallner University of Texas at Arlington Link to publication Pre-print Media Attached File Attached | ||
13:50 10mJournal Early-Feedback | A Decade of Code Comment Quality Assessment: A Systematic Literature Review ESEM Journal-First Papers Pooja Rani University of Zurich, Arianna Blasi Meta Platforms, Inc., Nataliia Stulova University of Bern, Switzerland, Sebastiano Panichella Zurich University of Applied Sciences, Alessandra Gorla IMDEA Software Institute, Oscar Nierstrasz University of Bern, Switzerland DOI Pre-print | ||
14:00 10mJournal Early-Feedback | Just-in-time Code Duplicates Extraction ESEM Journal-First Papers Eman Abdullah AlOmar Stevens Institute of Technology, Anton Ivanov HSE University, Zarina Kurbatova JetBrains Research, Yaroslav Golubev JetBrains Research, Mohamed Wiem Mkaouer Rochester Institute of Technology, Ali Ouni ETS Montreal, University of Quebec, Timofey Bryksin JetBrains Research, Le Nguyen Rochester Institute of Technology, Amit Kini Rochester Institute of Technology, Aditya Thakur Rochester Institute of Technology | ||
14:10 20mFull-paper | Beyond the Code: Investigating the Effects of Pull Request Conversations on Design Decay ESEM Technical Papers Caio Barbosa Pontifical Catholic University of Rio de Janeiro, Anderson Uchôa Federal University of Ceará, Daniel Coutinho PUC-Rio, Wesley Assunção North Carolina State University, Anderson Oliveira PUC-Rio, Alessandro Garcia PUC-Rio, Baldoino Fonseca , Matheus Feitosa de Oliveira Rabelo , José Eric Mesquita Coelho , Eryka Carvalho da Silva , Paulo Henrique Santos Marques | ||
14:30 10mJournal Early-Feedback | A Mixed-Method Approach to Recommend Corrections and Correct REST Antipatterns ESEM Journal-First Papers Fatima Sabir Punjab University College of Information Technology , University of the Punjab, Yann-Gaël Guéhéneuc Concordia University and Polytechnique Montréal, Francis Palma , Naouel Moha École de Technologie Supérieure (ETS), Ghulam Rasool , Hassan Akhtar | ||
14:40 10mJournal Early-Feedback | Tag that Issue: Applying API-domain Labels in Issue Tracking Systems ESEM Journal-First Papers Fabio Marcos De Abreu Santos Northern Arizona University, USA, Joseph Vargovich Northern Arizona University, Bianca Trinkenreich Oregon State University, USA, Italo Santos Northern Arizona University, Jacob Penney Northern Arizona University, Ricardo Britto Ericsson / Blekinge Institute of Technology, João Felipe Pimentel , Igor Wiese Federal University of Technology, Igor Steinmacher Northern Arizona University, Anita Sarma Oregon State University, Marco Gerosa Northern Arizona University | ||
14:50 10mJournal Early-Feedback | Do Names Echo Semantics? A large-scale Study of Identifiers used in C++'s Named Casts ESEM Journal-First Papers Constantin Cezar Petrescu University of Surrey, Sam Smith , Rafail Giavrimis Turing Intelligence Technology, Santanu Dash University of Surrey, UK Authorizer link |