Registered Reports TrackICSME 2024
Wed 9 OctDisplayed time zone: Arizona change
08:45 - 09:00 | ICSME Conference OpeningResearch Track / Industry Track / Tool Demo Track / New Ideas and Emerging Results Track / Registered Reports Track / Journal First Track / Artifact Evaluation Track and ROSE Festival at Humphreys Chair(s): Marco Gerosa Northern Arizona University, Igor Steinmacher Northern Arizona University, Fabio Calefato University of Bari, Sarah Nadi New York University Abu Dhabi, University of Alberta Welcome and opening remarks for the conference. | ||
09:00 - 10:00 | ICSME KeynoteResearch Track / Industry Track / Tool Demo Track / New Ideas and Emerging Results Track / Registered Reports Track / Journal First Track / Artifact Evaluation Track and ROSE Festival at Humphreys Chair(s): Fabio Calefato University of Bari | ||
09:00 60mKeynote | Maintaining Intelligence: Evolving Software Engineering Practices for AI-Enabled SystemsKeynote Research Track Foutse Khomh Polytechnique Montréal | ||
10:30 - 12:00 | Session 1: Code Understanding and OptimizationResearch Track / New Ideas and Emerging Results Track at Abineau Chair(s): Rosalia Tufano Università della Svizzera Italiana | ||
10:30 15m | Optimizing Decompiler Output by Eliminating Redundant Data Flow in Self-Recursive InliningResearch Track Paper Research Track Runze Zhang , Ying Cao Institute of Information Engineering at Chinese Academy of Sciences; University of Chinese Academy of Sciences, Ruigang Liang Institute of Information Engineering at Chinese Academy of Sciences; University of Chinese Academy of Sciences, Peiwei Hu , Kai Chen Institute of Information Engineering at Chinese Academy of Sciences; University of Chinese Academy of Sciences | ||
10:45 15m | Compilation of Commit Changes within Java Source Code RepositoriesOpen Research ObjectResearch Track Paper Research Track Stefan Schott Heinz Nixdorf Institut, Paderborn University, Wolfram Fischer SAP Security Research, Serena Elisa Ponta SAP Security Research, Jonas Klauke Heinz Nixdorf Institut, Paderborn University, Eric Bodden Pre-print | ||
11:00 15m | Understanding Code Change with Micro-ChangesResearch Track Paper Research Track Lei Chen Tokyo Institute of Technology, Michele Lanza Software Institute - USI, Lugano, Shinpei Hayashi Tokyo Institute of Technology DOI Pre-print Media Attached | ||
11:15 10m | What You Need is What You Get: Theory of Mind for an LLM-Based Code Understanding AssistantNIER Paper New Ideas and Emerging Results Track Pre-print | ||
11:25 15m | Decomposing God Header File via Multi-View Graph ClusteringResearch Track Paper Research Track Pre-print | ||
11:40 10m | How Far Have We Gone in Binary Code Understanding Using Large Language Models Research Track Xiuwei Shang University of Science and Technology of China, Shaoyin Cheng University of Science and Technology of China, Guoqiang Chen University of Science and Technology of China, Yanming Zhang , Li Hu , Xiao Yu , Gangyang Li , Weiming Zhang University of Science and Technology of China, Nenghai Yu Pre-print | ||
13:30 - 15:00 | Technical Briefing on srcML & srcDiff: Infrastructure to Support Exploring, Analyzing, and Differencing of Source CodeResearch Track at Humphreys This technology briefing is intended for those interested in constructing custom software analysis and manipulation tools to support research. The briefing is also aimed at researchers interested in leveraging syntactic differencing in their investigations. srcML (srcML.org) is an infrastructure consisting of an XML representation for C/C++/C#/Java source code along with efficient parsing technology to convert source code to-and-from the srcML format. srcDiff (srcDiff.org) is an infrastructure supporting syntactic source-code differencing and change analysis. srcDiff leverages srcML along with an efficient differencing algorithm to produce deltas that accurately model developer edits. In this tech briefing, we give an overview of srcML and srcDiff along with a tutorial of how to use them to support research efforts. The briefing is also a forum to seek feedback and input from the community on what new enhancements and features will better support software engineering research. | ||
17:00 - 18:00 | |||
Thu 10 OctDisplayed time zone: Arizona change
09:00 - 10:00 | ICSME KeynoteResearch Track / Industry Track / Tool Demo Track / New Ideas and Emerging Results Track / Registered Reports Track / Journal First Track / Artifact Evaluation Track and ROSE Festival at Humphreys Chair(s): Sarah Nadi New York University Abu Dhabi, University of Alberta | ||
09:00 60mKeynote | Disrupting Developer Dynamics: AI-Driven Innovations' Influence on CommunitiesKeynote Research Track Denae Ford Microsoft Research | ||
15:30 - 17:00 | Session 11: Mining Software RepositoriesTool Demo Track / Research Track / Registered Reports Track / New Ideas and Emerging Results Track at Fremont Chair(s): Gregorio Robles Universidad Rey Juan Carlos | ||
15:30 15m | “What Happened to my Models?” History-Aware Co-Existence and Co-Evolution of Metamodels and ModelsResearch Track Paper Research Track Marcel Homolka Institute for Software Systems Engineering, Johannes Kepler University, Linz, Luciano Marchezan Johannes Kepler University Linz, Wesley Assunção North Carolina State University, Alexander Egyed Johannes Kepler University Linz | ||
15:45 10m | MetaSim: A search engine for finding Similar GitHub RepositoriesTool Demo Paper Tool Demo Track Md Rayhanul Masud University of California, Riverside, Md Omar Faruk Rokon Sponsored Search, Walmart Global Tech, Qian Zhang University of California at Riverside, Michalis Faloutsos UCR Media Attached | ||
15:55 10m | SEART Data Hub: Streamlining Large-Scale Source Code Mining and Pre-ProcessingTool Demo Paper Tool Demo Track Ozren Dabic Software Institute, Università della Svizzera italiana (USI), Switzerland, Rosalia Tufano Università della Svizzera Italiana, Gabriele Bavota Software Institute @ Università della Svizzera Italiana | ||
16:05 10m | Diving into Software Evolution: Virtual Reality vs. On-ScreenRegistered Reports Paper Registered Reports Track David Moreno-Lumbreras Universidad Rey Juan Carlos, Jesus M. Gonzalez-Barahona Universidad Rey Juan Carlos, Gregorio Robles Universidad Rey Juan Carlos DOI Pre-print | ||
16:15 10m | Review-Pulse: A Dashboard for Managing User Feedback for Android ApplicationsTool Demo Paper Tool Demo Track Omar Adbealziz University of Saskatchewan, Zadia Codabux University of Saskatchewan, Kevin Schneider University of Saskatchewan | ||
16:25 10m | Monitoring Temporal Dynamics of Issues in Crowdsourced User Reviews and their Impact on Mobile App UpdatesNIER Paper New Ideas and Emerging Results Track Vitor Mesaque Alves de Lima Federal University of Mato Grosso do Sul, Jacson Rodrigues Barbosa Institute of Informatics (INF) / Federal University of Goiás (UFG), Ricardo Marcondes Marcacini University of São Paulo | ||
16:35 10m | Using Animations to Understand CommitsNIER Paper New Ideas and Emerging Results Track DOI Pre-print | ||
16:45 10m | Maven Unzipped: Packaging Impacts the Ecosystem Research Track Mehdi Keshani Delft University of Technology, Gideon Bot Delft University of Technology, Priyam Rungta , Maliheh Izadi Delft University of Technology, Arie van Deursen Delft University of Technology, Sebastian Proksch Delft University of Technology | ||
Fri 11 OctDisplayed time zone: Arizona change
08:45 - 10:00 | ICSME 2024 Awards CeremonyResearch Track / Industry Track / Tool Demo Track / New Ideas and Emerging Results Track / Registered Reports Track / Journal First Track / Artifact Evaluation Track and ROSE Festival at Humphreys Chair(s): Marco Gerosa Northern Arizona University, Igor Steinmacher Northern Arizona University, Fabio Calefato University of Bari, Sarah Nadi New York University Abu Dhabi, University of Alberta, Leon Moonen Simula Research Laboratory and BI Norwegian Business School | ||
08:45 35m | ICSME 2024 Awards Research Track | ||
09:20 40m | Most Influential Paper Talk: "An Exploratory Study on Self-Admitted Technical Debt" Research Track Link to publication DOI Pre-print | ||
10:30 - 12:00 | Session 12: Machine Learning in Software EngineeringTool Demo Track / Research Track / New Ideas and Emerging Results Track / Registered Reports Track at Abineau Chair(s): Mohammed Sayagh ETS Montreal, University of Quebec | ||
10:30 15m | Can We Do Better with What We Have Done? Unveiling the Potential of ML Pipeline in NotebooksResearch Track Paper Research Track | ||
10:45 10m | MergeRepair: Merging Task-Specific Adapters in Code LLMs for Automated Program RepairRegistered Reports Paper Registered Reports Track Meghdad Dehghan University of British Columbia, Jie JW Wu University of British Columbia (UBC), Fatemeh Hendijani Fard University of British Columbia, Ali Ouni ETS Montreal, University of Quebec Pre-print | ||
10:55 15m | On the Use of Deep Learning Models for Semantic Clone DetectionResearch Track Paper Research Track Subroto Nag Pinku University of Saskatchewan, Debajyoti Mondal , Chanchal K. Roy University of Saskatchewan, Canada | ||
11:10 10m | GlueTest: Testing Code Translation via Language InteroperabilityNIER Paper New Ideas and Emerging Results Track Muhammad Salman Abid Cornell University, Mrigank Pawagi Indian Institute of Science, Bengaluru, Sugam Adhikari Islington College, Xuyan Cheng Dickinson College, Ryed Badr University of Illinois Urbana Champaign, Md Wahiduzzaman BRAC University, Vedant Rathi Adlai E Stevenson High School, Ronghui Qi Wuhan University, Choiyin Li Po Leung Kuk Ngan Po Ling College, Lu Liu University of Washington, Rohit Sai Naidu Dublin High School, Licheng Lin Zhejiang University, Que Liu University of Shanghai for Science and Technology, Asif Zubayer Palak BRAC University, Mehzabin Haque University of Dhaka, Xinyu Chen University of Illinois Urbana Champaign, Darko Marinov University of Illinois at Urbana-Champaign, Saikat Dutta Cornell University | ||
11:20 10m | Does Co-Development with AI Assistants Lead to More Maintainable Code? A Registered ReportRegistered Reports Paper Registered Reports Track Markus Borg CodeScene, Dave Hewett Equal Experts, Donald Graham Equal Experts, Noric Couderc Lund University, Emma Söderberg Lund University, Luke Church University of Cambridge | Candela Inc, Dave Farley Continuous Delivery Pre-print | ||
11:30 15m | Leveraging Large Vision-Language Model For Better Automatic Web GUI TestingResearch Track Paper Research Track Siyi Wang , Sinan Wang Southern University of Science and Technology, Yujia Fan , Xiaolei Li , Yepang Liu Southern University of Science and Technology | ||
11:45 5m | StackRAG Agent: Improving Developer Answers with Retrieval-Augmented GenerationTool Demo Paper Tool Demo Track Davit Abrahamyan University of British Columbia, Fatemeh Hendijani Fard University of British Columbia | ||
17:00 - 17:30 | ICSME Conference ClosingResearch Track / Industry Track / Tool Demo Track / New Ideas and Emerging Results Track / Registered Reports Track / Journal First Track / Artifact Evaluation Track and ROSE Festival at Humphreys Join us for the closing plenary session to wrap up ICSME 2024 and take a peek at the 2025 edition. | ||
Accepted Papers
Author’s Guide
NB: Please contact the ICSME RR track chairs with any questions, feedback, or requests for clarification. Specific analysis approaches mentioned below are intended as examples, not mandatory components.
I. Title (required)
Provide the working title of your study. It may be the same title that you submit for publication of your final manuscript, but it is not mandatory.
Example: Should your family travel with you on the enterprise? Subtitle (optional): Effect of accompanying families on the work habits of crew members
II. Authors (required)
At this stage, we believe that a single anonymous review is most productive
III. Structured Abstract (required)
The abstract should describe the following in 200 words or so:
Background/Context What is your research about? Why are you doing this research, why is it interesting?
Example: “The enterprise is the flag ship of the federation, and it allows families to travel onboard. However, there are no studies that evaluate how this affects the crew members.”
Objective/Aim What exactly are you studying/investigating/evaluating? What are the objects of the study? We welcome both confirmatory and exploratory types of studies. Example (Confirmatory): We evaluate whether the frequency of sick days, work effectiveness and efficiency differ between science officers who bring their family with them, compared to science officers who are serving without their family.
Example (Exploratory): We investigate the problem of frequent Holodeck use in interpersonal relationships with an ethnographic study using participant observation, in order to derive specific hypotheses about Holodeck usage.
Method How are you addressing your objective? What data sources are you using?
Example: We conduct an observational study and use a between-subject design. To analyze the data, we use a t-test or Wilcoxon test, depending on the underlying distribution. Our data comes from computer monitoring of Enterprise crew members.
IV. Introduction
Give more details on the bigger picture of your study and how it contributes to this bigger picture. An important component of Stage 1 review is assessing the importance and relevance of the study questions, so be sure to explain this.
V. Hypotheses (required for confirmatory study) or research questions
Clearly state the research hypotheses that you want to test with your study, and a rationalization for the hypotheses.
Hypothesis example: Science officers with their family on board have more sick days than science officers without their family
Hypothesis rationale: Since toddlers are often sick, we can expect that crew members with their family on board need to take sick days more often.
VI. Variables (required for confirmatory study)
Independent Variable(s) and their operationalization
Dependent Variable(s) and their operationalization (e.g., time to solve a specified task)
Confounding Variable(s) and how their effect will be controlled (e.g., species type (Vulcan, Human, Tribble) might be a confounding factor; we control for it by separating our sample additionally into Human/Non-Human and using an ANOVA (normal distribution) or Friedman (non-normal distribution) to distill its effect).
For each variable, you should give: – name (e.g., presence of family) – abbreviation (if you intend to use one) – description (whether the family of the crew members travels on board) – scale type (nominal: either the family is present or not) – operationalization (crew members without family on board vs. crew members with family onboard)
VII. Participants/Subjects/Datasets (required)
Describe how and why you select the sample. When you conduct a meta-analysis, describe the primary studies / work on which you base your meta-analysis.
Example: We recruit crew members from the science department on a voluntary basis. They are our targeted population.
VIII. Execution Plan (required)
Describe the experimental setting and procedure. This includes the methods/tools that you plan to use (be specific on whether you developed it (and how) or whether it is already defined), and the concrete steps that you plan to take to support/reject the hypotheses or answer the research questions.
Example: Each crew member needs to sign the informed consent and agreement to process their data according to GDPR. Then, we conduct the interviews. Afterward, participants need to complete the simulated task …
Examples:
Confirmatory: https://osf.io/5fptj/ – Do Explicit Review Strategies Improve Code Review Performance?
Exploratory: https://osf.io/kfu9t – The Impact of Dynamics of Collaborative Software Engineering on Introverts: A Study Protocol
Study Protocol: https://osf.io/acnwk – Large-Scale Manual Validation of Bugfixing Changes
Call for Registrations
Empirical Software Engineering Journal (EMSE), in conjunction with the International Conference on Software Maintenance and Evolution (ICSME), is continuing the Registered Reports (RR) track.
The RR track of ICSME 2024 has two goals: (1) to prevent HARKing (hypothesizing after the results are known) for empirical studies; (2) to provide early feedback to authors in their initial study design. For papers submitted to the RR track, methods and proposed analyses are reviewed prior to execution. Pre-registered studies follow a two-step process:
-
Stage 1: A report is submitted that describes the planned study. The submitted report is evaluated by the reviewers of the RR track of ICSME 2024. Authors of accepted pre-registered studies will be given the opportunity to present their work at ICSME.
-
Stage 2: Once a report has passed Stage 1, the study will be conducted and actual data collection and analysis take place. The results may also be negative! The full paper is submitted for review to EMSE.
Paper Types, Evaluation Criteria, and Acceptance Types
The RR track of ICSME 2024 supports two types of papers:
- Confirmatory: The researchers have a fixed hypothesis (or several fixed hypotheses) and the objective of the study is to find out whether the hypothesis is supported by the facts/data. An example of a completed confirmatory study:
Inozemtseva, L., & Holmes, R. (2014, May). Coverage is not strongly correlated with test suite effectiveness. In Proceedings of the 36th International Conference on Software Engineering (pp. 435-445).
- Exploratory: The researchers do not have a hypothesis (or has one that may change during the study). Often, the objective of such a study is to understand what is observed and answer questions such as WHY, HOW, WHAT, WHO, or WHEN. We include in this category registrations for which the researchers have an initial proposed solution for an automated approach (e.g., a new generative AI approach to vulnerability detection) that serves as a starting point for their exploration to reach an effective solution. Examples of completed exploratory studies:
Gousios, G., Pinzger, M., & Deursen, A. V. (2014, May). An exploratory study of the pull-based software development model. In Proceedings of the 36th International Conference on Software Engineering (pp. 345-355). Rodrigues, I. M., Aloise, D., Fernandes, E. R., & Dagenais, M. (2020, June). A soft alignment model for bug deduplication. In Proceedings of the 17th International Conference on Mining Software Repositories (pp. 43-53).
Evaluation Criteria
The reviewers will evaluate RR track submissions based on the following criteria:
-
The importance of the research question(s).
-
The logic, rationale, and plausibility of the proposed hypotheses.
-
The soundness and feasibility of the methodology and analysis pipeline (including statistical power analysis where appropriate).
Evaluation Criteria Specific to Confirmatory Studies
- Whether the clarity and degree of methodological detail are sufficient to exactly replicate the proposed experimental procedures and analysis pipeline.
- Whether the authors have pre-specified sufficient outcome-neutral tests for ensuring that the results obtained can test the stated hypotheses, including positive controls and quality checks.
Evaluation Criteria Specific to Exploratory Studies
- (If applicable) The description of the data set that is the base for exploration.
Peer Review Outcomes
The outcome of the ICSME 2024 RR track review process is one of the following:
-
In-Principal Acceptance (IPA): The reviewers agree that the study is relevant, the outcome of the study (whether confirmation / rejection of hypothesis) is of interest to the community, the protocol for data collection is sound, and that the analysis methods are adequate. The authors can engage in the actual study for Stage 2. If the protocol is adhered to (or deviations are thoroughly justified), the study is published. Of course, this being a journal submission, a revision of the submitted manuscript may be necessary. Reviewers will especially evaluate how precisely the protocol of the accepted pre-registered report is followed, or whether deviations are justified.
-
Continuity Acceptance (CA): The reviewers agree that the study is relevant, that the (initial) methods appear to be appropriate. However, for exploratory studies, implementation details and post-experiment analyses or discussion (e.g., why the proposed automated approach does not work) may require follow-up checks. We’ll try our best to get the original reviewers. All PC members will be invited on the condition that they agree to review papers in both, Stage 1 and Stage 2. Four (4) PC members will review the Stage 1 submission, and three (3) will review the Stage 2 submission.
-
Rejection: The reviewers do not agree on the relevance of the study or are not convinced that the study design is sufficiently mature. Comments are provided to the authors to improve the study design before starting it.
For ICSME 2024, only confirmatory studies will be considered for IPA. Exploratory studies in software engineering often cannot be adequately assessed until after the study has been completed and the findings are elaborated and discussed in the full paper. For example, consider a study in an RR proposing vulnerability prediction using a new generative AI approach. This work falls under the exploratory category. It is difficult to offer IPA, as we do not know whether it is any better than a traditional approach based on, e.g., static code analysis. Negative results are welcome; however, it is important that the negative results paper goes beyond presenting “we tried and failed”, but rather provide deeper insights to readers, e.g., why the results are negative or what that means for further studies on this topic following criteria of REplication and Negative Results (RENE) tracks. Furthermore, it is important to note that authors are required to document all deviations (if any) in a section of the paper.
Submission Format and Instructions
- Submissions to the ICSME 2024 RR track must not exceed 6 pages (plus 1 additional page of references). The page limit is strict. All submissions must be in PDF and must be submitted online by the deadline via the ICSME 2024 EasyChair link.
- Submissions must strictly adhere to the two-column IEEE conference proceedings format. Please use the templates available here. LaTeX users should use the following configuration:
\documentclass[conference]{IEEEtran}. Microsoft Word users should use the US Letter format template. - All authors, reviewers, and organizers are expected to uphold the IEEE Code of Conduct.
- Any submission that does not comply with the instructions in this document or its referenced materials will be desk rejected. In addition, by submitting, the authors acknowledge that they are aware of and agree to be bound by the IEEE plagiarism policy. In particular, papers submitted to ICSME 2024 must not have been published elsewhere and must not be under review or submitted for review elsewhere whilst under consideration for ICSME 2024. Contravention of this concurrent submission policy will be deemed a serious breach of scientific ethics, and appropriate action will be taken in all such cases (including immediate rejection and reporting of the incident to IEEE). To check for double submission and plagiarism issues, the chairs reserve the right to (1) share the list of submissions with the PC Chairs of other conferences with overlapping review periods and (2) use external plagiarism detection software, under contract to the IEEE, to detect violations of these policies.
Submission Process
The timeline for ICSME 2024 RR track will be as follows:
May 31: Authors submit their initial report.
July 5: Authors receive PC members’ reviews.
July 26: Authors submit a response letter + revised report in a single PDF.
-
The response letter should address reviewer comments and questions.
-
The response letter + revised report must not exceed 12 pages (plus 1 additional page of references).
-
The response letter does not need to follow IEEE formatting instructions.
August 16: Notification of Stage 1. The outcome will be acceptance (CA/IPA) or rejection.
August 23: Authors submit their accepted RR report to arXiv. To be checked by PC members for Stage 2.
Note: Accepted RR track reports will not be published in the ICSME 2024 proceedings. However, the authors will present their RR at the conference.
Before May 30, 2025: Authors submit a full paper to EMSE. Instructions will be provided later. However, the following constraints will be enforced:
-
Justifications need to be provided for any deviation from the pre-registered report. If the authors are added/removed or the author order is changed between the original Stage 1 and the EMSE submission, all authors will need to complete and sign a “Change of authorship request form”. The Editors in Chief of EMSE and chairs of the RR track reserve the right to deny author changes. If you anticipate any authorship changes please reach out to the chairs of the RR track as early as possible.
-
PC members who reviewed an RR report in Stage 1 and their directly supervised students cannot be added as authors of the corresponding submission in Stage 2.
| Fri 23 Aug 2024 Authors submit their accepted RR report to arXiv |
| Fri 16 Aug 2024 Notification of Stage 1 |
| Fri 26 Jul 2024 Authors submit a response letter + revised report in a single PDF |
| Fri 5 Jul 2024 Authors receive PC members’ reviews |
| Fri 31 May 2024 Authors submit their initial report |
Shinpei HayashiCo-chair
Tokyo Institute of Technology
Japan
Shane McIntoshCo-chair
University of Waterloo
Canada
Eman Abdullah AlOmarCommittee Member
Stevens Institute of Technology
United States
Mahmoud AlfadelCommittee Member
University of Calgary
Canada
Tapajit DeyCommittee Member
Carnegie Mellon University Software Engineering Institute
United States
Daniel FeitosaCommittee Member
University of Groningen
Netherlands
Steffen HerboldCommittee Member
University of Passau
Germany
Ben HermannCommittee Member
TU Dortmund
Germany
Shin HongCommittee Member
Chungbuk National University
South Korea
Xianhao JinCommittee Member
ByteDance
Valentina LenarduzziCommittee Member
University of Oulu
Finland
Yepang LiuCommittee Member
Southern University of Science and Technology
China
Henry MucciniCommittee Member
University of L'Aquila, Italy
Italy
Anthony PerumaCommittee Member
University of Hawai‘i at Mānoa
United States
Kazumasa ShimariCommittee Member
Nara Institute of Science and Technology
Japan
Yuan TianCommittee Member
Queen's University, Kingston, Ontario
Canada
Shaowei WangCommittee Member
University of Manitoba
Canada
Marvin WyrichCommittee Member
Saarland University
Germany