ReplicationsICPC 2019

Accepted Papers

Replications Track

The 27th IEEE/ACM International Conference on Program Comprehension (ICPC) will be hosting a replications track for the first time in 2019. The role of replication studies is crucial in program comprehension and software engineering research. Replications can either strengthen the results of the original study by increasing external validity with additional data or provide new insights into the variables that may impact the results.

Calls for contribution

This track provides a venue for researchers to submit replications of all types of empirical studies related to program comprehension (see list of topics in Technical Research Track), including controlled experiments, case studies, and surveys. Papers can report replications of the author’s own work or replications of another researcher’s work. If the experiments have been replicated and reproduced either partially or fully, the replications track is the right avenue to submit your findings. Please note that if your replication study produces negative results, you might also consider submitting to the Negative Results Track.

We invite submissions in two categories: 10 page (+ 2 pages for references) full papers and 5 page (+ 2 for references) short papers. For the 10 page submissions, a fully replicated study is expected to be described. For the 5 page submissions, works in progress or preliminary experimental designs for replication may be submitted with the clear intent of running the replication in the near future.

Each paper should clearly describe the original study and motivate the need for the replication. The paper should describe any changes to the original study design made during the replication, along with a justification for each change. The papers should contain a discussion section that compares the findings of the original and replication studies and describe the new knowledge gained from the replication along with any lessons learned from performing the replication. Partial replications are also welcome as long as the paper clearly states which parts of the study were replicated and which parts are new.

Format and Submission

Submissions must not be longer than 10 pages (for full papers) or 5 pages (for short papers) for the main text, inclusive of figures, tables, appendices; references only may be included on up to 2 additional pages.The submission must conform to the IEEE Conference Proceedings Formatting Guidelines (title in 24pt font and full text in 10pt type, LaTEX users must use \documentclass[10pt,conference]{IEEEtran} without including the compsoc or compsocconf option).

The submission must also comply with the ACM plagiarism policy and procedures. In particular, it must not have been published elsewhere and must not be under review elsewhere while under review for ICPC. The submission must also comply with the IEEE Policy on Authorship.

All submissions must be in PDF format and must be submitted online via the ICPC 2019 EasyChair conference management system. Be sure to select the Replications Track option.

Review and Evaluation Criteria

Research papers will be reviewed by at least three members of the Program Committee. Submissions will be evaluated on the basis of their originality, importance of contribution, soundness, evaluation, quality, and consistency of presentation, and appropriate comparison to related work and the work being replicated.

ICPC 2019 will employ a double-blind review process. This means that the submissions should by no means disclose the identity of the authors. The authors must make every effort to honor the double-blind review process. In particular, the authors’ names must be omitted from the submission and references to their prior work should be in the third person. Further advice, guidance and explanation about the double-blind review process can be found in the Q&A page.

ICPC 2019 will follow the ACM SIGSOFT rules on Conflicts of Interest and Confidentiality of Submissions, and all authors, reviewers, organizers are expected to uphold the ACM Code of Conduct.

Publication and Presentation

Upon notification of acceptance, all authors of accepted papers will receive further instructions for preparing their camera-ready versions. At least one author of the paper must register and present the paper at the conference; otherwise the paper will be excluded from both the program and the proceedings. All accepted papers will be published in the conference electronic proceedings, which will also be available in the ACM Digital Library. Purchases of additional pages in the proceedings is not allowed.

The papers will be presented in a formal setting. More details on the presentations will follow the notifications.

Award

Authors of one long and one short paper from the replications track will be recognized with a best replication award. Best replications may be invited to submit extended versions of their work to a special issue of Springer Journal of Empirical Software Engineering.

• Di Chen, Kathryn Stolee and Timothy Menzies. Replication Can Improve Prior Results: A GitHub Study of Pull Request Acceptance
• Jennifer Bauer, Janet Siegmund, Norman Peitek, Johannes Hofmeister and Sven Apel. Indentation: Simply a Matter of Style or Support for Program Comprehension?
• Eliane Wiese, Anna Rafferty, Daniel Kopta and Jacqulyn MacHardy. Replicating Novices’ Struggles with Coding Style
• Davide Fucci, Daniela Girardi, Nicole Novielli, Luigi Quaranta and Filippo Lanubile. A Replication Study on Code Comprehension and Expertise using Lightweight Biometric Sensors