ICPC 2019
Sat 25 - Sun 26 May 2019 Montreal, QC, Canada
co-located with ICSE 2019

Accepted Papers


Replications Track

The 27th IEEE/ACM International Conference on Program Comprehension (ICPC) will be hosting a replications track for the first time in 2019. The role of replication studies is crucial in program comprehension and software engineering research. Replications can either strengthen the results of the original study by increasing external validity with additional data or provide new insights into the variables that may impact the results.

Calls for contribution

This track provides a venue for researchers to submit replications of all types of empirical studies related to program comprehension (see list of topics in Technical Research Track), including controlled experiments, case studies, and surveys. Papers can report replications of the author’s own work or replications of another researcher’s work. If the experiments have been replicated and reproduced either partially or fully, the replications track is the right avenue to submit your findings. Please note that if your replication study produces negative results, you might also consider submitting to the Negative Results Track.

We invite submissions in two categories: 10 page (+ 2 pages for references) full papers and 5 page (+ 2 for references) short papers. For the 10 page submissions, a fully replicated study is expected to be described. For the 5 page submissions, works in progress or preliminary experimental designs for replication may be submitted with the clear intent of running the replication in the near future.

Each paper should clearly describe the original study and motivate the need for the replication. The paper should describe any changes to the original study design made during the replication, along with a justification for each change. The papers should contain a discussion section that compares the findings of the original and replication studies and describe the new knowledge gained from the replication along with any lessons learned from performing the replication. Partial replications are also welcome as long as the paper clearly states which parts of the study were replicated and which parts are new.

Format and Submission

Submissions must not be longer than 10 pages (for full papers) or 5 pages (for short papers) for the main text, inclusive of figures, tables, appendices; references only may be included on up to 2 additional pages.The submission must conform to the IEEE Conference Proceedings Formatting Guidelines (title in 24pt font and full text in 10pt type, LaTEX users must use \documentclass[10pt,conference]{IEEEtran} without including the compsoc or compsocconf option).

The submission must also comply with the ACM plagiarism policy and procedures. In particular, it must not have been published elsewhere and must not be under review elsewhere while under review for ICPC. The submission must also comply with the IEEE Policy on Authorship.

All submissions must be in PDF format and must be submitted online via the ICPC 2019 EasyChair conference management system. Be sure to select the Replications Track option.

Review and Evaluation Criteria

Research papers will be reviewed by at least three members of the Program Committee. Submissions will be evaluated on the basis of their originality, importance of contribution, soundness, evaluation, quality, and consistency of presentation, and appropriate comparison to related work and the work being replicated.

ICPC 2019 will employ a double-blind review process. This means that the submissions should by no means disclose the identity of the authors. The authors must make every effort to honor the double-blind review process. In particular, the authors’ names must be omitted from the submission and references to their prior work should be in the third person. Further advice, guidance and explanation about the double-blind review process can be found in the Q&A page.

ICPC 2019 will follow the ACM SIGSOFT rules on Conflicts of Interest and Confidentiality of Submissions, and all authors, reviewers, organizers are expected to uphold the ACM Code of Conduct.

Publication and Presentation

Upon notification of acceptance, all authors of accepted papers will receive further instructions for preparing their camera-ready versions. At least one author of the paper must register and present the paper at the conference; otherwise the paper will be excluded from both the program and the proceedings. All accepted papers will be published in the conference electronic proceedings, which will also be available in the ACM Digital Library. Purchases of additional pages in the proceedings is not allowed.

The papers will be presented in a formal setting. More details on the presentations will follow the notifications.


Authors of one long and one short paper from the replications track will be recognized with a best replication award. Best replications may be invited to submit extended versions of their work to a special issue of Springer Journal of Empirical Software Engineering.


Sat 25 May

10:00 - 10:30: ICPC 2019 Presentations - Session I : Comprehending program comprehension I at Laurier
Chair(s): Dror FeitelsonHebrew University
icpc-2019-Technical-Research10:00 - 10:20
Sarah FakhouryWashington State University, Devjeet Roy, Sk. Adnan HassanInfosapex Limited, Venera ArnaoudovaWashington State University
icpc-2019-replications10:20 - 10:30
Eliane WieseUniversity of Utah, Anna RaffertyCarleton College, Daniel Kopta, Jacqulyn MacHardy
16:30 - 17:20: ICPC 2019 Presentations - Session V: Refactoring in Style at Laurier
Chair(s): Shinpei HayashiTokyo Institute of Technology
icpc-2019-replications16:30 - 16:50
Jennifer Bauer, Janet Siegmund, Norman PeitekLeibniz Institute for Neurobiology, Johannes Hofmeister, Sven ApelSaarland University
icpc-2019-Technical-Research16:50 - 17:00
Jianbo GaoPeking University, Han LiuTsinghua University, Zhiqiang YangTsinghua University, Chao Liu, Qingshan LiPeking University, Zhi GuanPeking University, Zhong Chen
icpc-2019-Technical-Research17:00 - 17:20
Ngoc Tran, Hieu TranThe University of Texas at Dallas, Son NguyenThe University of Texas at Dallas, Hoan Anh NguyenAmazon, Tien N. NguyenUniversity of Texas at Dallas

Sun 26 May

09:45 - 10:35: ICPC 2019 Presentations - Session VI : Tools for comprehension at Laurier
Chair(s): Alexandre BergelUniversity of Chile
icpc-2019-replications09:45 - 10:05
Di ChenNorth Carolina State University, USA, Kathryn StoleeNorth Carolina State University, Tim MenziesNorth Carolina State University
icpc-2019-Technical-Research10:05 - 10:15
Wasim SaidRobert Bosch GmbH, Jochen Quante, Rainer KoschkeUniversity of Bremen
icpc-2019-Technical-Research10:15 - 10:35
Gina BaiNorth Carolina State University, Brian Clee, Nischal Shrestha, Carl Chapman, Cimone Wright, Kathryn StoleeNorth Carolina State University
16:00 - 18:00: ICPC 2019 Presentations - Session IX: Information Retrieval, API, the Crowd, and Biosensors : The Magnificent Four at Laurier
Chair(s): Venera ArnaoudovaWashington State University
icpc-2019-replications16:00 - 16:20
Davide FucciUniversity of Hamburg, Daniela Girardi , Nicole NovielliUniversity of Bari, Luigi Quaranta, Filippo LanubileUniversity of Bari
icpc-2019-Technical-Research16:20 - 16:40
Xiaoqin FuWashington State University, Haipeng CaiWashington State University Pullman
Link to publication
icpc-2019-Technical-Research16:40 - 17:00
icpc-2019-Technical-Research17:00 - 17:20
Hussein AlrubayeRochester Institute of Technology, USA, Mohamed Wiem MkaouerRochester Institute of Technology, Ali Ouni
icpc-2019-Technical-Research17:20 - 17:40
Rodrigo Fernandes Gomes da Silva, Chanchal K. RoyUniversity of Saskatchewan, Masud RahmanUniversity of Saskatchewan , Kevin SchneiderUniversity of Saskatchewan, Klérisson PaixãoFederal University of Uberlândia, Marcelo De Almeida MaiaFederal University of Uberlandia
icpc-2019-Technical-Research17:40 - 18:00
Hongyu Kuang, Hui GaoNanjing University, China, Hao Hu, Xiaoxing Ma, Jian LuNanjing University, Patrick MäderTechnische Universität Ilmenau, Alexander Egyed
  • Di Chen, Kathryn Stolee and Timothy Menzies. Replication Can Improve Prior Results: A GitHub Study of Pull Request Acceptance
  • Jennifer Bauer, Janet Siegmund, Norman Peitek, Johannes Hofmeister and Sven Apel. Indentation: Simply a Matter of Style or Support for Program Comprehension?
  • Eliane Wiese, Anna Rafferty, Daniel Kopta and Jacqulyn MacHardy. Replicating Novices’ Struggles with Coding Style
  • Davide Fucci, Daniela Girardi, Nicole Novielli, Luigi Quaranta and Filippo Lanubile. A Replication Study on Code Comprehension and Expertise using Lightweight Biometric Sensors