This program is tentative and subject to change.
Tue 18 May Times are displayed in time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
10:10 - 10:50 | |||
10:10 10mPaper | Considerations and Pitfalls in Controlled Experiments on Code Comprehension Research Dror FeitelsonHebrew University Pre-print | ||
10:20 10mPaper | Let's Ask Students About Their Programs, Automatically Education Teemu LehtinenAalto University, André L. SantosUniversity Institute of Lisbon, Portugal, Juha SorvaAalto University Pre-print | ||
10:30 10mPaper | The effect of block-based formulas on formula comprehension in spreadsheets Research Pre-print | ||
10:40 10mPaper | Is Algorithm Comprehension Different from Program Comprehension? Education Philipp Kather, Jan VahrenholdDepartment of Computer Science, Westfälische Wilhelms-Universität Münster Pre-print |
11:00 - 11:30 | |||
11:00 10mPaper | Locating Faulty Methods with a Mixed RNN and Attention Model Research Shouliang YangSchool of Software, Shanghai Jiao Tong University, Junming CaoShanghai JiaoTong University, Hushuang ZengSchool of Software, Shanghai Jiao Tong University, Beijun ShenSchool of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University, Hao ZhongShanghai Jiao Tong University Pre-print | ||
11:10 10mPaper | EtherSolve: Computing an Accurate Control-Flow Graph from Ethereum Bytecode Research Filippo ControUniversità degli Studi di Verona, Marco CrosaraUniversity of Verona, Mariano CeccatoUniversity of Verona, Mila Dalla PredaUniversity of Verona, Italy Pre-print | ||
11:20 10mPaper | A systematic mapping study on architectural smells detection (JSS) Journal First Haris MumtazUniversity of Auckland, Paramvir SinghThe University of Auckland, Kelly BlincoeUniversity of Auckland |
11:30 - 12:00 | |||
18:00 - 19:00 | |||
Wed 19 May Times are displayed in time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
02:00 - 02:30 | |||
02:00 10mPaper | Weighing the Evidence: On Relationship Types in Microservice Extraction Research Lisa KirbyUniversity of British Columbia, Canada, Evelien Boerstra, Zachary John Christopher Anderson, Julia RubinUniversity of British Columbia, Canada Pre-print | ||
02:10 10mPaper | Comprehensive Integration of API Usage Patterns Research Pre-print | ||
02:20 10mPaper | Analyzing bug fix for automatic bug cause classification (JSS) Journal First Zhen Ni, Bin LiYangzhou University, Xiaobing SunYangzhou University, Tianhao Chen, Ben Tang, Xinchen Shi |
02:40 - 03:10 | Text Analysis for Program ComprehensionResearch / Replications and Negative Results (RENE) at ICPC Main Room | ||
02:40 10mPaper | Exploiting the Unique Expression for Improved Sentiment Analysis in Software Engineering Text Research Kexin Sun, Hui GaoNanjing University, Hongyu KuangNanjing University, Xiaoxing MaNanjing University, Guoping RongNanjing University, Dong ShaoNanjing University, He ZhangNanjing University Pre-print | ||
02:50 10mPaper | Predicting Community Smells’ Occurrence on Individual Developers by Sentiments Research Zijie HuangEast China University of Science and Technology, Zhiqing Shao, Guisheng Fan, Jianhua Gao, Ziyi Zhou, Kang Yang, Xingguang Yang Pre-print | ||
03:00 10mPaper | Checking App Behavior Against App Descriptions: What If There are No App Descriptions? Replications and Negative Results (RENE) Md. ShamsujjohaMonash University, Australia, John GrundyMonash University, Li LiMonash University, Hourieh KhalajzadehMonash University, Australia, Qinghua Lu Pre-print |
03:10 - 04:00 | |||
10:00 - 10:30 | |||
10:00 10mPaper | Improving deep-learning-based fault localization with resampling (JSEP) Journal First Zhuo ZhangPurdue University, Yan LeiSchool of Big Data & Software Engineering, Chongqing University, Xiaoguang MaoNational University of Defense Technology, Meng YanSchool of Big Data & Software Engineering, Chongqing University, Ling XuSchool of Big Data & Software Engineering, Chongqing University, Junhao Wen | ||
10:10 10mPaper | Imbalanced metric learning for crashing fault residence prediction (JSS) Journal First Zhou XuWuhan University, Kunsong ZhaoWuhan University, Meng Yan, Peipei Yuan, Ling XuSchool of Big Data & Software Engineering, Chongqing University, Yan LeiSchool of Big Data & Software Engineering, Chongqing University, Xiaohong ZhangChongqing University | ||
10:20 10mPaper | Adaptive Selection of Classifiers for Bug Prediction: A Large-Scale Empirical Analysis of Its Performances and a Benchmark Study (SCP) Journal First |
10:40 - 11:00 | |||
10:40 10mPaper | Understanding Architecture Erosion: The Practitioners' Perceptive Research Ruiyin LiWuhan University, China; University of Groningen, The Netherlands, Peng LiangWuhan University, Mohamed Soliman, Paris AvgeriouUniversity of Groningen, The Netherlands Pre-print Media Attached | ||
10:50 10mPaper | Understanding Code Smell Detection via Code Review: A Study of the OpenStack Community Research Xiaofeng HanWuhan University, China, Amjed TahirMassey University, Peng LiangWuhan University, Steve CounsellBrunel University London, Yajing Luo Pre-print Media Attached |
11:10 - 12:00 | |||
16:50 - 17:25 | Understanding Systems -2Research / Tool Demonstration / Replications and Negative Results (RENE) at ICPC Main Room | ||
16:50 5mDemonstration | ArCode: A Tool for Supporting Architectural Concerns Comprehension and Implementation Tool Demonstration Pre-print | ||
16:55 10mPaper | Assessing Semantic Frames to Support Program Comprehension Activities Research Arthur MarquesUniversity of British Columbia, Giovanni VivianiUniversity of British Columbia, Gail MurphyUniversity of British Columbia | ||
17:05 10mPaper | Warning-Introducing Commits vs Bug-Introducing Commits: A tool, statistical models, and a preliminary user study Replications and Negative Results (RENE) | ||
17:15 10mPaper | Project-Level Encoding for Neural Source Code Summarization of Subroutines Research Aakash BansalUniversity of Notre Dame, Sakib HaqueUniversity of Notre Dame, Collin McMillanUniversity of Notre Dame Pre-print |
17:25 - 17:50 | |||
17:50 - 18:10 | |||
17:50 10mPaper | Does Code Structure Affect Comprehension? On Using and Naming Intermediate Variables Research Pre-print | ||
18:00 10mPaper | Using Non-Verbal Expressions as a Tool in Naming Research Research Pre-print |
18:10 - 19:00 | |||
Thu 20 May Times are displayed in time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
02:00 - 02:30 | |||
02:00 10mPaper | Exploiting Method Names to Improve Code Summarization: A Deliberation Multi-Task Learning Approach Research Pre-print | ||
02:10 10mPaper | A Multi-Modal Transformer-based Code Summarization Approach for Smart Contracts Research Zhen YangCity University of Hong Kong, China, Jacky KeungCity University of Hong Kong, Xiao YuWuhan University of Technology, Xiaodong GuShanghai Jiao Tong University, China, Zhengyuan WeiCity University of Hong Kong, Hong Kong, Xiaoxue Ma, Miao ZHANGCity University of Hong Kong Pre-print | ||
02:20 10mPaper | Improving Code Summarization with Block-wise Abstract Syntax Tree Splitting Research Chen Lin, Zhichao Ouyang, Junqing Zhuang, Jianqiang Chen, Hui LiDepartment of Computer Science, Xiamen University, Rongxin WuDepartment of Cyber Space Security, Xiamen University Pre-print Media Attached |
02:40 - 03:00 | Source code AnalysisEarly Research Achievement (ERA) / Research / Tool Demonstration at ICPC Main Room | ||
02:40 5mDemonstration | RefactorHub: A Commit Annotator for Refactoring Tool Demonstration Ryo Kuramoto, Motoshi SaekiTokyo Institute of Technology, Shinpei HayashiTokyo Institute of Technology Pre-print Media Attached | ||
02:45 5mPaper | Detecting Inconsistent Thrown Exceptions Early Research Achievement (ERA) | ||
02:50 10mPaper | Where to Handle an Exception? Recommending Exception Handling Locations from a Global Perspective Research Xiangyang JiaSchool of Computer Science, Wuhan University, Songqiang ChenSchool of Computer Science, Wuhan University, Xingqi Zhou, Xintong Li, Run Yu, Xu Chen, Jifeng XuanWuhan University Pre-print |
03:00 - 04:00 | |||
10:50 - 11:30 | |||
10:50 10mPaper | Characterization and Prediction of Questions without Accepted Answers on Stack Overflow Research Pre-print | ||
11:00 10mPaper | Multi-Programming-Language Commits in OSS: An Empirical Study on Apache Projects Research Zengyang Li, Xiaoxiao Qi, Qinyi Yu, Peng LiangWuhan University, Ran MoDrexel University, Chen YangIBO Technology Co., Ltd Pre-print Media Attached | ||
11:10 10mPaper | Code Authors Hidden in File Revision Histories: An Empirical Study Research | ||
11:20 10mPaper | Predicting Crash Fault Residence via Simplified Deep Forest Based on A Reduced Feature Set Research Kunsong ZhaoWuhan University, Jin LiuWuhan University, Zhou XuChongqing University, China, Li LiMonash University, Meng YanChongqing University, Jiaojiao YuWuhan University, China, Yuxuan ZhouSyracuse University, USA Pre-print |
11:30 - 12:00 | |||
16:00 - 16:40 | Verification & ValidationResearch / Replications and Negative Results (RENE) / Early Research Achievement (ERA) at ICPC Main Room | ||
16:00 5mPaper | Understanding Execution Environment of File-Manipulation Scripts by Extracting Pre-Conditions Early Research Achievement (ERA) Rodney RodriguezUniversity of Texas at San Antonio, USA, Xiaoyin WangUniversity of Texas at San Antonio | ||
16:05 10mPaper | RAID: Tool Support for Refactoring-Aware Code Reviews Research Rodrigo BritoFederal University of Minas Gerais, Brazil, Marco Tulio ValenteFederal University of Minas Gerais, Brazil Pre-print | ||
16:15 5mPaper | Toward Automated Exploit Generation for Known Vulnerabilities in Open-Source Libraries Early Research Achievement (ERA) Emanuele IannoneUniversity of Salerno, Dario Di NucciTilburg University, Antonino SabettaSAP Security Research, Andrea De LuciaUniversity of Salerno Pre-print | ||
16:20 10mPaper | What is the Vocabulary of Flaky Tests? An Extended Replication Replications and Negative Results (RENE) Bruno Henrique Pachulski CamaraFederal University of Paraná, Marco Aurélio Graciotto SilvaFederal University of Technology - Paraná (UTFPR), André T. EndoFederal University of Technology - Paraná (UTFPR), Silvia Regina VergilioFederal University of Paraná Pre-print | ||
16:30 10mPaper | Shallow or Deep? An Empirical Study on Detecting Vulnerabilities using Deep Learning Research Alejandro Mazuera RozoUniversità della Svizzera italiana & Universidad de los Andes, Anamaria Mojica-HankeUniversidad de los Andes, Mario Linares-VásquezUniversidad de los Andes, Gabriele BavotaSoftware Institute, USI Università della Svizzera italiana Pre-print |
17:40 - 18:00 | |||
17:40 10mPaper | Atoms of Confusion in Java Research Pre-print | ||
17:50 10mPaper | Bug or not bug? That is the question Research Quentin Perez, Pierre-Antoine Jean, Christelle UrtadoEuroMov DHM, Univ Montpellier & IMT Mines Ales, Sylvain Vauttier Pre-print |
18:10 - 19:00 | |||
Fri 21 May Times are displayed in time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
02:00 - 02:40 | |||
02:00 10mPaper | ConfInLog: Leveraging Software Logs to Infer Configuration Constraints Research Shulin ZhouNational University of Defense Technology, Xiaodong LiuNational University of Defense Technology, Shanshan LiNational University of Defense Technology, Zhouyang JiaNational University of Defense Technology, Yuanliang ZhangNational University of Defense Technology, Teng WangNational University of Defense Technology, China, Wang Li, Xiangke LiaoNational University of Defense Technology, China Pre-print | ||
02:10 10mPaper | Using Grammar Patterns to Interpret Test Method Name Evolution Research Anthony PerumaRochester Institute of Technology, Emily Hu, Jiajun Chen, Eman Abdullah AlOmarRochester Institute of Technology, USA, Mohamed Wiem MkaouerRochester Institute of Technology, Christian D. NewmanRochester Institute of Technology Pre-print Media Attached | ||
02:20 10mPaper | Keywords Guided Method Name Generation Research Pre-print | ||
02:30 10mPaper | Automated Comment Update: How Far are We? Research Bo LinNational University of Defense Technology, Shangwen WangNational University of Defense Technology, Kui LiuNanjing University of Aeronautics and Astronautics, Xiaoguang MaoNational University of Defense Technology, Tegawendé F. BissyandéSnT, University of Luxembourg Pre-print Media Attached |
03:00 - 04:00 | |||
Call for Papers
The 29th IEEE/ACM International Conference on Program Comprehension (ICPC) will be hosting a REplications and NEgative results (RENE) track in 2021. The role of both replication and negative results studies is crucial in program comprehension and software engineering research. Replications can either strengthen the results of the original study by increasing external validity with additional data or provide new insights into the variables that may impact the results. Negative results contribute to scientific knowledge because by narrowing down the hypothesis space and by enabling critical insights and full understanding of existing approaches.
We seek replications and negative results papers for all types of program comprehension research areas (cognitive theories, visualization tools, comprehension of specific types of software systems, comprehension in the context of diverse software process models, etc.). Although authors of replication papers would choose any prior results in Program Comprehension research and replicate it, this year we encourage authors to replicate one of the papers in the list below. We compiled a list of papers addressing ‘classic’ program comprehension research questions, whose results were related to specific environments (e.g., programming language, technologies, styles, etc.) and their validity might have changed over time. Replications might lead to discovering new insights and effectively compare such new findings with respect to those achieved in the past. We strongly believe that this could foster discussion on how program comprehension has evolved over time, other than providing a mechanism to re-investigate aspects of program comprehension that could have not been studied for a long while. The papers to be replicated have been recommended by ICPC community members and their authors kindly agree to provide support to prospective authors as well as replication material.
Recommended papers to be replicated
In the following we provide the list of papers to be replicated, with link to the original paper, replication package (when available), and indication of the contact authors.
-
Susan Elliott Sim, Charles. L.A. Clarke, Richard C. Holt (1998), Archetypal Source Code Searches: A Survey of Software Developers and Maintainers. In Proceedings of IWPC’98, IEEE, DOI: 10.1109/WPC.1998.693351
Contact authors: Susan Elliott Sim (ses_at_drsusansim_dot_org), Charles. L.A. Clarke (claclark_at_gmail_dot_com).
The replication instrument is reported in Appendix A of the original paper. -
Thomas D. LaToza, David Garlan, James Herbsleb, Brad A. Myers (2007), Program comprehension as fact finding. In Proceedings of ESEC-FSE ’07, ACM, DOI: 10.1145/1287624.1287675.
Contact author: Thomas D. LaToza (tlatoza_at_gmu_dot_edu)
Replication package -
Amy J. Ko, Brad A. Myers (2008), Debugging reinvented: asking and answering why and why not questions about program behavior. In Proceedings of ICSE ’08, ACM, DOI: 10.1145/1368088.1368130.
Contact author: Amy J. Ko (ajko_at_uw_dot_edu)
Replication package -
Jonathan Sillito, Gail Murphy, Kris De Volder (2008), Asking and answering questions during a programming change task. IEEE Transactions on Software Engineering, DOI: 10.1109/TSE.2008.26.
Contact author: Jonathan Sillito (sillito_at_cs_dot_byu_dot_edu)
The replication package is not available, please refer to the paper for detailed information on the instrumentation and study design -
Valentina Grigoreanu, Margaret Burnett, Susan Wiedenbeck, Jill CAo, Kyle Rector, Irwin Kwan (2012), End-user debugging strategies: A sensemaking perspective. ACM Transactions on Computer-Human Interaction, ACM, DOI: 10.1145/2147783.2147788.
Contact author: Kyle Rector (kyle-rector_at_uiowa_dot_edu)
Replication package -
Zéphyrin Soh, Zohreh Sharafi, Bertrand Van den Plas, Gerardo C. Porras, Yann-Gaël Guéhéneuc, Giuliano Antoniol (2012), Professional status and expertise for UML class diagram comprehension: An empirical study. In Proceedings of ICPC 2012, IEEE, DOI: 10.1109/ICPC.2012.6240484.
Contact author: Yann-Gaël Guéhéneuc (yann-gael_dot_gueheneuc_at_concordia_dot_ca)
Replication package
Format and Submission
For replication studies, a paper should quickly summarize the contributions and the methodologies of the original study, explicitly state what claims of the original work are being replicated, what are the changes to the original experiment, if any, and whether the replication was successful or not. For negative results, the paper should explicitly state why the authors expect the claim to hold. We recommend the authors of RENE papers to include a paragraph/subsection specifically devoted to providing the aforementioned information (e.g., in the Introduction of the paper). Submissions must not be longer than 10 pages for the main text, inclusive of figures, tables, appendices. References only may be included on up to 2 additional pages. Purchases of additional pages in the proceedings are not allowed.
Submissions must conform to the IEEE formatting instructions IEEE Conference Proceedings Formatting Guidelines, (title in 24pt font and full text in 10pt type, LaTeX users must use \documentclass[10pt,conference]{IEEEtran}
without including the compsoc
or compsocconf
options.
The submissions must comply with the ACM Policy on Plagiarism, Misrepresentation, and Falsification. In particular, they must not have been published elsewhere and must not be under review elsewhere while under review for ICPC. The submission must also comply with the IEEE Policy on Authorship, part of the IEEE Plagiarism FAQ.
Submissions should not disclose the identity of the authors to comply with the double-blind review process employed by ICPC. The authors’ names must be omitted from the submissions and references to their prior work should be in the third person. Further advice, guidance, and explanation about the double-blind review process are available on the Q & A of ICSE 2021.
ICPC 2021 follows the ACM SIGSOFT rules on Conflicts of Interest and Confidentiality of Submissions and all authors, reviewers, and organizers will uphold the ACM Code of Ethics and Professional Conduct.
By submitting to the ICPC, authors acknowledge that they conform to the authorship policy of the ACM, and the authorship policy of the IEEE.
Submissions to the RENE Track that meet the above requirements can be made via the Easychair submission site (https://www.easychair.org/my/conference?conf=icpc2021) by the submission deadline. When submitting, select the track ‘ICPC 2021 Replication and Negative Results (RENE)’. We encourage the authors to upload their paper information early (and can submit the PDF later) to properly enter conflicts for double-blind reviewing. Any submission that does not comply with these requirements may be desk rejected by the RENE Track PC Chairs without further review.
Supplementary Material
Supplementary material can be uploaded via the EasyChair site or anonymously linked from the paper submission. Although PC members are not obligated to look at this material, we strongly encourage submitters to use supplementary material to provide access to anonymized code or data, whenever possible. Please carefully review any supplementary material to ensure it conforms to the double-blind policy (described above). For example, code and data repositories may be exported to remove version control history, scrubbed of names in comments and metadata, and anonymously uploaded to a sharing site to support review. One resource that may be helpful in accomplishing this task is this blog post by Daniel Graziotin: https://ineed.coffee/5205/how-to-disclose-data-for-double-blind-review-and-make-it-archived-open-data-upon-acceptance/.
Review and Evaluation Criteria
Papers will be reviewed by at least three members of the Program Committee. Submissions will be evaluated on the basis of their originality, importance of contribution, soundness, evaluation, quality, and consistency of presentation, and appropriate comparison to related work and the work being replicated. The authors of outstanding RENE papers will receive a Distinguished Paper Award.
Publication and Presentation
Upon notification of acceptance, all authors of accepted papers will receive further instructions for preparing the camera-ready versions of their submissions. If a submission is accepted, at least one author of the paper is required to register for ICPC 2021 and present the paper. All accepted papers will be published in the conference electronic proceedings, which will also be available in the IEEEXplore Library. The presentation is expected to be delivered in person, unless this is impossible due to travel limitations (related to, e.g., health, visa, or COVID-19 prevention). Details about the presentations will follow the notifications.
Accepted Papers
Title | |
---|---|
API2Com: On the Improvement of Automatically Generated Code Comments Using API Documentations Replications and Negative Results (RENE) Pre-print | |
Checking App Behavior Against App Descriptions: What If There are No App Descriptions? Replications and Negative Results (RENE) Pre-print | |
Warning-Introducing Commits vs Bug-Introducing Commits: A tool, statistical models, and a preliminary user study Replications and Negative Results (RENE) | |
What is the Vocabulary of Flaky Tests? An Extended Replication Replications and Negative Results (RENE) Bruno Henrique Pachulski Camara, Marco Aurélio Graciotto Silva, André T. Endo, Silvia Regina Vergilio Pre-print |