ICPC 2023 - Replications and Negative Results (RENE)

Time Zone

The program is currently displayed in (GMT+10:00) Hobart.

Use conference time zone: (GMT+10:00) HobartSelect other time zone

The GMT offsets shown reflect the offsets at the moment of the conference.

Time Band

By setting a time band, the program will dim events that are outside this time window. This is useful for (virtual) conferences with a continuous program (with repeated sessions).
The time band will also limit the events that are included in the personal iCalendar subscription service.

Display full programSpecify a time band

Save

You're viewing the program in a time zone which is different from your device's time zone change time zone

Mon 15 May
Displayed time zone: Hobart change

11:00 - 12:30	Keynote / Documentation and Stack OverflowTool Demonstration / Research / ICPC Keynotes / Replications and Negative Results (RENE) / Discussion at Meeting Room 106 Chair(s): Bonita Sharif University of Nebraska-Lincoln, USA, Raula Gaikovina Kula Nara Institute of Science and Technology, Chanchal K. Roy University of Saskatchewan

11:00 45m Keynote		April Wensel: Applications of Emotional Intelligence in Program Comprehension ICPC Keynotes
11:45 9m Full-paper		APIContext2Com: Code Comment Generation by Incorporating Pre-Defined API Documentation Research Ramin Shahbazi , Fatemeh Hendijani Fard University of British Columbia Pre-print
11:54 5m Short-paper		PyVerDetector: A Chrome Extension Detecting the Python Version of Stack Overflow Code Snippets Tool Demonstration SHIYU YANG , Tetsuya Kanda Osaka University, Davide Pizzolotto Osaka University, Daniel M. German University of Victoria, Yoshiki Higo Osaka University
11:59 5m Short-paper		RCGraph - A Tool to Integrate Readme and Commits through Temporal Knowledge Graphs Tool Demonstration Akhila Sri Manasa Venigalla IIT Tirupati, Mir Sameed Ali Indian Institute of Technology Tirupati, Nikhil Manjunath Indian Institute of Technology Tirupati, Sridhar Chimalakonda IIT Tirupati
12:04 9m Full-paper		Performance Prediction From Source Code Is Task and Domain Specific Replications and Negative Results (RENE) Markus Böck TU Wien, Sarra Habchi Ubisoft, Mathieu Nayrolles Ubisoft Montreal, Jürgen Cito TU Wien
12:13 17m Panel		Discussion 2 Discussion

15:45 - 17:15	Code Summarization and VisualizationReplications and Negative Results (RENE) / Discussion / Research at Meeting Room 106 Chair(s): Banani Roy University of Saskatchewan, Akhila Sri Manasa Venigalla IIT Tirupati

15:45 9m Full-paper		An Extensive Study of the Structure Features in Transformer-based Code Semantic Summarization Research Kang Yang , Xinjun Mao National University of Defense Technology, Shangwen Wang National University of Defense Technology, Yihao Qin National University of Defense Technology, Yao Lu National University of Defense Technology, Tanghaoran Zhang , Kamal Al-Sabahi University Of Technology and Applied Sciences-ibra Pre-print
15:54 9m Full-paper		Label Smoothing Improves Neural Source Code Summarization Research Sakib Haque University of Notre Dame, Aakash Bansal University of Notre Dame, Collin McMillan University of Notre Dame Pre-print
16:03 9m Full-paper		Interpretation-based Code Summarization Research Mingyang Geng National University of Defense Technology, Shangwen Wang National University of Defense Technology, Dezun Dong NUDT, Haotian Wang National University of Defense Technolog, Shaomeng Cao Peng Cheng Laboratory, Kechi Zhang Peking University, China, Zhi Jin Peking University Pre-print
16:12 9m Full-paper		Naturalness in Source Code Summarization. How Significant is it? Replications and Negative Results (RENE) Claudio Ferretti University of Milano-Bicocca, Martina Saletta University of Milano-Bicocca
16:21 9m Full-paper		Comparing 2D and Augmented Reality Visualizations for Microservice System Understandability: A Controlled Experiment Research Amr Elsayed Baylor University, Tomas Cerny Baylor University, Davide Taibi Tampere University , Sira Vegas Universidad Politecnica de Madrid DOI Pre-print
16:30 9m Full-paper		ChameleonIDE: Untangling Type Errors Through Interactive Visualization and Exploration Research Shuai Fu Monash University, Tim Dwyer Monash University, Peter J. Stuckey Monash University, Jackson Wain Monash University, Jesse Linossier Monash University Pre-print
16:39 36m Panel		Discussion 4 Discussion

Tue 16 May
Displayed time zone: Hobart change

13:45 - 15:15	Programming Languages, Types, and ComplexityDiscussion / Research / Replications and Negative Results (RENE) / Journal First at Meeting Room 106 Chair(s): Vittoria Nardone

13:45 9m Full-paper		How Well Static Type Checkers Work with Gradual Typing? A Case Study on Python Research Wenjie Xu Nanjing University, Lin Chen Nanjing University, Chenghao Su Nanjing University, Yimeng Guo Nanjing University, Yanhui Li Nanjing University, Yuming Zhou Nanjing University, Baowen Xu Nanjing University
13:54 9m Full-paper		Too Simple? Notions of Task Complexity used in Maintenance-based Studies of Programming Tools Research Patrick Rein University of Potsdam; Hasso Plattner Institute, Tom Beckmann Hasso Plattner Institute, Eva Krebs Hasso Plattner Institute (HPI), University of Potsdam, Germany, Toni Mattis University of Potsdam; Hasso Plattner Institute, Robert Hirschfeld University of Potsdam; Hasso Plattner Institute
14:03 9m Full-paper		Path Complexity Predicts Code Comprehension Effort Research Sofiane Dissem Harvey Mudd College, Eli Pregerson Harvey Mudd College, Adi Bhargava Harvey Mudd College, Josh Cordova Harvey Mudd College, Lucas Bang Harvey Mudd College
14:12 5m Short-paper		Revisiting Deep Learning for Variable Type Recovery Replications and Negative Results (RENE) Kevin Cao Vanderbilt University, Kevin Leach Vanderbilt University Pre-print
14:17 9m Talk		Programming language implementations for context-oriented self-adaptive systems Journal First Nicolás Cardozo Universidad de los Andes, Kim Mens Université catholique de Louvain, ICTEAM institute, Belgium Link to publication DOI Media Attached
14:26 9m Full-paper		Improving Code Search with Multi-Modal Momentum Contrastive Learning Research Zejian Shi Fudan University, Yun Xiong Fudan University, Yao Zhang Fudan University, Zhijie Jiang National University of Defense Technology, Jinjing Zhao National Key Laboratory of Science and Technology on Information System Security, Lei Wang National University of Defense Technology, Shanshan Li National University of Defense Technology Pre-print
14:35 9m Full-paper		Revisiting Lightweight Compiler Provenance Recovery on ARM Binaries Replications and Negative Results (RENE) Jason Kim Georgia Tech, Daniel Genkin Georgia Tech, Kevin Leach Vanderbilt University Pre-print
14:44 31m Panel		Discussion 7 Discussion

Accepted Papers

	Title
	Naturalness in Source Code Summarization. How Significant is it? Replications and Negative Results (RENE) Claudio Ferretti, Martina Saletta
	Performance Prediction From Source Code Is Task and Domain Specific Replications and Negative Results (RENE) Markus Böck, Sarra Habchi, Mathieu Nayrolles, Jürgen Cito
	Revisiting Deep Learning for Variable Type Recovery Replications and Negative Results (RENE) Kevin Cao, Kevin Leach Pre-print
	Revisiting Lightweight Compiler Provenance Recovery on ARM Binaries Replications and Negative Results (RENE) Jason Kim, Daniel Genkin, Kevin Leach Pre-print

Call for Papers

The 31st edition of the International Conference on Program Comprehension (ICPC’23) would like to encourage researchers to (1) reproduce results from previous papers and (2) publish studies with important and relevant negative or null results (results which fail to show an effect, yet demonstrate the research paths that did not pay off).

We would also like to encourage the publication of the negative results or reproducible aspects of previously published work. For example, authors of a published paper reporting a working solution for a given problem can document in a “negative results paper” other (failed) attempts they made before defining the working solution they published.

Reproducibility studies. Inspired by ISSTA’18 Reproducibility studies, the papers in this category must go beyond simply reimplementing an algorithm and/or re-running the artifacts provided by the original paper. Such submissions should at least apply the approach on new data sets (open-source or proprietary). A reproducibility study should clearly report on results that the authors were able to reproduce as well as on the aspects of the work that were irreproducible. We encourage reproducibility studies to follow the ACM guidelines on reproducibility (different team, different experimental setup): “The measurement can be obtained with stated precision by a different team, a different measuring system, in a different location on multiple trials. For computational experiments, this means that an independent group can obtain the same result using artifacts which they develop completely independently.”
Negative results papers. We seek papers that report on negative results. We seek negative results for all types of software engineering research in any empirical area (qualitative, quantitative, case study, experiment, etc.). For example, did your controlled experiment not show an improvement over the baseline? Even if negative, results obtained are still valuable when they are either not obvious or disprove widely accepted wisdom. As Walter Tichy writes, “Negative results, if trustworthy, are extremely important for narrowing down the search space. They eliminate useless hypotheses and thus reorient and speed up the search for better approaches.”

Evaluation Criteria

Both Reproducibility Studies and Negative Results submissions will be evaluated according to the following standards:

Depth and breadth of the empirical studies
Clarity of writing
Appropriateness of conclusions
Amount of useful, actionable insights
Amount of useful, actionable insights
Availability of artifacts
Underlying methodological rigor. A negative result due primarily to misaligned expectations or due to lack of statistical power (small samples) is not a good submission. The negative result should be a result of a lack of effect, not lack of methodological rigor.

Most importantly, we expect reproducibility studies to clearly point out the artifacts the study is built upon, and to provide the links to all the artifacts in the submission (the only exception will be given to those papers that reproduce the results on proprietary datasets that can not be publicly released).

Submission Instructions

Submissions must be original, in the sense that the findings and writing have not been previously published or under consideration elsewhere. However, as either reproducibility studies or negative results, some overlap with previous work is expected. Please make that clear in the paper.

Publication format should follow the ICPC guidelines. Submissions to the RENE Track can be made via the ICPC RENE track submission site by the submission deadline.

Length: There are two formats. (1) New reproducibility studies and new descriptions of negative results will have a length of 10 pages, plus 2 pages which may only contain references. (2) Appendices to conference submissions or previous work by the authors can be described in 4 pages, plus 1 page which may only contain references (e.g., as previously said, authors of a published paper can document negative results they got while working on it, such as solutions that did not work).

Important note: the RENE track of ICPC 2023 does not follow a double-anonymous review process.

The official publication date is the date the proceedings are made available in the ACM or IEEE Digital Libraries. This date may be up to two weeks prior to the first day of ICSE 2023. The official publication date affects the deadline for any patent filings related to published work.

Purchases of additional pages in the proceedings is not allowed.

09:00 - 10:30	Conference Introductions / MIP Talk / Documentation and Stack OverflowDiscussion / Research / Opening / Journal First / MIP Talk at Meeting Room 106 Chair(s): Christoph Treude University of Melbourne, Akhila Sri Manasa Venigalla IIT Tirupati

09:00 15m Day opening		Conference Opening Opening G: Christoph Treude University of Melbourne, P: Raula Gaikovina Kula Nara Institute of Science and Technology, P: Bonita Sharif University of Nebraska-Lincoln, USA
09:15 40m Talk		MIP Talk on ICPC 2013 Paper titled "Automatic generation of natural language summaries for Java classes" MIP Talk Laura Moreno CQSE America, Jairo Aponte Universidad Nacional de Colombia, Giriprasad Sridhara IBM Research Labs, Andrian Marcus University of Texas at Dallas, Lori Pollock University of Delaware, USA, K. Vijay-Shanker
09:55 9m Full-paper		QTC4SO: Automatic Question Title Completion for Stack Overflow Research Yanlin Zhou School of Information Science and Technology, Nantong University, ShaoYu Yang School of Information Science and Technology, Nantong University, Xiang Chen Nantong University, Zichen Zhang School of Information Science and Technology, Nantong University, Jiahua Pei School of Information Science and Technology, Nantong University Pre-print
10:04 9m Talk		A Study of Update Request Comments in Stack Overflow Answer Posts Journal First Mohammad Sadegh Sheikhaei School of Computing, Queen's University, Yuan Tian Queens University, Kingston, Canada, Shaowei Wang University of Manitoba Link to publication
10:13 9m Talk		Machine Translation-based Fine-grained Comments Generation for Solidity Smart Contracts Journal First Chaochen Shi Deakin University, Yong Xiang Deakin University, Jiangshan Yu Monash University, Keshav Sood Deakin University, Longxiang Gao Qilu University of Technology
10:22 8m Panel		Discussion 1 Discussion

13:45 - 15:15	Human Aspects, Testing and LogsTool Demonstration / Discussion / Journal First / Early Research Achievements (ERA) / Research at Meeting Room 106 Chair(s): Michael J. Decker Bowling Green State University

13:45 9m Full-paper		Understanding initial API comprehension Research Ava Heinonen Aalto University, Fabian Fagerholm Aalto University
13:54 5m Short-paper		Evaluating a Language Workbench: from Working Memory Capacity to Comprehension to Acceptance Early Research Achievements (ERA) Giovanna Broccia ISTI-CNR, FMT Lab, Alessio Ferrari CNR-ISTI, Maurice ter Beek ISTI-CNR, Pisa, Italy, Walter Cazzola Università degli Studi di Milano, Luca Favali University of Milan, Francesco Bertolotti
13:59 5m Short-paper		Conversation Disentanglement As-a-Service Tool Demonstration Edoardo Riggio Software Institute - USI, Lugano, Marco Raglianti Software Institute - USI, Lugano, Michele Lanza Software Institute - USI, Lugano
14:04 5m Short-paper		Slicito: Using Computational Notebooks for Program Comprehension Tool Demonstration Robert Husak , Jan Kofroň Charles University, Filip Zavoral
14:09 9m Talk		Selection of human evaluators for design smell detection using dragonfly optimization algorithm: An empirical study Journal First Sultan M. Al Khatib Department of Software Engineering, Prince Abdullah bin Ghazi Faculty of Information and Communication Technology, Al-Balqa Applied University (BAU), Al-Salt, 19117, Jordan, Khalid Alkharabsheh Department of Software Engineering, Prince Abdullah bin Ghazi Faculty of Information and Communication Technology, Al-Balqa Applied University (BAU), Al-Salt, 19117, Jordan, Sadi Alawadi Center for Applied Intelligent Systems Research, School of Information Technology, Halmstad University, 30118, Halmstad, Sweden
14:18 5m Short-paper		SYN: Ultra-Scale Software Evolution Comprehension Tool Demonstration Gianlorenzo Occhipinti Software Institute, USI - Lugano, Switzerland, Csaba Nagy Software Institute - USI, Lugano, Roberto Minelli Software Institute - USI, Lugano, Michele Lanza Software Institute - USI, Lugano
14:23 5m Short-paper		Microusity: A testing tool for Backends for Frontends (BFF) Microservice Systems Tool Demonstration Pattarakrit Rattanukul Mahidol University, Chansida Makaranond Mahidol University, Pumipat Watanakulcharus Mahidol University, Chaiyong Ragkhitwetsagul Mahidol University, Thailand, Tanapol Nearunchorn Lineman Wongnai, Vasaka Visoottiviseth Mahidol University, Morakot Choetkiertikul Mahidol University, Thailand, Thanwadee Sunetnanta Mahidol University
14:28 5m Short-paper		WebEV: A Dataset on the Behavior of Testers for Web Application End to End Testing Early Research Achievements (ERA) Fuad Mridha University of Dhaka, Kazi Sakib Institute of Information Technology, University of Dhaka
14:33 5m Short-paper		Towards a Classification of Log Parsing Errors Early Research Achievements (ERA) Issam Sedki Concordia University, Wahab Hamou-Lhadj Concordia University, Montreal, Canada, Otmane Ait-Mohamed Concordia University, Naser Ezzati Jivan
14:38 37m Panel		Discussion 3 Discussion

09:00 - 10:30	Keynote / Code AnalysisDiscussion / Tool Demonstration / Research / Early Research Achievements (ERA) / ICPC Keynotes at Meeting Room 106 Chair(s): Christoph Treude University of Melbourne, Nicolás Cardozo Universidad de los Andes, Raula Gaikovina Kula Nara Institute of Science and Technology, Chaiyong Rakhitwetsagul Mahidol University, Thailand

09:00 45m Keynote		Kobi Leins: Guidance on more than just standing upright to create safe models, software and use of data ICPC Keynotes
09:45 9m Full-paper		Implant Global and Local Hierarchy Information to Sequence based Code Representation Models Research Kechi Zhang Peking University, China, Zhuo Li , Zhi Jin Peking University, Ge Li Peking University Pre-print
09:54 9m Full-paper		Pathways to Leverage Transcompiler based Data Augmentation for Cross-Language Clone Detection Research Subroto Nag Pinku University of Saskatchewan, Debajyoti Mondal University of Saskatchewan, Chanchal K. Roy University of Saskatchewan Pre-print
10:03 5m Short-paper		Investigating the Generalizability of Deep Learning-based Clone Detectors Early Research Achievements (ERA) Eunjong Choi Kyoto Institute of Technology, Norihiro Fuke Osaka University, Yuji Fujiwara Osaka University, Norihiro Yoshida Ritsumeikan University, Katsuro Inoue Nanzan University
10:08 5m Short-paper		UnityLint: A Bad Smell Detector for Unity Tool Demonstration Matteo Bosco University of Sannio, Italy, Pasquale Cavoto University of Sannio, Italy, Augusto Ungolo University of Sannio, Italy, Biruk Asmare Muse Polytechnique Montréal, Foutse Khomh Polytechnique Montréal, Vittoria Nardone , Massimiliano Di Penta University of Sannio, Italy Pre-print
10:13 17m Panel		Discussion 5 Discussion

11:00 - 12:30	Empirical Studies and RecommendationsResearch / Discussion / Early Research Achievements (ERA) / Journal First at Meeting Room 106 Chair(s): Issam Sedki Concordia University, Vittoria Nardone

11:00 9m Full-paper		REMS: Recommending Extract Method Refactoring Opportunities via Multi-view Representation of Code Property Graph Research Di Cui , Qiangqiang Wang Xidian University, Siqi Wang , Jianlei Chi , Jianan Li Xidian University, Lu Wang Xidian University, Qingshan Li Xidian University
11:09 9m Full-paper		Automating Method Naming with Context-Aware Prompt-Tuning Research Jie Zhu Institute of Software, Chinese Academy of Sciences;University of Chinese Academy of Sciences, Lingwei Li Institute of Software at Chinese Academy of Sciences; University of Chinese Academy of Sciences, Li Yang Institute of Software at Chinese Academy of Sciences, Xiaoxiao Ma Institute of Software, Chinese Academy of Sciences, Chun Zuo Sinosoft Pre-print
11:18 9m Full-paper		Generation-based Code Review Automation: How Far Are We? Research Xin Zhou Singapore Management University, Singapore, Kisub Kim Singapore Management University, Bowen Xu North Carolina State University, DongGyun Han Royal Holloway, University of London, Junda He Singapore Management University, David Lo Singapore Management University Pre-print
11:27 9m Full-paper		Reanalysis of Empirical Data on Java Local Variables with Narrow and Broad Scope Research Dror Feitelson Hebrew University Pre-print
11:36 9m Talk		Predicting vulnerability inducing function versions using node embeddings and graph neural networks Journal First ecem mine özyedierler Istanbul Technical University, Ayse Tosun Istanbul Technical University, Sefa Eren Sahin Faculty of Computer and Informatics Engineering, Istanbul Technical University
11:45 5m Short-paper		Properly Offer Options to Improve the Practicality of Software Document Completion Tools Early Research Achievements (ERA) Zhipeng Cai School of Computer Science, Wuhan University, Songqiang Chen School of Computer Science, Wuhan University, Xiaoyuan Xie School of Computer Science, Wuhan University, China Media Attached
11:50 40m Panel		Discussion 6 Discussion

15:45 - 17:15	Bugs and Machine Learning / Steering Committee Meeting / ClosingResearch / Journal First / Closing at Meeting Room 106 Chair(s): Banani Roy University of Saskatchewan

15:45 9m Full-paper		Mitigating the Effect of Class Imbalance in Fault Localization Using Context-aware Generative Adversarial Network Research Yan Lei Chongqing University, Tiantian Wen , Huan Xie , Lingfeng Fu Chongqing University, Chunyan Liu Chongqing University, Lei Xu Haier Smart Home Co., Ltd., Hongxia Sun Qingdao Haidacheng Purchasing Service Co., Ltd. Pre-print Media Attached
15:54 9m Full-paper		Still Confusing for Bug-Component Triaging? Deep Feature Learning and Ensemble Setting to Rescue Research Yanqi Su Australian National University, Zheming Han , Zhipeng Gao Shanghai Institute for Advanced Study of Zhejiang University, Zhenchang Xing , Qinghua Lu CSIRO’s Data61, Xiwei (Sherry) Xu CSIRO’s Data61
16:03 9m Full-paper		Understanding Bugs in Multi-Language Deep Learning Frameworks Research Zengyang Li Central China Normal University, Sicheng Wang Central China Normal University, Wenshuo Wang , Peng Liang Wuhan University, China, Ran Mo Central China Normal University, Bing Li Wuhan University Link to publication Pre-print Media Attached
16:12 9m Full-paper		FVA: Assessing Function-Level Vulnerability by Integrating Flow-Sensitive Structure and Code Statement Semantic Research Chao Ni Zhejiang University, Liyu Shen Zhejiang University, Wei Wang Zhejiang University, Xiang Chen Nantong University, Xin Yin The State Key Laboratory of Blockchain and Data Security, Zhejiang University, Lexiao Zhang School of Software Technology, Zhejiang University
16:21 9m Talk		Event-Aware Precise Dynamic Slicing for Automatic Debugging of Android Applications Journal First Hsu Myat Win University of Technology Sydney (UTS), Shin Hwei Tan Southern University of Science and Technology, Yulei Sui University of New South Wales, Sydney Link to publication
16:30 15m Panel		Discussion 8 Closing
16:45 30m Meeting		Steering Committee Meeting and Closing Closing Alexander Serebrenik Eindhoven University of Technology, Igor Steinmacher Northern Arizona University

Replications and Negative Results (RENE)ICPC 2023

Program Display Configuration

Mon 15 MayDisplayed time zone: Hobart change

Tue 16 MayDisplayed time zone: Hobart change

Accepted Papers

Call for Papers

Evaluation Criteria

Submission Instructions

Eleni ConstantinouProgram Co-Chair

University of Cyprus

Cyprus

Peter RigbyProgram Co-Chair

Concordia University; Meta

Canada

Moritz BellerPC Member

Meta Platforms, Inc., USA

United States

Emad FallahzadehPC Member

Concordia University

Canada

Audris MockusPC Member

The University of Tennessee

Csaba NagyPC Member

Software Institute - USI, Lugano

Christian D. NewmanPC Member

Rochester Institute of Technology

Sebastiano PanichellaPC Member

Zurich University of Applied Sciences

Switzerland

Md Tajmilur RahmanPC Member

Gannon University

United States

Parth SureshPC Member

Meta Platforms, Inc.

United States

Nikolaos TsantalisPC Member

Concordia University

Canada

Ahmed ZeroualiPC Member

Vrije Universiteit Brussel

Belgium

Mon 15 May
Displayed time zone: Hobart change

Tue 16 May
Displayed time zone: Hobart change