ICPC 2023
Mon 15 - Tue 16 May 2023 Melbourne, Australia
co-located with ICSE 2023
Performance is key to the success and adoption of software systems. In video games, performance is commonly highlighted as one of the top quality concerns raised by players. To check the performances of their systems, development teams tend to rely on profiling and monitoring tools, which observe program executions to identify regressions, and the usage of static analysis tools for this purpose has been so far limited. Lately, the success of Large Language Models in many code analytics tools led to attempts to leverage them in static performance analysis. These studies showed promising results in predicting runtime and regressions on large public datasets. In this paper, we evaluate the usability of such models in practice, and particularly in the domain of video games. We train a state-of-the-art neural network on the Code4Bench dataset to predict runtime regressions for programming competition tasks, then evaluate its ability to generalize to new domains. Our results show that these models achieve great results (e.g. 95.73% accuracy for performance comparison) on the original domain for program solving in-sample programming tasks, yet fail to generalize to out- of-sample tasks. Furthermore, we show that transfer techniques such as domain adversarial adaptation and model fine-tuning are not sufficient to transfer these models to the target industrial domain of AAA games.

Mon 15 May

Displayed time zone: Hobart change

11:00 - 12:30
Keynote / Documentation and Stack OverflowTool Demonstration / Research / ICPC Keynotes / Replications and Negative Results (RENE) / Discussion at Meeting Room 106
Chair(s): Bonita Sharif University of Nebraska-Lincoln, USA, Raula Gaikovina Kula Nara Institute of Science and Technology, Chanchal K. Roy University of Saskatchewan
11:00
45m
Keynote
April Wensel: Applications of Emotional Intelligence in Program Comprehension
ICPC Keynotes

11:45
9m
Full-paper
APIContext2Com: Code Comment Generation by Incorporating Pre-Defined API Documentation
Research
Ramin Shahbazi , Fatemeh Hendijani Fard University of British Columbia
Pre-print
11:54
5m
Short-paper
PyVerDetector: A Chrome Extension Detecting the Python Version of Stack Overflow Code Snippets
Tool Demonstration
SHIYU YANG , Tetsuya Kanda Osaka University, Davide Pizzolotto Osaka University, Daniel M. German University of Victoria, Yoshiki Higo Osaka University
11:59
5m
Short-paper
RCGraph - A Tool to Integrate Readme and Commits through Temporal Knowledge Graphs
Tool Demonstration
Akhila Sri Manasa Venigalla IIT Tirupati, Mir Sameed Ali Indian Institute of Technology Tirupati, Nikhil Manjunath Indian Institute of Technology Tirupati, Sridhar Chimalakonda IIT Tirupati
12:04
9m
Full-paper
Performance Prediction From Source Code Is Task and Domain Specific
Replications and Negative Results (RENE)
Markus Böck TU Wien, Sarra Habchi Ubisoft, Mathieu Nayrolles Ubisoft Montreal, Jürgen Cito TU Wien
12:13
17m
Panel
Discussion 2
Discussion