FORGE 2024 - Keynotes

FORGE 2024 Keynotes

Keynote 1: Large Language Models for Test Case Repair

Time: Sun 14 April 2024 09:10 - 09:50

Abstract: Ensuring the quality of software systems through testing is essential, yet maintaining test cases poses significant challenges. The need for frequent updates to align with the evolving system under test often entails high complexity and cost for maintaining these test cases. Further, unrepaired broken test cases can degrade test suite quality and disrupt the software development process, wasting developers’ time. In addition, flaky tests are problematic because they non-deterministically pass or fail for the same software version under test, causing confusion and wasting development effort. In this presentation, I will report on recent work using language models to help automate test repair and will reflect on current results, limitations, and future work.

Prof. Lionel C. Briand is professor of software engineering and has shared appointments between (1) School of Electrical Engineering and Computer Science, University of Ottawa, Canada and (2) The Lero SFI Centre for Software Research, University of Limerick, Ireland. He is a Canada research chair in Intelligent Software Dependability and Compliance (Tier 1) and the director of Lero. He has conducted applied research in collaboration with industry for more than 25 years, including projects in the automotive, aerospace, manufacturing, financial, and energy domains. He is a fellow of the IEEE, ACM, and Royal Society of Canada. He was also granted the IEEE Computer Society Harlan Mills Award (2012), the IEEE Reliability Society Engineer-of-the-year award (2013), and the ACM SIGSOFT Outstanding Research Award (2022) for his work on software testing and verification. His research interests include software testing and verification (including security and AI aspects), trustworthy AI, applications of AI in software engineering, model-driven software development, requirements engineering, and empirical software engineering. Further details can be found on: www.lbriand.info.

Keynote 2: Towards an Interpretable Science of Deep Learning for Software Engineering: A Causal Inference View

Time: Sun 14 April 2024 14:00 - 14:40

Abstract: Neural Code Models (NCMs) are rapidly progressing from research prototypes to commercial developer tools. As such, understanding the capabilities and limitations of such models is becoming critical. However, the abilities of these models are typically measured using automated metrics that often only reveal a portion of their real-world performance. While, in general, the performance of NCMs appears promising, currently much is unknown about how such models arrive at decisions or whether practitioners trust NCMs’ outcomes. In this talk, I will introduce doCode, a post hoc interpretability framework specific to NCMs that can explain model predictions. doCode is based upon causal inference to enable programming language-oriented explanations. While the theoretical underpinnings of doCode are extensible to exploring different model properties, we provide a concrete instantiation that aims to mitigate the impact of spurious correlations by grounding explanations of model behavior in properties of programming languages. doCode can generate causal explanations based on Abstract Syntax Tree information and software engineering-based interventions. To demonstrate the practical benefit of doCode, I will present empirical results of using doCode for detecting confounding bias in NCMs.

Prof. Denys Poshyvanyk is a Chancellor Professor and a Graduate Director in the Computer Science Department at William & Mary. He currently serves as a Guest Editor-in-Chief of the AI-SE Continuous Special Section at the ACM Transactions on Software Engineering and Methodology (TOSEM) and a Program Co-Chair for FSE’25. He is a recipient of multiple ACM SIGSOFT Distinguished paper awards, the NSF CAREER award (2013). He is an IEEE Fellow and an ACM distinguished member. Further details can be found on: https://conf.researchr.org/profile/icse-2024/denysposhyvanyk.

Time Zone

The program is currently displayed in (GMT+01:00) Lisbon.

Use conference time zone: (GMT+01:00) LisbonSelect other time zone

The GMT offsets shown reflect the offsets at the moment of the conference.

Time Band

By setting a time band, the program will dim events that are outside this time window. This is useful for (virtual) conferences with a continuous program (with repeated sessions).
The time band will also limit the events that are included in the personal iCalendar subscription service.

Display full programSpecify a time band

Save

You're viewing the program in a time zone which is different from your device's time zone change time zone

Sun 14 Apr
Displayed time zone: Lisbon change

09:00 - 10:30	FORGE2024 Opening / Keynote 1 / PanelKeynotes / Panel at Luis de Freitas Branco Chair(s): Xin Xia Huawei Technologies, Xing Hu Zhejiang University

09:00 10m Day opening		Introduction from The Chairs Keynotes
09:10 40m Keynote		Keynote 1: Large Language Models for Test Case Repair Keynotes Lionel Briand University of Ottawa, Canada; Lero centre, University of Limerick, Ireland
09:50 40m Panel		Theme: Is There Space for Software Engineering Researchers to Contribute to AI4SE in The Era of Foundation Models? Panel Lionel Briand University of Ottawa, Canada; Lero centre, University of Limerick, Ireland, Denys Poshyvanyk William & Mary, Prem Devanbu University of California at Davis, Massimiliano Di Penta University of Sannio, Italy, David Lo Singapore Management University

14:00 - 15:30	Keynote 2 & Properties of Foundation ModelsResearch Track / Keynotes at Luis de Freitas Branco Chair(s): David Lo Singapore Management University, Feifei Niu University of Ottawa

14:00 40m Keynote		Keynote 2: Towards an Interpretable Science of Deep Learning for Software Engineering: A Causal Inference View Keynotes Denys Poshyvanyk William & Mary
14:40 14m Full-paper		Exploring the Impact of the Output Format on the Evaluation of Large Language Models for Code TranslationFull Paper Research Track Marcos Macedo Queen's University, Kingston, Ontario, Yuan Tian Queen's University, Kingston, Ontario, Filipe Cogo Centre for Software Excellence, Huawei Canada, Bram Adams Queen's University Pre-print
14:54 7m Short-paper		Is Attention All You Need? Toward a Conceptual Model for Social Awareness in Large Language ModelsNew Idea Paper Research Track Gianmario Voria University of Salerno, Gemma Catolino University of Salerno, Fabio Palomba University of Salerno Pre-print
15:01 14m Full-paper		An Exploratory Investigation into Code License Infringements in Large Language Model Training DatasetsFull Paper Research Track Jonathan Katzy Delft University of Technology, Răzvan Mihai Popescu Delft University of Technology, Arie van Deursen Delft University of Technology, Maliheh Izadi Delft University of Technology
15:15 15m Other		Discussion Research Track

Accepted Papers

	Title
	Keynote 2: Towards an Interpretable Science of Deep Learning for Software Engineering: A Causal Inference View Keynotes Denys Poshyvanyk

11:00 - 12:30	Foundation Models for Software Quality AssuranceResearch Track at Luis de Freitas Branco Chair(s): Matteo Ciniselli Università della Svizzera Italiana

11:00 14m Full-paper		Deep Multiple Assertions GenerationFull Paper Research Track Hailong Wang Zhejiang University, Tongtong Xu Huawei, Bei Wang Huawei
11:14 14m Full-paper		MeTMaP: Metamorphic Testing for Detecting False Vector Matching Problems in LLM Augmented GenerationFull Paper Research Track Guanyu Wang Beijing University of Posts and Telecommunications, Yuekang Li The University of New South Wales, Yi Liu Nanyang Technological University, Gelei Deng Nanyang Technological University, Li Tianlin Nanyang Technological University, Guosheng Xu Beijing University of Posts and Telecommunications, Yang Liu Nanyang Technological University, Haoyu Wang Huazhong University of Science and Technology, Kailong Wang Huazhong University of Science and Technology
11:28 14m Full-paper		Planning to Guide LLM for Code Coverage PredictionFull Paper Research Track Hridya Dhulipala University of Texas at Dallas, Aashish Yadavally University of Texas at Dallas, Tien N. Nguyen University of Texas at Dallas
11:42 7m Short-paper		The Emergence of Large Language Models in Static Analysis: A First Look through Micro-BenchmarksNew Idea Paper Research Track Ashwin Prasad Shivarpatna Venkatesh University of Paderborn, Samkutty Sabu University of Paderborn, Amir Mir Delft University of Technology, Sofia Reis Instituto Superior Técnico, U. Lisboa & INESC-ID, Eric Bodden
11:49 14m Full-paper		Reality Bites: Assessing the Realism of Driving Scenarios with Large Language ModelsFull Paper Research Track Jiahui Wu Simula Research Laboratory and University of Oslo, Chengjie Lu Simula Research Laboratory and University of Oslo, Aitor Arrieta Mondragon University, Tao Yue Beihang University, Shaukat Ali Simula Research Laboratory and Oslo Metropolitan University
12:03 7m Short-paper		Assessing the Impact of GPT-4 Turbo in Generating Defeaters for Assurance CasesNew Idea Paper Research Track Kimya Khakzad Shahandashti York University, Mithila Sivakumar York University, Mohammad Mahdi Mohajer York University, Alvine Boaye Belle York University, Song Wang York University, Timothy Lethbridge University of Ottawa
12:10 20m Other		Discussion Research Track

16:00 - 17:30	FORGE2024 Awards & Foundation Models for Code and Documentation GenerationResearch Track at Luis de Freitas Branco Chair(s): Antonio Mastropaolo Università della Svizzera italiana

16:00 10m Awards		Award Ceremony Research Track
16:10 7m Short-paper		Fine Tuning Large Language Model for Secure Code GenerationNew Idea Paper Research Track Junjie Li Concordia University, Aseem Sangalay Delhi Technological University, Cheng Cheng Concordia University, Yuan Tian Queen's University, Kingston, Ontario, Jinqiu Yang Concordia University
16:17 14m Full-paper		Investigating the Performance of Language Models for Completing Code in Functional Programming Languages: a Haskell Case StudyFull Paper Research Track Tim van Dam Delft University of Technology, Frank van der Heijden Delft University of Technology, Philippe de Bekker Delft University of Technology, Berend Nieuwschepen Delft University of Technology, Marc Otten Delft University of Technology, Maliheh Izadi Delft University of Technology
16:31 7m Short-paper		On Evaluating the Efficiency of Source Code Generated by LLMsNew Idea Paper Research Track Changan Niu Software Institute, Nanjing University, Ting Zhang Singapore Management University, Chuanyi Li Nanjing University, Bin Luo Nanjing University, Vincent Ng Human Language Technology Research Institute, University of Texas at Dallas, Richardson, TX 75083-0688
16:38 14m Full-paper		PathOCL: Path-Based Prompt Augmentation for OCL Generation with GPT-4Full Paper Research Track Seif Abukhalaf Polytechnique Montreal, Mohammad Hamdaqa Polytechnique Montréal, Foutse Khomh École Polytechnique de Montréal
16:52 7m Short-paper		Creative and Correct: Requesting Diverse Code Solutions from AI Foundation ModelsNew Idea Paper Research Track Scott Blyth Monash University, Christoph Treude Singapore Management University, Markus Wagner Monash University, Australia
16:59 7m Short-paper		Commit Message Generation via ChatGPT: How Far Are We?New Idea Paper Research Track Yifan Wu Peking University, Ying Li School of Software and Microelectronics, Peking University, Beijing, China, Siyu Yu The Chinese University of Hong Kong, Shenzhen (CUHK-Shenzhen)
17:06 24m Other		Discussion Research Track

KeynotesFORGE 2024