ITG: Trace Generation via Iterative Interaction between LLM Query and Trace Checking
Linear temporal logic (LTL) trace generation is a key step in SAT-verifiable LTL satisfiability checking, which provides satisfiable evidence for satisfiable results. Due to the complexity of the problem (PSPACE-Complete), we discover that existing neural network-based approaches will fail as the formula sizes increase. Recently, large language models (LLMs) have demonstrated remarkable reasoning capabilities, benefiting from efficient training on hyper-scale data. Inspired by this, we propose an iterative interaction framework for applying LLMs, exemplified by ChatGPT, to generate a trace satisfying a given LTL formula. The key insight behind it is to transfer the powerful reasoning capabilities of LLM to LTL trace generation via iterative interaction between LLM reasoning and logical reasoning. Preliminary results show that compared with the state-of-the-art approach, the accuracy is relatively improved by 9.7%-23.4%. Besides, we show that our framework is able to produce heuristics for new tasks, which provides a reference for other reasoning-heavy tasks requiring heuristics.
Wed 17 AprDisplayed time zone: Lisbon change
16:00 - 17:30 | LLM, NN and other AI technologies 2Journal-first Papers / Software Engineering in Practice / New Ideas and Emerging Results / Research Track / Software Engineering in Society at Luis de Freitas Branco Chair(s): Jane Cleland-Huang University of Notre Dame | ||
16:00 15mTalk | Large Language Models for Test-Free Fault Localization Research Track Aidan Z.H. Yang Carnegie Mellon University, Claire Le Goues Carnegie Mellon University, Ruben Martins Carnegie Mellon University, Vincent J. Hellendoorn Carnegie Mellon University | ||
16:15 15mTalk | Dataflow Analysis-Inspired Deep Learning for Efficient Vulnerability Detection Research Track Benjamin Steenhoek Iowa State University, Hongyang Gao Dept. of Computer Science, Iowa State University, Wei Le Iowa State University Pre-print | ||
16:30 15mTalk | An Empirical Study on Compliance with Ranking Transparency in the Software Documentation of EU Online Platforms Software Engineering in Society Francesco Sovrano University of Zurich, Michaël Lognoul University of Namur (CRIDS, NADI), Alberto Bacchelli University of Zurich | ||
16:45 15mTalk | An Industry Case Study on Adoption of AI-based Programming Assistants Software Engineering in Practice Nicole Davila Universidade Federal do Rio Grande do Sul, Igor Wiese Federal University of Technology, Igor Steinmacher Northern Arizona University, Lucas Lucio Federal University of Technology - Paraná (UTFPR), André Kawamoto Federal University of Technology - Paraná (UTFPR), Gilson José Peres Favaro , Ingrid Nunes Universidade Federal do Rio Grande do Sul (UFRGS), Brazil | ||
17:00 7mTalk | Assessing LLMs for High Stakes Applications Software Engineering in Practice Shannon K. Gallagher Software Engineering Institute, Carnegie Mellon University, Jasmine Ratchford Software Engineering Institute, Carnegie Mellon University, Tyler Brooks Software Engineering Institute, Carnegie Mellon University, Bryan P. Brown Software Engineering Institute, Carnegie Mellon University, Eric Heim Software Engineering Institute, Carnegie Mellon University, William R. Nichols Software Engineering Institute, Carnegie Mellon University, Scott McMillan Software Engineering Institute, Carnegie Mellon University, Swati Rallapalli Software Engineering Institute, Carnegie Mellon University, Carol J. Smith Software Engineering Institute, Carnegie Mellon University, Nathan VanHoudnos Software Engineering Institute, Carnegie Mellon University, Nick Winski Software Engineering Institute, Carnegie Mellon University, Andrew O. Mellinger Software Engineering Institute, Carnegie Mellon University | ||
17:07 7mTalk | ITG: Trace Generation via Iterative Interaction between LLM Query and Trace Checking New Ideas and Emerging Results Weilin Luo SUN YAT-SEN UNIVERSITY, Weiyuan Fang SUN YAT-SEN UNIVERSITY, Junming Qiu SUN YAT-SEN UNIVERSITY, Hai Wan School of Data and Computer Science, Sun Yat-sen University, Yanan Liu SUN YAT-SEN UNIVERSITY, Rongzhen Ye Sun Yat-Sen University | ||
17:14 7mTalk | Improving Cross-Language Code Clone Detection via Code Representation Learning and Graph Neural Networks Journal-first Papers NIKITA MEHROTRA Indraprastha Institute of Information Technology, Akash Sharma IIIT-Delhi, Anmol Jindal IIIT-Delhi, Rahul Purandare UNL, USA |