Towards Trustworthy AI Software Development Assistance
It is expected that in the near future, AI software development assistants will play an important role in the software industry. However, current AI SD assistants tend to be unreliable, often producing incorrect, unsafe, or low-quality code. We seek to resolve these issues by introducing a holistic architecture for constructing, training, and using trustworthy AI software development assistants. In the center of the architecture, there is a foundational LLM trained on datasets representative of real-world coding scenarios and complex software architectures, and fine-tuned on code quality criteria beyond correctness. The LLM will make use of graph-based code representations for advanced semantic comprehension. We envision a knowledge graph integrated into the system to provide up-to-date background knowledge and to enable the assistant to provide appropriate explanations. Finally, a modular framework for constrained decoding will ensure that certain guarantees (e.g., for correctness and security) hold for the generated code.
Fri 19 AprDisplayed time zone: Lisbon change
16:00 - 17:30 | Language Models and Generated Code 4New Ideas and Emerging Results / Research Track at Almada Negreiros Chair(s): Shin Yoo Korea Advanced Institute of Science and Technology | ||
16:00 15mTalk | Lost in Translation: A Study of Bugs Introduced by Large Language Models while Translating Code Research Track Rangeet Pan IBM Research, Ali Reza Ibrahimzada University of Illinois Urbana-Champaign, Rahul Krishna IBM Research, Divya Sankar IBM Research, Lambert Pouguem Wassi IBM Research, Michele Merler IBM Research, Boris Sobolev IBM Research, Raju Pavuluri IBM T.J. Watson Research Center, Saurabh Sinha IBM Research, Reyhaneh Jabbarvand University of Illinois at Urbana-Champaign DOI Pre-print Media Attached | ||
16:15 15mTalk | Traces of Memorisation in Large Language Models for Code Research Track Ali Al-Kaswan Delft University of Technology, Netherlands, Maliheh Izadi Delft University of Technology, Arie van Deursen Delft University of Technology Pre-print | ||
16:30 15mTalk | Language Models for Code Completion: A Practical Evaluation Research Track Maliheh Izadi Delft University of Technology, Jonathan Katzy Delft University of Technology, Tim van Dam Delft University of Technology, Marc Otten Delft University of Technology, Răzvan Mihai Popescu Delft University of Technology, Arie van Deursen Delft University of Technology Pre-print | ||
16:45 15mTalk | Evaluating Large Language Models in Class-Level Code Generation Research Track Xueying Du Fudan University, Mingwei Liu Fudan University, Kaixin Wang Fudan University, Hanlin Wang Fudan University, Junwei Liu Huazhong University of Science and Technology, Yixuan Chen Fudan University, Jiayi Feng Fudan University, Chaofeng Sha Fudan University, Xin Peng Fudan University, Yiling Lou Fudan University Pre-print | ||
17:00 7mTalk | Naturalness of Attention: Revisiting Attention in Code Language Models New Ideas and Emerging Results Pre-print | ||
17:07 7mTalk | Towards Trustworthy AI Software Development Assistance New Ideas and Emerging Results DOI Pre-print |