Abstract: Large Language Model (LLM) have demonstrated significant potential in automating an array of software engineering activities, from code summarization and search to program analysis and vulnerability detection. These tasks span generative, ranking, and classification tasks. To advance LLM for Automated Software Engineering (LLM4ASE), it is vital to discern not just its strengths but also its shortcomings. This talk delves into our discoveries when probing LLM4ASE’s boundaries, especially considering the intricacies of software engineering—a multifaceted human-in-the-loop activity. We will spotlight the model’s weaknesses and existing constraints, alongside suggesting strategies to mitigate these challenges. While this talk may not provide comprehensive solutions, it aims to foster a dialogue around pertinent questions, nudging the community toward collaboratively elevating Automated Software Engineering (ASE) by harnessing LLM’s strengths and addressing its limitations.

David Lo is a Professor of Computer Science and Director of the Information and Systems Cluster at School of Computing and Information Systems, Singapore Management University. He leads the Software Analytics Research (SOAR) group. His research interest is in the intersection of software engineering, cybersecurity, and data science, encompassing socio-technical aspects and analysis of different kinds of software artifacts, with the goal of improving software quality and security and developer productivity. His work has been published in major and premier conferences and journals in the area of software engineering, AI, and cybersecurity attracting substantial interest from the community. His work has been supported by NRF, MOE, NCR, AI Singapore, and several international research projects. He has won more than 15 international research and service awards including 6 ACM SIGSOFT Distinguished Paper Awards. He has received a number of international honors including IEEE Fellow (class of 2022, through Computer Society), Fellow of Automated Software Engineering (class of 2021), and ACM Distinguished Member (class of 2019).