This talk delves into the transformative effects of AI-driven conversational test automation, specifically examining the impact of large language models (LLMs) like ChatGPT/GPT-4 on software testing. We posit that conversational LLMs are poised to fundamentally change software testing, with several characteristics - some previously deemed negative - likely being advantageous for testing applications. The presentation will explore several implications of this technological shift, including enhanced test case generation, streamlined test maintenance, and democratized testing expertise. LLMs have the potential to generate comprehensive, context-aware test cases, and identify elusive edge cases. They can not only augment existing test suites but also dynamically update them in response to evolving requirements and software changes. Additionally, LLMs empower non-expert and non-technical stakeholders to contribute to the testing process using natural language. The unique conversational mode of these models facilitates immediate feedback, fostering a more detailed and nuanced understanding of testers’ goals while enabling rapid clarification and learning. In conclusion, the talk will address potential risks and explore the future trajectory of LLM technologies in test automation. We will consider the subsequent developments and long-term implications arising from the anticipated surge in research and industry interest in harnessing LLMs for software test automation.

Robert Feldt is professor of software engineering at Chalmers University of Technology, Sweden, and at Blekinge Institute of Technology, Sweden. He has broad research interests spanning from human factors to hardcore automation and statistics, and including software testing and quality, requirements engineering, human-centred (behavioural) software engineering. He was an early contributor to search-based software engineering and have recently argued for increased application of psychology and social science to better understand and improve software engineering. Most of his research is empirical and conducted in close collaboration with industry partners in Sweden, Europe and Asia but he also leads more basic research. He received a PhD in computer engineering from the Chalmers University of Technology in 2002, studied psychology at Gothenburg University in the ’90s and has also worked as an IT and software consultant for more than 25 years. He is passionate about empirical research and methods and to change organisations through technological innovation.