KeynotesICST 2025

ICST 2025 will host two keynote speeches delivered by distinguished experts Atif Memon and Antonia Bertolino. Key details about each keynote are outlined below.

Keynote 1: Repurposing automated testing for ML software development and deployment

Speaker: Atif Memon

Abstract: ML-powered features in software are notoriously challenging to test. As the behaviors of an ML feature emerge, so do user-level bugs. Traditional testing methods conducted during code development, or metrics collected during model development fail to provide an accurate view of such bugs. Moreover, online metrics collected during A/B experiments do not directly expose them. I will talk about how we are filling this gap by repurposing automated testing for ML software development and deployment. We leverage large volumes of system test cases, automatically obtained with their test oracles, to derive defect classes, which represent a partition of the input space on which the ML-based software does measurably worse. These defect classes are reported early during development, thereby enabling rapid improvement.

Atif Memon

Biography: Atif Memon is a Distinguished Engineer at Apple, where he leads the development of machine learning-driven tools for evaluating conventional and AI-powered software, processing millions of evaluations daily. His work includes developing methodologies for automatically authoring and maintaining tests for ML systems, and adapting automated testing to meet the challenges of machine learning software development. He has developed a holistic approach for evaluating emergent behavior in ML-based information retrieval systems. Additionally, he has created algorithms for flakiness scoring that substantially reduce test flakiness with minimal loss in fault detection. Before joining Apple, Dr. Memon was a Professor in the Department of Computer Science at the University of Maryland, where he founded the Event Driven Software Lab (EDSL) and published over 100 research papers on software testing and engineering. He designed GUITAR, a model-based GUI testing tool, and served on numerous program and steering committees. Dr. Memon earned his Ph.D. from the University of Pittsburgh and has held visiting researcher positions at Google, the Chinese Academy of Sciences, and Tata Research. Dr. Memon’s contributions have been recognized with the NSF CAREER Award, among other honors. He has received grants from the NSF, NIH, NASA, and DARPA for his research work.

Keynote 2: LLMs: killers or boosters of software testing research?

Speaker: Antonia Bertolino

Abstract: After the big waves in the last two decades of leveraging AI and ML-based technologies for pushing test automation, in the latest couple of years the new big thing is undoubtedly LLM-assisted testing. LLMs are increasingly used for test case generation at differing levels of the testing process, for oracle derivation, for improving test code, as well as for debugging and code repairing. Several studies already show that LLMs can be an effective collaborator in different testing activities, but also warn about possible latent issues. Beyond the specific technique or tool, in this keynote I will talk on how LLMs are already largely transforming testing research. Questions that I would like to put on the discussion table are: can LLMs really substitute human testers? And to what extent can they solve the longstanding challenge of test automation? on what problems should software testing research focus? The talk will be based on a critical reflection of white and grey literature.

Antonia Bertolino

Biography: Antonia Bertolino is currently an Adjunct Professor at the Gran Sasso Science Institute (GSSI), L'Aquila, Italy. Previously, she has been a Research Director of the Italian National Research Council (CNR), at the Institute for Information Science and Technologies ‘‘Alessandro Faedo’’ (ISTI), in Pisa, Italy. Her research interests cover software and services validation, testing, and monitoring, and on these topics she has published ~250 papers in international journals, conferences and workshops. She has led or participated in several collaborative projects, including more recently the H2020 European Project ElasTest (Elastic platform for testing complex distributed large software systems), the National project GAUSS (Governing Adaptive and Unplanned Systems of Systems), the FP7 CP Learn PAd (Model-Based Social Learning for Public Administrations), and the FP7 IP CHOReOS (Large Scale Choreographies for the Future Internet). Currently she is the Software Testing Area Chair for Elsevier Journal of Systems and Software, and an Associate Editor of Wiley Journal of Software: Evolution and Process. She serves regularly in the Program Committee of the most renowned conferences in the field of Software Engineering, such as ESEC-FSE, ASE and ICSE, and in software testing and analysis, such as ISSTA and ICST. She was the General Chair of the 2015 edition of the flagship ACM/IEEE ICSE in Florence (Italy), and General/Program Co-Chair, among others, of AST2020, QUATIC2018, ICST2012, CBSE2011, ESEC/FSE2007.

Events