Empirical research provides two-fold support to the development of approaches and tools aimed at supporting software engineers. On the one hand, empirical studies help to understand a phenomenon or a context of interest. On the other hand, studies compare approaches and evaluate how software engineers benefit from them. Over the past decades, there has been a tangible evolution in how empirical evaluation is conducted in software engineering. This is due to multiple reasons. On the one hand, the research community has matured a lot thanks also to guidelines developed by several researchers. On the other hand, the large availability of data and artifacts, mainly from the open-source, has made it possible to conduct larger evaluations, and in some cases to reach study participants. In this keynote, I will first overview how empirical research has been used over the past decades to evaluate tools, and how this is changing over the years. I will also emphasize the importance of combining quantitative and qualitative evaluations, and how sometimes depth turns out to be more useful than just breadth. I will also emphasize how research is not a straightforward path, and negative results are often an essential component for future advances. Last, but not least, I will emphasize how the role of empirical evaluation is changing with the pervasiveness of artificial intelligence methods in software engineering research.

Massimiliano Di Penta is a full professor at the University of Sannio, Italy. His research interests include software maintenance and evolution, mining software repositories, empirical software engineering, search-based software engineering, and software testing. He is an author of over 260 papers that appeared in international journals, conferences, and workshops. He has received several awards for research and service, including four ACM SIGSOFT Distinguished paper awards. Most importantly, he has received several distinguished reviewer awards. He serves and has served in the organizing and program committees of more than 100 conferences, including ICSE, FSE, ASE, and ICSME. He will be program co-chair of ESEC/FSE 2021 and of ICSE 2023. He is co-editor in chief of the Journal of Software: Evolution and Processes edited by Wiley, editorial board member of ACM Transactions on Software Engineering and Methodology and Empirical Software Engineering Journal edited by Springer, and has served the editorial board of the IEEE Transactions on Software Engineering.