The following four technical briefings will be held during the main conference. You do not need to register for these separately. Each will be 90 minutes. Please note that details on this page are subject to minor change as the conference program is developed in more detail.
Click to jump to the details
Technical Briefing 1 (Wed 11 a.m.): Qualitative Research and Qualitative Data Analysis
Technical Briefing 2 (Wed 4 p.m. ): Using Mixed Methods Research in Software Engineering
Technical Briefing 3 (Thu 11 a.m.): Codellm-Devkit: A Framework To Contextualize Code LLMs
Technical Briefing 4 (Fri 11 a.m.): Architecting Digital Twin-Based Systems
Technical Briefing 1: Qualitative Research and Qualitative Data Analysis
Wednesday April 30: 11:00 to 12:30
Presenters: Rashina Hoda, Monash University
This technical briefing will help ICSE attendees gain an understanding of the basics of qualitative research and hands-on experience with conducting systematic and rigorous qualitative data analysis using socio-technical grounded theory (STGT). It will be useful for students and professional researchers with all levels of experience, including those attempting qualita- tive research for the first time. The briefing will cover selected content from the new “Qualitative Research with Socio-Technical Grounded Theory” Springer book and include plenty of hands- on examples of qualitative data analysis to interactively work through. It will equip attendees with the knowledge, skills, and resources required to handle qualitative data from human and LLM sources, and to derive rich descriptive findings, taxonomies, theoretical frameworks, models, and theories. (Full pdf description).
Learning objectives
- Identifying different types of qualitative data
- Identifying and selecting from different qualitative data collection techniques for various study contexts
- Assessing the quality of qualitative data for inclusion in research studies
- Applying STGT’s data analysis techniques of open cod- ing, constant comparison, and memoing for making sense of different types of qualitative data
- Learning to progress from first, often unsatisfactory, attempts to satisfactory and good qualitative data analysis
- Deriving rich, socio-technical codes, concepts, and cate- gories, and
- Understanding different expected outcomes, e.g. rich de- scriptive findings, insights, taxonomies, theoretical frame- works, and theories, through several examples from pub- lished studies
Outline
- An introduction to qualitative research and socio- technical research framework
- Types of qualitative data, such as those derived from texts, audio and video recordings, field notes, observational notes, participant reflections, discussion forums, commits logs, and chats
- Qualitative data collection techniques, such as through surveys, interviews, observations, focus groups, usability evaluations, data mining, LLM outputs
- STGT’s basic qualitative data analysis techniques, such as open coding constant comparison Role of memoing in qualitative data analysis and theoretical sampling
- Examples of possible outcomes of qualitative data anal- ysis, such as rich descriptive findings, preliminary or mature taxonomies, theoretical frameworks, and theories
- Question and Answers
Hands-on Exercises
The briefing will include hands-on experience with qualitative data analysis applying a socio-technical lens. Participants will have the opportunity to apply qualitative data analysis on a piece of text/data (relevant to HCI studies). Together, we will go through three iterations: ‘do on your own’, ‘discuss as a group’, ‘debrief’, moving from first attempt and its debrief to the second attempt and its debrief and then to the final attempt and its debrief. In the process, examples of poor coding, satisfactory coding, and good analytical coding will be shared to demonstrate how we can iteratively improve the quality of our analysis. No devices are required for the hands-on exercises.
Bios of the presenter
Rashina Hoda is a Professor of Software Engineering and Associate Dean Equity Diversity and Inclusion at the Faculty of Information Technology at Monash University, Melbourne. She is a world leading expert in qualitative empirical research. Based on 15+ years of conducting, supervising, reviewing, and editing experience, she has developed Socio-Technical Grounded Theory (STGT), a modern socio-technical version of sociological Grounded Theory methods. This technical briefing will draw from her book Qualitative Research with Socio-technical Grounded Theory (Springer, 2024).
Technical Briefing 2 : Using Mixed Methods Research in Software Engineering
Wednesday April 30: 16:00 to 17:30
Presenters: Margaret-Anne Storey, University of Victoria; Rashina Hoda, Monash University; Maria Teresa Baldassarre, University of Bari Aldo Moro; Alessandra Maciel Paz Milani, University of Victoria
This technical briefing will introduce participants to the foundational principles and practical application of mixed methods research (MMR) in software engineering (SE). Participants will explore the rationale behind using mixed methods, learn how to design mixed methods studies, and understand how to integrate qualitative and quantitative data to produce novel insights. This session will also cover best practices and highlight common pitfalls, or “antipatterns” to avoid. This briefing is ideal for both novice and experienced researchers interested in enhancing their methodological toolkits in socio-technical SE contexts. (Full pdf description).
Target audience
This session is designed for software engineering re- searchers from academia and industry, particularly those who want to explore or refine mixed methods research designs. Attendees should have a basic understanding of research methods, but no prior experience with mixed methods is necessary. It is suitable for Ph.D. students, postdoctoral researchers, experienced researchers, and industry professionals.
Learning objectives
- Understand the core principles of mixed methods re- search, including methodological rationale, novel inte- grated insights, procedural rigor, and ethical research.
- Learn to choose an appropriate mixed methods design based on their research questions and constraints.
- Explore how to effectively integrate qualitative and quan- titative data to generate rich insights.
- Identify common “antipatterns” in mixed methods re- search and how to avoid them.
- Gain hands-on experience in designing and critiquing mixed methods research through scenarios and examples drawn from SE research.
Outline
- Overview of Principles
- Mixed Methods Research Designs (Exploratory Sequential Design; Explanatory Sequential Design; Convergent Parallel Design; Embedded Design)
- Practical Scenarios in Software Engineering
Bios of the presenters
Margaret-Anne Storey is a Professor at the University of Victoria and a leading expert in mixed methods research in software engineering. With extensive experience in designing MMR studies, she has contributed to several key papers on the subject and continues to mentor the next generation of SE researchers.
Rashina Hoda is a Professor of Software Engineering at Monash University, Melbourne. She is a leading expert in qualitative and mixed methods empirical research. Much of her methodological knowledge is preserved in her new book “Qualitative Research with Socio-Technical Grounded Theory” published by Springer. Rashina has delivered several technical briefings on qualitative research and qualitative data analysis at ICSE and will be delivering another one this year.
Maria Teresa Baldassarre is an Associate Professor in Software Engifneering at the University of Bari Aldo Moro, Italy. Her research expertise includes software quality, empir- ical software engineering, and software process improvement. She has been carrying out research on qualitative and quanti- tative methods through international research collaborations.
Alessandra Milani is a PhD student in the Department of Computer Science at the University of Victoria, British Columbia, Canada. Her research focuses on software engi- neering and developer productivity, with a particular interest in improving collaboration tools and practices in software development teams.
Technical Briefing 3: Codellm-Devkit: A Framework To Contextualize
Code LLMs with Program Analysis Insights Tool Demo
Thursday May 1: 11:00 to 12:30
Presenters: Rahul Krishna and Rangeet Pan, IBM Research
Large Language Models for Code (or code LLMs) are increasingly gaining popularity and capabilities, offering a wide array of functionalities such as code completion, code generation, code summarization, test generation, code repair, refactoring, translation, and more. To leverage code LLMs to their full potential, developers must provide code-specific contextual information to the models. These are typically derived and distilled using program analysis tools. However, there exists a significant gap—these static analysis tools are often language-specific and come with a steep learning curve, making their effective use challenging. These tools are tailored to specific program languages, requiring developers to learn and manage multiple tools to cover various aspects of their code base. Moreover, the complexity of configuring and integrating these tools into the existing development environments adds an additional layer of difficulty. This challenge limits the potential benefits that could be gained from the more widespread and effective use of static analysis in conjunction with code LLMs.
In this tutorial, we present Codellm-Devkit (CLDK)—an open-source library that significantly simplifies the process of performing program analysis at various levels of granularity. As a Python-based library, CLDK offers developers an intuitive and user-friendly interface, making it incredibly easy to provide rich program analysis context to code LLMs. With this library, developers can effortlessly integrate detailed, code-specific insights that enhance the operational efficiency and effectiveness of LLMs in coding tasks. This hands-on session will enable participants to perform static analysis to build LLM-based solutions for coding tasks such as: (1) code generation, (2) code summarization, and (3) test generation across different programming languages. Through practical exercises, developers will gain hands-on experience in enhancing the functionality and applicability of code LLMs using CLDK’s APIs.
Learning objectives
- Understanding Static Analysis: Learn how to use off-the-shelf static analysis tools for multiple programming languages.
- LLM Integration: Explore how to apply these static analysis results to LLM-based code understanding, generation, or transformation.
- Tool Familiarity: Gain hands-on experience with WALA, Tree-sitter, and CodeQL within a unified framework.
- Cross-Language Code Analysis: Learn to handle multi-language projects with minimal overhead using the provided interface.
- Extensibility: Explore how to extend the devkit for custom analyses or models.
Outline
- Install CLDK and other pre-requisites.
- Understand the static analysis backend of CLDK.
- Understand API frontend of CLDK.
- Build a prototype for generating unit test cases for multiple programming languages using CLDK.
- Build a prototype for generating code summary for multiple programming languages using CLDK.
- Build a prototype for translating Java code to Python and validating the translated code using CLDK.
- Add a new functionality to CLDK.
Hands-on component
There will be hands-on sessions for each topic. Since CLDK is public and available in PyPi, attendees can install that directly and use it as any other Python library.
Bios of the presenters
Rangeet Pan is a Staff Research Scientist at IBM Research. His research interests are in the field of software engineering, deep learning, and program analysis. His current research works are related to incorporating program analysis while using Large Language Models (LLM) for various code-to-code tasks, e.g., code translation, test generation, and modernizing applications. His works are published in top-tier software engineering conferences, i.e., ICSE, ESEC/FSE, and PLDI. He has served as a program committee member at ESEC/FSE, ICSE, ASE, and OOPSLA.
Rahul Krishna is a Staff Research Scientist with an interest in AI-driven application modernization. He currently works on the intersection of AI and program analysis to guide application refactoring and testing. His research also explores the ways in which Machine Learning can be used to trigger, detect, and repair various problems in a software engineering pipeline. His works are published in top-tier software engineering conferences, i.e., ICSE, ESEC/FSE, and ASE. At ASE 2021, he organized a tutorial on decomposing monoliths to microservices with Mono2Micro.
Architecting Digital Twin-Based Systems
Friday May 2: 11:00 to 12:30
Presenter: Bedir Tekinerdogan, Wageningen University
Details TBD