ICST 2025
Mon 31 March - Fri 4 April 2025 Naples, Italy
Fri 4 Apr 2025 11:00 - 11:15 at Aula Magna (AM) - Automated Testing Chair(s): Cristian Cadar

As the popularity of Internet of Things (IoT) platforms grows, users gain unprecedented control over their homes, health monitoring, and daily task automation. However, the testing of software for these platforms poses significant challenges due to their diverse composition, \eg common smart home platforms are often composed of varied types of devices that use a diverse array of communication protocols, connections to mobile apps, cloud services, as well as integration among various platforms. This paper is the first to uncover both the practices and perceptions behind testing in IoT platforms, particularly open-source smart home platforms. Our study is composed of two key components. First, we mine and empirically analyze the code and integrations of two highly popular and well maintained open-source IoT platforms, \openhab and \homeassistant. Our analysis involves the identification of functional and related test methods based on the \textit{focal method approach}. We find that \openhab has only $0.04$ test ratio ($\approx 4K$ focal test methods from $\approx 76K$ functional methods) in Java files, while \homeassistant exhibits a higher test ratio of $0.42$, which reveals a significant dearth of testing. Second, to understand the developers’ perspective on testing in IoT, and to explain our empirical observations, we survey 80 open-source developers actively engaged in IoT platform development. Our analysis of survey responses reveals a significant focus on automated (unit) testing, and a lack of manual testing, which supports our empirical observations, as well as testing challenges specific to IoT. Together, our empirical analysis and survey yield 10 key findings that uncover the current state of testing in IoT platforms, and reveal key perceptions and challenges. These findings provide valuable guidance to the research community in navigating the complexities of effectively testing IoT platforms.

Fri 4 Apr

Displayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

11:00 - 12:30
Automated TestingIndustry / Research Papers / Journal-First Papers / Education at Aula Magna (AM)
Chair(s): Cristian Cadar Imperial College London
11:00
15m
Talk
Testing Practices, Challenges, and Developer Perspectives in Open-Source IoT Platforms
Research Papers
Daniel Rodriguez-Cardenas William & Mary, Safwat Ali Khan George Mason University, Prianka Mandal William & Mary, Adwait Nadkarni William & Mary, Kevin Moran University of Central Florida, Denys Poshyvanyk William & Mary
Pre-print
11:15
15m
Talk
Many-Objective Neuroevolution for Testing Games
Research Papers
Patric Feldmeier University of Passau, Katrin Schmelz University of Passau, Gordon Fraser University of Passau
Pre-print
11:30
15m
Talk
Black-Box Testing for Practitioners
Education
Matthias Hamburg IEEE Computer Society; International Software Testing Qualifications Board, Adam Roman Jagiellonian University, Faculty of Mathematics and Computer Science; International Software Testing Qualifications Board
11:45
15m
Talk
CUBETESTERAI: Automated JUnit Test Generation using the LLaMA Model
Industry
Daniele Gorla Department of Computer Science, Sapienza University of Rome, Shivam Kumar , Pietro Nicolaus Roselli Lorenzini , Alireza Alipourfaz
12:00
15m
Talk
Can Search-Based Testing with Pareto Optimization Effectively Cover Failure-Revealing Test Inputs?
Journal-First Papers
Lev Sorokin Technische Universität München, Germany, Damir Safin fortiss, Shiva Nejati University of Ottawa
12:15
15m
Talk
[prerecorded] ADGE: Automated Directed GUI Explorer for Android Applications
Research Papers
Yue Jiang Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China, Xiaobo Xiang Singular Security Lab, Beijing, China, Qingli Guo Institute of Information Engineering, Chinese Academy of Sciences, Qi Gong Key Laboratory of Network Assessment Technology, Institute of Information Engineering, Chinese Academy of Sciences, China, Xiaorui Gong Institute of Information Engineering, Chinese Academy of Science
:
:
:
: