The 10th International Workshop on Testing Extra-Functional Properties and Quality Characteristics of Software Systems

SCOPE

The rapid development towards increased integration of software with the social and physical world that we see today means that quality aspects such as performance, safety, security, and robustness become more important in an increasing number of the systems and devices, which we use and depend on. In this context, the success of a software product may not only depend on the logical correctness of its functions, but also on the system quality characteristics. Such system characteristics, which are referred to and captured as Extra-Functional Properties (EFPs) or Non-Functional Properties, are particularly important in resource constrained systems such as in the domains of real-time embedded and cyber-physical systems. Therefore, such systems need to be tested with a special attention to the EFPs. Testing EFPs is challenging and often requires different approaches compared to testing normal functionality. ITEQS provides a focused forum with the goal of bringing together researchers and practitioners to share ideas, identify challenges, propose solutions and techniques, and in general expand the state of the art in testing EFPs and quality characteristics of software systems and services. The workshop endorses contributions in a wide range of topics related to testing of EFPs in the form of full papers and short yet solid work-in-progress/position papers.

Note: The workshop does not accept papers that focus purely on functional testing!

Check out the Call for papers for the list of topics and more details.

Contact

iteqs2026[at]easychair.org

Mehrdad Saadatmand, RISE Research Institutes of Sweden, Sweden (mehrdad.saadatmand[at]ri.se)

Mahshid Helali Moghadam, Traton AB, Sweden (mahshid.helali.moghadam[at]se.traton.com)

Fitash Ul Haq, Luxembourg Institute of Science and Technology, Luxembourg (fitash.ulhaq[at]list.lu)

Tanwir Ahmad, Åbo Akademi University, Finland (tanwir.ahmad[at]abo.fi)

Accepted Papers

	Title
	Comp-AFL: Towards Complete Fuzzing ITEQS A: Sangharatna Godboley, A: Kanika Gupta, A: Golla Monika Rani, A: P. Radha Krishna
	Efficient Software Security Evaluation: A Human-in-the-Loop Approach ITEQS A: Christian Banse, A: Immanuel Kunz, A: Alexander Küchler, A: Shala Leutrim, A: Konrad Weiss, A: Maximilian Kaul
	Friends or Foes? Combining Static Analysis Tools and LLMs for Vulnerability Detection ITEQS A: Rafael Ramires, A: Sarmad Bashir, A: Muhammad Abbas Khan, A: Mehrdad Saadatmand, A: Ibéria Medeiros Pre-print
	ITEQS 2026 Opening ITEQS Mehrdad Saadatmand, Fitash Ul Haq
	SafeBound: A Modular Toolchain for End-to-End Safety Evaluation of ADS ITEQS A: Fauzia Khan, A: Ali Gullu, A: Hina Anwar, A: Dietmar Pfahl
	Test Design and Review Argumentation in AI-Assisted Test Generation ITEQS A: Eduard Paul Enoiu, A: Robert Feldt
	Verifying an Elevator Scheduling Control System ITEQS A: HUAN ZHANG, A: Haoyang Lu, Long Cheng, A: Hao Wu

Call for Papers

10th International Workshop on Testing Extra-Functional Properties and Quality Characteristics of Software Systems (ITEQS) Co-located with the 19th IEEE International Conference on Software Testing, Verification and Validation (ICST 2026)

Scope: In today’s rapidly evolving technological landscape, we are witnessing an unprecedented integration of software into our social and physical worlds. This integration has elevated the significance of quality aspects in the systems and devices we rely on daily. In this context, the success of a software product is no longer solely dependent on the logical correctness of its functions but also its quality characteristics. Such system characteristics, which are referred to and captured as Extra-Functional Properties (EFPs), have determinant importance, particularly in resource-constrained systems, such as real-time embedded domains, cyber-physical systems, Internet of Things (IoT) devices, and edge-based services. Therefore, such systems need to be tested with special attention to EFPs such as performance, safety, security, robustness, scalability, and energy efficiency. Testing EFPs presents unique challenges that often require approaches distinct from functional testing methodologies. These challenges stem from the complex, often interdependent nature of EFPs and their sensitivity to environmental factors. ITEQS provides a well-focused forum with the goal of bringing together researchers and practitioners to share ideas, identify challenges, propose solutions and techniques, and, in general, expand the state of the art in testing EFPs and quality characteristics of software systems and services. The workshop endorses contributions on a wide range of topics related to the testing of EFPs in the form of short position/work-in-progress and full papers.

Note: The workshop does not accept papers that focus purely on functional testing!

TOPICS

Topics of interest include, but are not limited to the following:

1) Methods and foundations for testing EFPs / quality characteristics:

• Model-based testing of EFPs
• Mutation-based testing for EFPs
• Search-based and AI-assisted techniques for testing EFPs
• Coverage criteria and test-stop criteria in EFP testing
• Requirements engineering for EFPs and testable EFP specifications
• Testability, observability, and the role of the platform
• Fault localization, debugging, and diagnosis for EFP violations
• Static analysis for evaluation of EFPs
• Formal methods, model checking, and reasoning about EFPs
• Empirical studies, benchmarks, and experience reports on EFP testing

2) EFP testing of modern platforms and domains:

• Testing real-time, embedded, and cyber-physical systems and their challenges
• Testing of multicore applications
• Testing quality characteristics of distributed, mobile, cloud, and edge applications
• Testing quality attributes in software product lines and variant-intensive systems
• Continuous Integration and Continuous Deployment (CI/CD) in relation to EFP testing
• Quality assurance processes, standards, and their impact on testing EFPs

3) Trustworthy AI/ML quality properties and compliance:

• Testing AI and ML systems for quality attributes (robustness, reliability, security/privacy, fairness, transparency, bias)
• Testing for fairness, transparency, and bias in AI/ML systems, especially for LLMs
• Compliance-oriented testing and assurance (e.g., regulatory or standards-driven)

4) Emerging platforms/toolchains (e.g., quantum, low-code/no-code) when tied to EFP testing methods or evidence.

SUBMISSION GUIDELINES

Submissions must conform to the IEEE conference proceedings template, specified in the IEEE Conference Proceedings Formatting Guidelines.

Paper formats:

• 1) full papers: 6-10 pages
• 2) solid work-in-progress and position papers: 4 pages

in IEEE double column format. Submission site: https://easychair.org/conferences/?conf=icst2026 .

Accepted papers will be published in IEEE Digital Library.

Title: Language in the Loop, Promise and Risk in Verifying Intelligent Cyber-Physical Systems

Track

ITEQS 2026

Abstract

Artificial intelligence (AI) is reshaping cyber-physical systems (CPS) at a remarkable pace. Modern CPS increasingly rely on AI-enabled components that reason, plan, and execute with limited human intervention while interacting continuously with uncertain physical environments. But this shift has not made engineering easier. It has made verification and validation more challenging. Failures now arise from subtle interactions among learned behavior, software logic, environment assumptions, and continuous dynamics, often in ways that are difficult to test, explain, or characterize precisely. Safety testing, debugging, and maintaining the safety of evolving systems have therefore become central concerns. Yet the fundamental question remains the same: how do we build systems that work safely and correctly in the world they are meant to operate in? In this keynote, I will argue that Language in the Loop offers a promising direction, not as a substitute for rigorous engineering, but as a disciplined layer within it. I will discuss how large language models (LLMs) can support CPS verification by guiding test generation, producing debugging explanations, and assisting the restoration of safety operational rules. The promise is clear; these methods can help engineers hypothesize, explain, and repair more effectively. But the risks are equally real; syntactic invalidity, semantically unjustified outputs, and misplaced trust in LLMs. Through concrete examples from recent work and the broader literature on debugging, specification mining, and language-guided formal reasoning for safety-critical systems, I will argue that the software testing community should move beyond building new tools alone and develop principled ways to evaluate Language-in-the-Loop methods, with clear map for assessing their validity and truthfulness in the verification of intelligent CPS.

Bio: Khouloud Gaaloul is an Assistant Professor in the Department of Computer and Information Science at the University of Michigan Dearborn. Her research advances software engineering for cyber physical systems, with a focus on AI for software engineering, automated verification, and formal methods. Her work is supported by the National Science Foundation through a CRII grant on temporal logic property falsification for AI enabled cyber physical systems. Her research has been published in leading venues, including IEEE Transactions on Software Engineering, ACM Foundations of Software Engineering, and the IEEE International Conference on Software Testing, Verification and Validation (ICST).

Khouloud GaaloulKeynote Speaker

University of Michigan-Dearborn

• 2025 - Naples, Italy
• 2024 - Toronto, Canada
• 2023 - Dublin, Ireland
• 2022 - Virtual
• 2021 - Virtual
• 2020 - Porto, Portugal
• 2019 - Xi’an, China
• 2018 - Västerås - Sweden
• 2017 - Tokyo - Japan