The 9th International Workshop on Testing Extra-Functional Properties and Quality Characteristics of Software Systems

SCOPE

The rapid development towards increased integration of software with the social and physical world that we see today means that quality aspects such as performance, safety, security, and robustness become more important in an increasing number of the systems and devices, which we use and depend on. In this context, the success of a software product may not only depend on the logical correctness of its functions, but also on the system quality characteristics. Such system characteristics, which are referred to and captured as Extra-Functional Properties (EFPs) or Non-Functional Properties, are particularly important in resource constrained systems such as in the domains of real-time embedded and cyber-physical systems. Therefore, such systems need to be tested with a special attention to the EFPs. Testing EFPs is challenging and often requires different approaches compared to testing normal functionality. ITEQS provides a focused forum with the goal of bringing together researchers and practitioners to share ideas, identify challenges, propose solutions and techniques, and in general expand the state of the art in testing EFPs and quality characteristics of software systems and services. The workshop endorses contributions in a wide range of topics related to testing of EFPs in the form of full papers and short yet solid work-in-progress/position papers.

Note: The workshop does not accept papers that focus purely on functional testing!

Check out the Call for papers for the list of topics and more details.

Contact

Muhammad Abbas, RISE Research Institutes of Sweden, Sweden (muhammad.abbas[at]ri.se)

Mahshid Helali Moghadam, Scania CV AB, Sweden (mahshid.helali.moghadam[at]scania.com)

Fitash Ul Haq Luxembourg Institute of Science and Technology, Luxembourg(fitash.ulhaq[at]list.lu)

Call for Papers

9th International Workshop on Testing Extra-Functional Properties and Quality Characteristics of Software Systems (ITEQS) Co-located with the 18th IEEE International Conference on Software Testing, Verification and Validation (ICST 2025)

Scope: In today’s rapidly evolving technological landscape, we are witnessing an unprecedented integration of software into our social and physical worlds. This integration has elevated the significance of quality aspects in the systems and devices we rely on daily. In this context, the success of a software product is no longer solely dependent on the logical correctness of its functions but also its quality characteristics. Such system characteristics, which are referred to and captured as Extra-Functional Properties (EFPs), have determinant importance, particularly in resource-constrained systems, such as real-time embedded domains, cyber-physical systems, Internet of Things (IoT) devices, and edge-based services. Therefore, such systems need to be tested with special attention to EFPs such as performance, safety, security, robustness, scalability, and energy efficiency. Testing EFPs presents unique challenges that often require approaches distinct from functional testing methodologies. These challenges stem from the complex, often interdependent nature of EFPs and their sensitivity to environmental factors. ITEQS provides a well-focused forum with the goal of bringing together researchers and practitioners to share ideas, identify challenges, propose solutions and techniques, and, in general, expand the state of the art in testing EFPs and quality characteristics of software systems and services. The workshop endorses contributions on a wide range of topics related to the testing of EFPs in the form of short position/work-in-progress and full papers.

Note: The workshop does not accept papers that focus purely on functional testing!

TOPICS

Topics of interest include, but are not limited to the following:

• Model-based testing of EFPs; e.g., choice of modeling languages to capture EFPs and their role on testability, model-based test case generation, etc.
• Testing for fairness, transparency, and bias in AI/ML systems, especially for LLMs
• Performance, Robustness, and Security Testing
• Validation and Verification of AI-based systems wrt to properties such as robustness, reliability, safety
• Mutation-based testing for EFPs; e.g., application of mutation techniques for testing of EFPs particularly introduction of EFP-specific mutation operators
• Testing quality attributes in software product lines and variant-intensive system
• Testing quality characteristics of distributed, mobile, and cloud applications
• Search-based testing techniques for EFPs
• Testability, observability, controllability and the role of the platform; e.g., how the choice of operating system can impact testability of EFPs, for instance, a real-time operating system, introducing testability mechanisms into a platform, designing - middlewares for testing of EFPs
• Empirical studies and experience reports; e.g., on the importance of testing EFPs, evaluation of testing methods, case-study and reports on project failures due to EFPs, comparison of methods and techniques
• Quality assurance, standards, and their impact on testing EFPs
• Requirements and testing EFPs; e.g., identification and generation of test oracles for EFPs from requirements, requirements for testability, traceability
• Coverage criteria in testing EFPs
• Processes and their role in testing EFPs; e.g., agile and TDD
• Fault localization for EFPs and debugging
• Formal methods, model-checking, and reasoning about EFPs
• Parallelism, Concurrency, and Testing of multicore applications
• Testing real-time, embedded, and cyber-physical systems, and their challenges
• Static analysis for evaluation of EFPs;
• Continuous Integration and Continuous Deployment (CI/CD) in relation to EFP testing
• Quantum computing implications for EFP testing;

SUBMISSION GUIDELINES

Submissions must conform to the IEEE conference proceedings template, specified in the IEEE Conference Proceedings Formatting Guidelines.

Paper formats:

• 1) full papers: 6-10 pages
• 2) solid work-in-progress and position papers: 4 pages

in IEEE double column format. Submission site: https://easychair.org/conferences/?conf=icst2025 .

Select the “The 9th International Workshop on Testing Extra-Functional Properties and Quality Characteristics of Software Systems” track on the submission page.

Accepted papers will be published in IEEE Digital Library.

• 2024 - Toronto, Canada
• 2023 - Dublin, Ireland
• 2022 - Virtual
• 2021 - Virtual
• 2020 - Porto, Portugal
• 2019 - Xi’an, China
• 2018 - Västerås - Sweden
• 2017 - Tokyo - Japan