ICST 2023
Sun 16 - Thu 20 April 2023 Dublin, Ireland

alt text

The 7th International Workshop on Testing Extra-Functional Properties and Quality Characteristics of Software Systems


SCOPE

The rapid development towards increased integration of software with the social and physical world that we see today means that quality aspects such as performance, safety, security, and robustness become more important in an increasing number of the systems and devices, which we use and depend on. In this context, the success of a software product may not only depend on the logical correctness of its functions, but also on the system quality characteristics. Such system characteristics, which are referred to and captured as Extra-Functional Properties (EFPs) or Non-Functional Properties, are particularly important in resource constrained systems such as in the domains of real-time embedded and cyber-physical systems. Therefore, such systems need to be tested with a special attention to the EFPs. Testing EFPs is challenging and often requires different approaches compared to testing normal functionality. ITEQS provides a focused forum with the goal of bringing together researchers and practitioners to share ideas, identify challenges, propose solutions and techniques, and in general expand the state of the art in testing EFPs and quality characteristics of software systems and services. The workshop endorses contributions in a wide range of topics related to testing of EFPs in the form of full papers and short yet solid work-in-progress/position papers.

Note: The workshop does not accept papers that focus purely on functional testing!

TOPICS

  • Model-based testing of EFPs; e.g., choice of modeling languages to capture EFPs and their role on testability, model-based test case generation, etc.
  • Performance, Robustness, and Security Testing
  • Testing of AI and Machine Learning based systems with respect to quality attributes; e.g., such as safety, security, and ethical concerns
  • Mutation-based testing for EFPs; e.g., application of mutation techniques for testing of EFPs particularly introduction of EFP-specific mutation operators
  • Testing quality in software product lines and variant-intensive system
  • Testing quality characteristics of distributed, mobile, and cloud applications
  • Search-based testing techniques for EFPs
  • Testability, observability, controllability and the role of the platform; e.g., how the choice of operating system can impact testability of EFPs, for instance, a real-time operating system, introducing testability mechanisms into a platform, designing - middlewares for testing of EFPs
  • Empirical studies and experience reports; e.g., on the importance of testing EFPs, evaluation of testing methods, case-study and reports on project failures due to EFPs, comparison of methods and techniques
  • Quality assurance, standards, and their impact on testing EFPs
  • Requirements and testing EFPs; e.g., identification and generation of test oracles for EFPs from requirements, requirements for testability, traceability
  • Coverage criteria in testing EFPs
  • Processes and their role in testing EFPs; e.g., agile and TDD
  • Fault localization for EFPs and debugging
  • Formal methods, model-checking, and reasoning about EFPs
  • Parallelism, Concurrency, and Testing of multicore applications
  • Testing real-time, embedded, and cyber-physical systems, and their challenges

Official Workshop Website

Contact

  • iteqs2023[at]easychair.org

Mehrdad Saadatmand, RISE Research Institutes of Sweden, Sweden (mehrdad.saadatmand[at]ri.se)

Eduard Enoiu, Mälardalen University, Sweden (eduard.paul.enoiu[at]mdh.se)

Dragos Truscan, Åbo Akademi University, Finland (dragos.truscan[at]abo.fi)

Plenary
You're viewing the program in a time zone which is different from your device's time zone change time zone

Sun 16 Apr

Displayed time zone: Dublin change

09:00 - 10:30
ITEQS IITEQS at Macken
09:00
5m
Talk
ITEQS Opening
ITEQS

09:05
55m
Keynote
Keynote: Learning-based Testing of Robustness and Time
ITEQS
K: Bernhard Aichernig Graz University of Technology
10:00
30m
Talk
Automated and Systematic Digital Twins Testing for Industrial Processes
ITEQS
Yunpeng Ma Karlstads Universitet, Khalil Younis Karlstads Universitet, Bestoun S. Ahmed Karlstad University, Andreas Kassler Karlstad University, Pavel Krakhmalev Karlstad University, Andreas Thore RISE Research Institutes of Sweden, Hans Lindback Bharat Forge Kilsta AB
Pre-print
10:30 - 11:00
Coffee breakSocial
10:30
30m
Coffee break
Break
Social

11:00 - 12:30
ITEQS IIITEQS at Macken
11:00
30m
Talk
Preliminary results in using attention for increasing attack identification efficiency
ITEQS
Tanwir Ahmad Åbo Akademi University, Dragos Truscan Åbo Akademi University, Jüri Vain Tallinn University of Technology, Estonia
11:30
30m
Talk
Lightweight Method for On-the-fly Detection of Multivariable Atomicity Violations (Best Paper)Best Paper Award
ITEQS
Changhui Bae Gyeongsang National University, Euteum Choi Gyeongsang National Unviersity, Yong-Kee Jun Gyeongsang National University, Ok-Kyoon Ha Kyungwoon University
12:00
30m
Talk
Using Assurance Cases to assure the fulfillment of non-functional requirements of AI-based systems - Lessons learned
ITEQS
Marc Hauer TU Kaiserslautern - Algorithm Accountability Lab , Lena Müller-Kress winnovation consulting gmbh, Gertraud Leimüller winnovation consulting gmbh & leiwand.ai gmbh, Katharina Zweig TU Kaiserslautern - Algorithm Accountability Lab
12:30 - 14:00
LunchSocial
12:30
90m
Lunch
Lunch
Social

14:00 - 15:30
ITEQS IIIITEQS at Macken
14:00
30m
Talk
Bottleneck Analysis via Grammar-based Performance Fuzzing
ITEQS
Yavuz Koroglu Graz University of Technology, Franz Wotawa Graz University of Technology
14:30
30m
Talk
Test Generation and Mutation Analysis of Energy Consumption using UPPAAL SMC and MATS
ITEQS
Jonatan Larsson Mälardalen University, Eduard Paul Enoiu Mälardalen University
15:00
10m
Talk
Best paper award and closing
ITEQS

15:30 - 16:00
Coffee breakSocial
15:30
30m
Coffee break
Break
Social

Call for Papers

7th International Workshop on Testing Extra-Functional Properties and Quality Characteristics of Software Systems (ITEQS) Co-located with the 16th IEEE International Conference on Software Testing, Verification and Validation (ICST 2023)

Scope: The rapid development towards increased integration of software with the social and physical world that we see today means that quality aspects such as performance, safety, security, and robustness become more important in an increasing number of the systems and devices, which we use and depend on. In this context, the success of a software product may not only depend on the logical correctness of its functions, but also on the system quality characteristics. Such system characteristics, which are referred to and captured as Extra-Functional Properties (EFPs) or Non-Functional Properties, are particularly important in resource constrained systems such as in the domains of real-time embedded and cyber-physical systems. Therefore, such systems need to be tested with a special attention to the EFPs. Testing EFPs is challenging and often requires different approaches compared to testing normal functionality. ITEQS provides a focused forum with the goal of bringing together researchers and practitioners to share ideas, identify challenges, propose solutions and techniques, and in general expand the state of the art in testing EFPs and quality characteristics of software systems and services. The workshop endorses contributions in a wide range of topics related to testing of EFPs in the form of full papers and short yet solid work-in-progress/position papers.

Note: The workshop does not accept papers that focus purely on functional testing!

SUBMISSION GUIDELINES

Paper formats:

  • 1) full papers: 6-10 pages
  • 2) solid work-in-progress and position papers: 4 pages

in IEEE double column format. Submission site: https://easychair.org/conferences/?conf=iteqs2023 Accepted papers will be published in IEEE Digital Library.

TOPICS

  • Model-based testing of EFPs; e.g., choice of modeling languages to capture EFPs and their role on testability, model-based test case generation, etc.
  • Performance, Robustness, and Security Testing
  • Testing of AI and Machine Learning based systems with respect to quality attributes; e.g., such as safety, security, and ethical concerns
  • Mutation-based testing for EFPs; e.g., application of mutation techniques for testing of EFPs particularly introduction of EFP-specific mutation operators
  • Testing quality in software product lines and variant-intensive system
  • Testing quality characteristics of distributed, mobile, and cloud applications
  • Search-based testing techniques for EFPs
  • Testability, observability, controllability and the role of the platform; e.g., how the choice of operating system can impact testability of EFPs, for instance, a real-time operating system, introducing testability mechanisms into a platform, designing - middlewares for testing of EFPs
  • Empirical studies and experience reports; e.g., on the importance of testing EFPs, evaluation of testing methods, case-study and reports on project failures due to EFPs, comparison of methods and techniques
  • Quality assurance, standards, and their impact on testing EFPs
  • Requirements and testing EFPs; e.g., identification and generation of test oracles for EFPs from requirements, requirements for testability, traceability
  • Coverage criteria in testing EFPs
  • Processes and their role in testing EFPs; e.g., agile and TDD
  • Fault localization for EFPs and debugging
  • Formal methods, model-checking, and reasoning about EFPs
  • Parallelism, Concurrency, and Testing of multicore applications
  • Testing real-time, embedded, and cyber-physical systems, and their challenges

Contact: iteqs2023[at]easychair.org

Keynote 1: Learning-based Testing of Robustness and Time

This talk will explore learning-based testing techniques, with a focus on automata learning, that can verify robustness and timing requirements of complex software systems. Traditional testing approaches can be inadequate in detecting subtle errors and corner cases, leading to unexpected system behavior and failures. By leveraging machine learning and automata-based techniques, high-quality test cases can be generated, subtle bugs can be detected, and timing constraints can be verified. We we will discuss our research and present case studies in fuzzing and model learning of IOT protocols (Bluetooth LE, MQTT), automotive controllers, and web-services.

Questions? Use the ITEQS contact form.