14th International Workshop on Combinatorial Testing (IWCT 2025)IWCT 2025

About IWCT

The 14th International Workshop on Combinatorial Testing (IWCT 2025) is to be held in conjunction with ICST 2025, focusing on combinatorial methods in testing, which can target software, systems-of-systems or other domains. The workshop welcomes academic research submissions as well as industrial experience reports. We would like to highlight that this year, IWCT 2025 seeks to broaden its application domains of combinatorial methods used in general testing problems and therefore not only invites contributions from the software testing domain concerning - but not limited to - AI/ML-based, autonomous, concurrent or real-time systems, security of hard- and software, cloud computing and cloud architectures and software product lines, but also welcomes contributions on using combinatorial testing in other domains such as - including, but not limited to - biotechnology applications, mechanical engineering, manufacturing, quantum computing, finance and disaster management research.

Combinatorial Testing (CT), or Combinatorial Interaction Testing (CIT), is a widely applicable generic methodology and technology for software verification and validation, considered a testing best practice. In a combinatorial test plan, all interactions between parameters up to a certain level are covered. For example, in pairwise testing, for every pair of parameters, every pair of values will appear at least once. Studies show that CT is more efficient and effective than random testing of software.

CT has gained significant interest in recent years, both in research and in practice. However, many issues still remain unresolved, and much research is still needed in the field. For example, while pairwise testing is a well recognized and popular test planning method, investigations of actual failures in a number of software and systems convincingly show that pairwise testing is usually not sufficient, so higher strength CT (i.e., t-way for t > 2) may be needed.

In addition, the combinatorial test plans need to exclude invalid combinations of test values that cannot be executed, which limits the degrees of freedom the algorithms have, thus complicating the problem. Moreover, modeling languages and tools for easily capturing the input test space are also required for real-life applicability of CT. Other obstacles for wide acceptance of CT in industry are the gap between the generated test plans and executable tests, and the difficulty in determining expected results for the generated tests. Finally, empirical studies on CT, as well as thorough comparison with other methods, are also required.

In this workshop, we plan to bring together researchers actively working on combinatorial testing, and create a productive and creative environment for sharing and collaboration. IWCT being the primary venue dedicated to CT, yet there are many researchers working in the field, we expect, like in previous years, to see high responsiveness to take part in the workshop. Researchers attending the workshop will have an opportunity to publish their work in a dedicated venue, create new collaborations and take active part in the growing community of researchers working in the field.

The workshop will also be a meeting place between academia and industry, thus uniting academic excellence and industrial experience and needs. This will allow participants from academia to learn about the industrial experience in practical application of CT to real-life testing problems, and together with the colleagues from industry identify the difficulties that are obstacle to wider application of CT, and should be addressed in future research. Industrial participants will have an opportunity to meet the leading scientists in the field, and hear about the latest advances and innovations.

Please direct any questions concerning submissions or IWCT 2025 in general to: iwct2025@sba-research.org

Previous Editions of IWCT

We are proud of more than ten previous editions of IWCT, showing the maturity of the field and the support of our community:

13th IWCT 2024

12th IWCT 2023

11th IWCT 2022

10th IWCT 2021

9th IWCT 2020

8th IWCT 2019

7th IWCT 2018

6th IWCT 2017

5th IWCT 2016

4th IWCT 2015

3rd IWCT 2014

2nd IWCT 2013

1st IWCT 2012

Disclaimer: All text on the IWCT website has been carefully checked. Nevertheless, the organizers cannot guarantee the correctness, completeness and topicality of the information. Liability of the organizers is therefore excluded. The links to other websites have been carefully chosen. However, since the organizers have no say on their content, they assumes no responsibility for these.

Call for Papers

We invite submissions of high-quality papers presenting original work on both theoretical and practical aspects of applying combinatorial methods to testing problems, targeting software, systems-of-systems or other domains (including, but not limited to, biotechnology, mechanical engineering, manufacturing, quantum computing, finance and disaster research).

Each submission (in any category) must conform to the IEEE conference proceedings template, specified in the IEEE Conference Proceedings Formatting Guidelines (https://www.ieee.org/conferences/publishing/templates.html). All papers will be reviewed by at least three members from the program committee. If a submission is accepted, at least one author of the paper is required to attend the workshop in-person with an author registration and present the paper for it to be published in the ICST 2025 workshop proceedings.

Please direct any questions concerning submissions to IWCT 2025 to: iwct2025@sba-research.org

Submission

EasyChair instance of ICST 2025, please select track ‘IWCT 2025’: https://easychair.org/conferences/?conf=icst2025

Full and short papers

We invite submissions for full papers (up to 10 pages, including references) as well as short papers (up to 4 pages, including references). Accepted papers in these two categories – full and short – will be part of the proceedings of IWCT 2025.

Journal-First

The aim of the Journal-First (JF) submission category is to further enrich the program of IWCT, as well as to provide an overall more flexible path to publication and dissemination of original research that is within the scope of IWCT. A submission in this category must adhere to the following criteria:

1. It should be clearly within the scope of the workshop.
2. It should be recent: it should have been accepted and made publicly available in a journal (online or in print) by January 1, 2023 or more recently.
3. It has not been presented at, and is not under consideration for, Journal-First tracks of other conferences or workshops.
4. The submission has to be in the form of a 2-page extended abstract and has to provide a concise summary of the published journal paper.

Journal-First submissions must be marked as such in the submission’s title, and must explicitly include full bibliographic details (including a DOI) of the journal publication they are based on. Since the referenced journal papers have already been reviewed and accepted by the corresponding journals, submissions in the JF category will not be reviewed again for technical content. Submissions will be judged on the basis of the above criteria, but also considering how well they would complement the workshop’s technical program. Accepted submissions in this category will be part of the proceedings of IWCT 2025, with the title equal to the original title of the article with the prefix “Summary of”.

Posters

In addition to the traditional program topics from the past years, and taking advantage of the fact that some of the lead CT tool developers have expressed interest in participating again in the fourth iteration of the CT competition, we would like to offer with the poster session a low-threshold opportunity – especially targeting industry – to present activities within the scope of the workshop to the participants of IWCT. A tool demonstration can be given alongside a poster. In case a poster contribution wishes to provide more details or information, but does not want to submit a short or full paper, the authors can choose to submit in addition to the poster a maximal 2-page extended abstract detailing the content of the poster. The title of the extended abstract has to have the prefix “Extended Abstract of Poster” and these extended abstracts – if accepted – will also appear in the proceedings of IWCT 2025.

Post-proceedings of IWCT 2025 in Section VI of Journal SN Computer Science

We plan to organize a call among the best papers of the workshop to submit extended versions to the journal SN Computer Science in the Section ‘Combinatorial Methods and Models in System Testing’ (https://resource-cms.springernature.com/springer-cms/rest/v1/content/23087782/data/v2). More information will be provided at the workshop.

Topics for submissions

Topics of interest for submissions include, but are not limited to:

Combinatorial testing workflow

• Modeling the input space for CT – the input to CT algorithms consists of the desired strength of coverage, a set of parameters together with their respective values, and constraints on value combinations. This input should capture correctly the points of variability in testing the system. While this input is clearly crucial for the effectiveness of CT, it is a difficult problem.
• Efficient algorithms to generate test plans with small size for t-way testing for t > 2, involving support of constraints on combinations that are possible.
• Determination of expected system behavior for each test case – while the test cases are automatically generated by the CT algorithm, determining the expected system behavior is currently usually a manual task.
• Executing CT test plans – the result of CT algorithms is a list of tests, where a test is represented by a value assignment for each parameter. There is still significant effort in transforming this representation into actual tests that a tester or testing tool can execute.
• Combinatorial testing based fault localization.
• Implementation of CT in existing testing infrastructures.
• Handling changes in test requirements – current CT methods focus on one-time generation of a test plan from a given set of requirements, but test requirements are almost never static, and tend to change between different releases and versions. However, generating a new plan of tests for each release may not be practical either.

Real-world experience in deployment of combinatorial testing

• Empirical studies and feedback from practical applications of CT.
• CT targeting AI/ML-based systems.
• CT targeting security, including hardware and chip security.
• CT targeting autonomous, concurrent and real-time systems.
• CT targeting cloud computing systems and use of combinatorial methods in cloud architecture.
• CT targeting feature models for software product lines.
• CT targeting other domains; e.g., study of gene regulations and other biotechnology applications, mechanical engineering, testing in manufacturing hardware, quantum computing domain, financial domain or disaster management.

Applicability of combinatorial testing

• Comparison and combination of CT with other dynamic verification methods.
• Investigation of historical records of failures to determine the kind of CT which may have detected underlying faults.
• Methodology used for test space modeling and determination of interaction coverage requirements.
• Evaluation and return of investment metrics to assess the degree of usefulness of CT.
• Discussion of challenges and open problems in the application of CT in an industrial setting.

Combinatorial and complementing methods

• Combinatorial analysis of existing test plans – analyzing test plans not generated by a CT algorithm in light of a combinatorial test space.
• Test plan reduction and completeness – both under stable, and under changing test requirements.
• CT and coverage metrics – combining the two, and studying the relation between them.

Disclaimer: The links to websites have been carefully chosen. However, since the organizers have no say on their content, they assumes no responsibility for these.