ICST 2023
Sun 16 - Thu 20 April 2023 Dublin, Ireland

ICST 2023 (https://conf.researchr.org/home/icst-2023) invites high-quality submissions in all areas of software testing, verification, and validation. Papers for the research track should present novel and original work that advances the state-of-the-art. Case studies and empirical research papers are also welcome.

Supporters
Diamond Sponsor
Platinum Sponsor
Platinum Sponsor
Gold Sponsor
Silver Sponsor
Bronze Sponsor
Dates
Mon 17 Apr 2023
Tue 18 Apr 2023
Wed 19 Apr 2023
Tracks
ICST Industry
ICST Journal-First Papers
ICST MIPs
ICST Posters
ICST Previous Editions
ICST Research Papers
ICST Social
ICST Testing Tools
ICST Tool Demo
Plenary
Hide plenary sessions
You're viewing the program in a time zone which is different from your device's time zone change time zone

Mon 17 Apr

Displayed time zone: Dublin change

11:00 - 12:30
Session 1: Automated Testing Journal-First Papers / Research Papers / Previous Editions / Testing Tools / Tool Demo at Grand canal
Chair(s): Gilles Perrouin Fonds de la Recherche Scientifique - FNRS & University of Namur
11:20
20m
Talk
Metamorphic Testing with Causal Graphs
Research Papers
Andrew Graham Clark University of Sheffield, Michael Foster University of Sheffield, Neil Walkinshaw University of Sheffield, Robert Hierons University of Sheffield
11:00 - 12:30
Session 2: Autonomous & Cyberphysical Systems IResearch Papers / Journal-First Papers at Pearse suite
Chair(s): Shaukat Ali Simula Research Laboratory
11:20
20m
Talk
A Coverage-Driven Systematic Test Approach for Simultaneous Localization and Mapping
Research Papers
Philip Tasche University of Twente, Paula Herber University of Munster, Germany
11:40
20m
Talk
Property-Based Mutation Testing
Research Papers
Ezio Bartocci Technische Universität Wien, Leonardo Mariani University of Milano-Bicocca, Dejan Nickovic Austrian Institute of Technology, Drishti Yadav Technische Universität Wien
12:00
20m
Talk
Constraint-Guided Automatic Side Object Placement for Steering Control Testing in Virtual Environment
Research Papers
14:00 - 15:30
Session 3: Autonomous & Cyberphysical Systems IIPrevious Editions / Industry / Research Papers at Grand canal
Chair(s): Fabrizio Pastore University of Luxembourg
15:00
20m
Talk
Simulation-based Test Case Generation for Unmanned Aerial Vehicles in the Neighborhood of Real Flights
Research Papers
Sajad Khatiri USI-Lugnao & Zurich University of Applied Sciences, Sebastiano Panichella Zurich University of Applied Sciences, Paolo Tonella USI Lugano
Pre-print
14:00 - 15:30
Session 4: Fault Localization & DebuggingResearch Papers / Tool Demo / Industry / Journal-First Papers at Pearse suite
Chair(s): Shin Yoo KAIST
14:40
20m
Talk
A Case Against Coverage-Based Program Spectra
Research Papers
Péter Attila Soha Department of Software Engineering, University of Szeged, Tamás Gergely Department of Software Engineering, University of Szeged, Ferenc Horv�th University of Szeged, Department of Software Engineering, Béla Vancsics Department of Software Engineering, University of Szeged, Árpád Beszédes Department of Software Engineering, University of Szeged
16:00 - 18:00
Session 5: Testing AI/ML systemsResearch Papers / Previous Editions at Grand canal
Chair(s): Jie M. Zhang King's College London
16:40
20m
Talk
Distributed Repair of Deep Neural Networks
Research Papers
Davide Li Calsi Politecnico di Milano, Matias Duran National Institute of Informatics, Xiao-Yi Zhang School of Computer and Communication Engineering, University of Science and Technology Beijing, Paolo Arcaini National Institute of Informatics , Fuyuki Ishikawa National Institute of Informatics
17:00
20m
Talk
Mutation Testing of Deep Reinforcement Learning Based on Real Faults
Research Papers
Florian Tambon Polytechnique Montréal, Vahid Majdinasab Polytechnique Montréal, Amin Nikanjam École Polytechnique de Montréal, Foutse Khomh Polytechnique Montréal, Giuliano Antoniol Polytechnique Montréal
Pre-print
17:20
20m
Talk
Repairing DNN Architecture: Are We There Yet?
Research Papers
Jinhan Kim KAIST, Nargiz Humbatova USI Lugano, Gunel Jahangirova King's College London, Paolo Tonella USI Lugano, Shin Yoo KAIST
Pre-print
16:00 - 18:00
Session 7: Testing for Safery & Security Industry / Research Papers / Journal-First Papers / Previous Editions at Hanover
Chair(s): Eric Bodden Heinz Nixdorf Institut, Paderborn University and Fraunhofer IEM
17:00
20m
Talk
Heap Fuzzing: Automatic Garbage Collection Testing with Directed Random Events
Research Papers
Guillermo Polito Inria, Cristal, UMR 9189, Université de Lille, Pablo Tesone Univ. Lille, Inria, CNRS, Centrale Lille, UMR 9189 CRIStAL, Pharo Consortium, Jean Privat Université du Québec à Montréal (UQAM), Nahuel Palumbo Université Lille, CNRS, Centrale Lille, Inria, UMR 9189 - CRIStAL, Stéphane Ducasse Inria; University of Lille; CNRS; Centrale Lille; CRIStAL
17:20
20m
Talk
MagicMirror: Towards High-Coverage Fuzzing of Smart Contracts
Research Papers
Huadong Feng University of Texas at Arlington, Xiaolei Ren University of Texas at Arlington, Qiping Wei University of Texas at Arlington, Jeff Yu Lei University of Texas at Arlington, Raghu Kacker National Institute of Standards and Technology, Richard Kuhn National Institute of Standards and Technology, Dimitris Simos SBA Research
16:00 - 18:00
Session 6: GUI/API testingTesting Tools / Previous Editions / Research Papers / Posters at Pearse suite
Chair(s): Phil McMinn University of Sheffield
16:20
20m
Talk
RIDA: Cross-App Record and Replay for Android
Research Papers
Jiayuan Liang Southern University of Science and Technology, Sinan Wang Southern University of Science and Technology, Xiangbo Deng Southern University of Science and Technology, Yepang Liu Southern University of Science and Technology

Tue 18 Apr

Displayed time zone: Dublin change

11:00 - 12:30
Session 8: Mutation TestingJournal-First Papers / Research Papers at Grand canal
Chair(s): Renzo Degiovanni SnT, University of Luxembourg
11:20
20m
Talk
How Closely are Common Mutation Operators Coupled to Real Faults?
Research Papers
Gregory Gay Chalmers | University of Gothenburg, Alireza Salahirad University of South Carolina
11:40
20m
Talk
Batching Non-Conflicting Mutations for Efficient, Safe, Parallel Mutation Analysis in Rust
Research Papers
Zalán Lévai University of Sheffield, Phil McMinn University of Sheffield
12:00
20m
Talk
MUTAGEN: Reliable Coverage-Guided, Property-Based Testing using Exhaustive Mutations
Research Papers
Agustín Mista Chalmers University of Technology, Alejandro Russo Chalmers University of Technology, Sweden
11:00 - 12:30
Session 9: FuzzingPrevious Editions / Posters / Industry / Research Papers at Pearse suite
Chair(s): Xavier Devroey University of Namur
11:40
20m
Talk
Android Fuzzing: Balancing User-Inputs and Intents
Research Papers
Michael Auer University of Passau, Andreas Stahlbauer University of Passau, Gordon Fraser University of Passau
12:00
20m
Talk
Homo in Machina: Improving Fuzz Testing Coverage via Compartment Analysis
Research Papers
Joshua Bundt Northeastern University, Andrew Fasano Northeastern University, Brendan Dolan-Gavitt New York University, William Robertson Northeastern University, USA, Tim Leek MIT Lincoln Laboratory
14:00 - 15:30
Session 10: Program RepairResearch Papers / Previous Editions / Posters at Grand canal
Chair(s): Gunel Jahangirova USI Lugano, Switzerland
14:20
20m
Talk
Embedding Context as Code Dependencies for Neural Program Repair
Research Papers
Noor Nashid University of British Columbia, Mifta Sintaha University of British Columbia, Ali Mesbah University of British Columbia (UBC)
14:40
20m
Talk
CorCA: An Automatic Program Repair Tool for Checking and Removing Effectively C Flaws
Research Papers
João Inácio LASIGE, Faculdade de Ciências da Universidade de Lisboa, Ibéria Medeiros LaSIGE, Faculdade de Ciências da Universidade de Lisboa
15:00
20m
Talk
Set the right example when teaching programming: Test Informed Learning with Examples (TILE)
Research Papers
Niels Doorn Open Universiteit and NHL Stenden University of Applied Sciences, Tanja E. J. Vos Universitat Politècnica de València and Open Universiteit, Beatriz Marín Universitat Politècnica de València, Erik Barendsen Open Universiteit
14:00 - 15:30
Session 11: Test GenerationJournal-First Papers / Previous Editions / Research Papers / Tool Demo at Pearse suite
Chair(s): Gregory Gay Chalmers | University of Gothenburg
14:40
20m
Talk
Spectacular: Finding Laws from 25 Trillion Programs
Research Papers
Matthías Páll Gissurarson Chalmers University of Technology, Sweden, Diego Roque Dark Forest Technologies, James Koppel Massachusetts Institute of Technology, USA
15:00
20m
Talk
Pairwise Testing Revisited for Structured Data With Constraints
Research Papers
Luca V. Sartori LAAS-CNRS, Helene Waeselynck LAAS-CNRS, Jérémie Guiochet LAAS-CNRS

Wed 19 Apr

Displayed time zone: Dublin change

11:00 - 12:30
Session 13: Testing with Static Analysis MethodsResearch Papers / Previous Editions at Pearse suite
Chair(s): Paolo Arcaini National Institute of Informatics
11:40
20m
Talk
Two Sparsification Strategies for Accelerating Demand-Driven Pointer Analysis
Research Papers
Kadiray Karakaya Heinz Nixdorf Institut, Paderborn University, Eric Bodden
Pre-print Media Attached
12:00
20m
Talk
Model Generation For Java Frameworks
Research Papers
Linghui Luo Amazon Web Services, Goran Piskachev Amazon Web Services, Ranjith Krishnamurthy Fraunhofer IEM, Julian Dolby IBM Research, Eric Bodden Heinz Nixdorf Institut, Paderborn University and Fraunhofer IEM, Martin Schäf Amazon Web Services
14:00 - 15:40
Session 14: Web TestingJournal-First Papers / Industry / Research Papers at Grand canal
Chair(s): Thomas Laurent JSPS@National Institute of Informatics, Japan
15:00
20m
Talk
Robust web element identification for evolving applications by considering visual overlaps
Research Papers
Michel Nass Blekinge Institute of Technology, Riccardo Coppola Politecnico di Torino, Emil Alégroth Blekinge Institute of Technology, Robert Feldt Chalmers University of Technology, Sweden
14:00 - 15:40
Session 15: Flaky TestsPrevious Editions / Research Papers at Pearse suite
Chair(s): John Micco VMware
15:00
20m
Talk
Practical Flaky Test Prediction using Common Code Evolution and Test History Data
Research Papers
Martin Gruber BMW Group, University of Passau, Michael Heine BMW Group; Friedrich-Alexander Universität Erlangen-Nürnberg (FAU), Programming Systems Group, Norbert Oster Friedrich-Alexander Universität Erlangen-Nürnberg (FAU), Programming Systems Group, Michael Philippsen Friedrich-Alexander Universität Erlangen-Nürnberg (FAU), Programming Systems Group, Gordon Fraser University of Passau
Pre-print

Accepted Papers

Title
A Case Against Coverage-Based Program Spectra
Research Papers
A Coverage-Driven Systematic Test Approach for Simultaneous Localization and Mapping
Research Papers
Android Fuzzing: Balancing User-Inputs and Intents
Research Papers
Batching Non-Conflicting Mutations for Efficient, Safe, Parallel Mutation Analysis in Rust
Research Papers
Constraint-Guided Automatic Side Object Placement for Steering Control Testing in Virtual Environment
Research Papers
CorCA: An Automatic Program Repair Tool for Checking and Removing Effectively C Flaws
Research Papers
Distributed Repair of Deep Neural Networks
Research Papers
Embedding Context as Code Dependencies for Neural Program Repair
Research Papers
Heap Fuzzing: Automatic Garbage Collection Testing with Directed Random Events
Research Papers
Homo in Machina: Improving Fuzz Testing Coverage via Compartment Analysis
Research Papers
How Closely are Common Mutation Operators Coupled to Real Faults?
Research Papers
MagicMirror: Towards High-Coverage Fuzzing of Smart Contracts
Research Papers
Metamorphic Testing with Causal Graphs
Research Papers
Model Generation For Java Frameworks
Research Papers
MUTAGEN: Reliable Coverage-Guided, Property-Based Testing using Exhaustive Mutations
Research Papers
Mutation Testing of Deep Reinforcement Learning Based on Real Faults
Research Papers
Pre-print
Pairwise Testing Revisited for Structured Data With Constraints
Research Papers
Practical Flaky Test Prediction using Common Code Evolution and Test History Data
Research Papers
Pre-print
Property-Based Mutation Testing
Research Papers
Repairing DNN Architecture: Are We There Yet?
Research Papers
Pre-print
RIDA: Cross-App Record and Replay for Android
Research Papers
Robust web element identification for evolving applications by considering visual overlaps
Research Papers
Set the right example when teaching programming: Test Informed Learning with Examples (TILE)
Research Papers
Simulation-based Test Case Generation for Unmanned Aerial Vehicles in the Neighborhood of Real Flights
Research Papers
Pre-print
Spectacular: Finding Laws from 25 Trillion Programs
Research Papers
Two Sparsification Strategies for Accelerating Demand-Driven Pointer Analysis
Research Papers
Pre-print Media Attached

Call for Papers

ICST 2023 (https://conf.researchr.org/home/icst-2023) invites high-quality submissions in all areas of software testing, verification, and validation. Papers for the research track should present novel and original work that advances the state-of-the-art. Case studies and empirical research papers are also welcome.

Topics of Interest

Topics of interest include, but are not limited to, the following:

  • Fuzz testing
  • Manual testing practices and techniques
  • Search-based software testing
  • Security testing
  • Model-based testing
  • Test automation
  • Static analysis and symbolic execution
  • Formal verification and model checking
  • Software reliability
  • Social aspects of software testing process
  • Testability and design
  • Testing and development processes
  • Testing education
  • Testing in specific domains, such as mobile, web, embedded, concurrent, distributed, cloud, GUI, and real-time systems
  • Testing for learning-enabled software, including deep learning
  • Testing video games, augmented reality
  • Testing for cyber-physical systems
  • Testing/debugging tools
  • Theory of software testing
  • Empirical studies
  • Experience reports

Each submission will be reviewed by at least three members of the ICST Program Committee.

Papers that have a strong industrial/practical component and focus more on impact rather than (technical) novelty are encouraged to consider the industry track instead.

Submission Format

Full Research papers, as well as Industry papers, must conform to the two-column IEEE conference publication format, not exceed 10 pages, including all text, figures, tables, and appendices; two additional pages containing only references are permitted. It must conform to the IEEE Conference Proceedings Formatting Guidelines (please use the letter format template and conference option). The ICST 2023 research track only accepts full research papers. Short papers are not accepted to the research track.

The submission must also comply with the ACM plagiarism policy and procedures. In particular, it must not have been published elsewhere and must not be under review elsewhere while under review for ICST. The submission must also comply with the IEEE Policy on Authorship.

Lastly, the ICST 2023 Research papers track will employ a double-anonymous review process. Thus, no submission may reveal its authors’ identities. The authors must make every effort to honor the double-anonymous review process. In particular, the authors’ names must be omitted from the submission, and references to their prior work should be in the third person. Further advice, guidance, and explanation about the double-anonymous review process can be found in the Q&A page.

Submissions to the Research Papers Track that meet the above requirements can be made via EasyChair.

Any submission that does not comply with the above requirements may be rejected by the PC Chairs without further review.

If a submission is accepted, at least one author of the paper is required to attend the conference and present the paper for it to be published in the ICST 2023 conference proceedings.

Replication Material

Submissions must supply all information that is needed to replicate the results, and therefore are expected to include or point to a replication package with the necessary software, data, and instructions. Reviewers may consult these packages to resolve open issues. There can be good reasons for the absence of a replication package, such as confidential code and/or data, the research being mostly qualitative, or the paper being fully self-contained. If a paper does not come with a replication package, authors should comment on its absence in the submission data; reviewers will take such comments into account.

Submitting to ICST2023: Q&A

Please note that the Double Anonymous Review process is not used by all tracks, e.g., Industry Track. Check in the call for papers whether it is used or not.

Q: How does one prepare an ICST 2023 submission for double anymous reviewing?

In order to comply, you do not have to make your identity undiscoverable; the double­ anonymous aspect of the review process is not an adversarial identity discovery process. Essentially, the guiding principle should be to maximize the number of people who could plausibly be authors, subject to the constraint that no change is made to any technical details of the work. Therefore, you should ensure that the reviewers are able to read and review your paper without having to know who any of the authors are. Specifically, this involves at least the following four points:

  1. Omit all authors’ names, affiliations, emails, and related information from the title page as well as from the content of the paper itself.
  2. Refer to your own work in the third person. You should not change the names of your own tools, approaches or systems, since this would clearly compromise the review process. It breaks the constraint that “no change is made to any technical details of the work”. Instead, refer to the authorship or provenance of tools, approaches, or systems in the third person, so that it is credible that another author could have written your paper.
  3. Do not rely on supplementary material (your website, GitHub repository, YouTube channel, a companion technical report, or thesis) in the paper. Supplementary information might result in revealing author identities.
  4. Anonymize project and grant names and numbers or those of funding agencies or countries as well as any acknowledgments of support to the work you report on.

We further expect you to follow the excellent advice on anonymization from ACM.

When anonymizing your email, affiliations, name, etc., try to refrain from being overly creative or “funny” by coming up with your own, anonymized versions. For emails preferably use author1@anon.org, author2@anon.org, etc., since initial DBR screening will be done by an automated tool.

Q: I previously published an earlier version of this work in a venue that does not have double anonymous reviews. What should I do about acknowledging that previous work?

Double anonymous reviewing does not and cannot mean that it is impossible for the referees to discover the identity of the author. However, we require authors to help make it easy for author identity to not play a role in the reviewing process. Therefore, we ask that in the materials you submit to us to be reviewed author identity is not revealed.

If the work you are submitting for review has previously been published in a non-peer-reviewed venue (e.g., arXiv departmental tech report), there is no need to cite it, because unrefereed work is not truly part of the scientific literature. If the previous work is published in a peer-reviewed venue, then it should be cited, but in the third person so that it is not clear whether or not this work was done by the author of the submitted paper or some other set of authors unknown. However, if citing in the third person would still risk that it is easy to identify the authors please err on the side of caution by also anonymizing the papers being extended (both when cited and in the reference list).

Q: Our submission makes use of work from a Ph.D./master’s thesis dissertation/report which has been published. Citing the dissertation might compromise anonymity. What should we do?

It is perfectly OK to publish work from a Ph.D./master’s thesis, and there is no need to cite it in the version submitted for review because prior dissertation publication does not compromise novelty. In the final (post-review, camera-ready) version of the paper, please do cite the dissertation to acknowledge its contribution, but in the refereed version of the paper that you submit, please refrain from citing the dissertation.

However, you need not worry whether or not the dissertation has appeared, since your job is only to help the committee review your work without awareness of author identity, but not to make it impossible for them to discover the identity of authors. The referees will be trying hard not to discover the authors’ identity, so they will likely not be searching the web to check whether there is a dissertation related to this work.

Q: I am submitting to the industry track. Should I double anonymous my submission?

No, you should not. Since industry papers typically rely heavily on the industrial or practical context in which the work was carried out it would be too much to ask to require this context to be anonymized.

Q: I want to include a link to an online appendix in my submission. How should I do this?

Ideally, the information in the appendix should be anonymous and it should be uploaded to an anonymous service such as for example figshare or create a new Github (or other) sharing account that is not associated with your real name. These sites will give you a link that is anonymous. Later, if the paper is accepted you can turn that link into a non-anonymized link or just put the appendix on your own site and change the link in the camera-ready version of the paper. An alternative solution is to not include the link in the submission; normally papers should be possible to review based on only the material of the paper itself.

To upload material on Figshare please create an account there, then add a new item, use the keywords “Supplemental Materials” and add the other item-specific data and then select “Make file(s) confidential” and select “Generate private link”. Copy the URL generated there and then “Save changes”. Your file(s) can now be accessed anonymously at the given URL so you can put it in your ICST submission.

Q: What if we want to cite some unpublished work of our own (as motivation for example)

If the unpublished paper is an earlier version of the paper you want to submit to ICST and is currently under review, then you have to wait until your earlier version is through its review process before you can build on it with further submissions (this would be considered double-submission and violates ACM plagiarism policy and procedures). Otherwise, if the unpublished work is not an earlier version of the proposed ICST submission, then you should simply make it available on a website, for example, and cite it in the third person to preserve anonymity, as you are doing with others of your works.

Q: Can I disseminate a non anonymized version of my submitted work by discussing it with colleagues, giving talks, publishing it at ArXiV, etc.?

You can discuss and present your work that is under submission at small meetings (e.g., job talks, visits to research labs, a Dagstuhl or Shonan meeting), but you should avoid broadly advertising it in a way that reaches the reviewers even if they are not searching for it. For example, you are allowed to put your submission on your home page and present your work at small professional meetings. However, you should not discuss your work with members of the program committee, publicize your work on mailing lists or media that are widely shared and can reach the program committee, or post your work on ArXiV or a similar site just before or after submitting it to the conference.

Be sure to first consult the guidelines on double-blind submissions outlined in the Q&A page.

Please make your submission via EasyChair using the link below, select the “ICST2023 research papers” option when prompted for the track you are submitting to: EasyChair

:
: