The Demonstrations track seeks to bridge the gap between research and practice, focusing on software testing, verification, and validation. Its main goal is to advance both the state of the art and the state of the practice.
The submission details:
- Demonstrations of testing-related tools and prototypes with an extended abstract (max 2 pages + references) and a demo video of 3-5 mins
Accepted extended abstracts will be published in ICST conference proceedings.
Mon 17 AprDisplayed time zone: Dublin change
11:00 - 12:30 | Session 1: Automated Testing Journal-First Papers / Research Papers / Previous Editions / Testing Tools / Tool Demo at Grand canal Chair(s): Gilles Perrouin Fonds de la Recherche Scientifique - FNRS & University of Namur | ||
11:00 20mTalk | JUGE: An infrastructure for benchmarking Java unit test generators Journal-First Papers Xavier Devroey University of Namur, Alessio Gambi IMC University of Applied Sciences Krems, Juan Pablo Galeotti University of Buenos Aires, René Just University of Washington, Fitsum Kifetew Fondazione Bruno Kessler, Annibale Panichella Delft University of Technology, Sebastiano Panichella Zurich University of Applied Sciences DOI Authorizer link Pre-print | ||
11:20 20mTalk | Metamorphic Testing with Causal Graphs Research Papers Andrew Graham Clark University of Sheffield, Michael Foster University of Sheffield, Neil Walkinshaw University of Sheffield, Robert Hierons University of Sheffield | ||
11:40 20mTalk | QEX: Automated Testing Observability and QA Developer Experience Framework Testing Tools | ||
12:00 10mTalk | ASDF - A Differential Testing Framework for Automatic Speech Recognition Systems Tool Demo Daniel Hao Xian Yuen School of Information Technology, Monash University Malaysia, Andrew Yong Chen Pang School of Information Technology, Monash University Malaysia, Zhou Yang Singapore Management University, Chun Yong Chong Monash University Malaysia, Mei Kuan Lim Monash University Malaysia, David Lo Singapore Management University | ||
12:10 20mTalk | A Framework for Automated API Fuzzing at Enterprise Scale Previous Editions Riyadh Mahmood The Aerospace Corporation, Jay Pennington The Aerospace Corporation, Danny Tsang The Aerospace Corporation, Tan Tran The Aerospace Corporation, Andrea Bogle The Aerospace Corporation DOI |
11:00 - 12:30 | Session 2: Autonomous & Cyberphysical Systems IResearch Papers / Journal-First Papers at Pearse suite Chair(s): Shaukat Ali Simula Research Laboratory | ||
11:00 20mTalk | A Robustness-Based Confidence Measure for Hybrid System Falsification Journal-First Papers Toru Takisaka University of Electronic Science and Technology of China, Zhenya Zhang Kyushu University, Paolo Arcaini National Institute of Informatics
, Ichiro Hasuo National Institute of Informatics, Japan Link to publication DOI | ||
11:20 20mTalk | A Coverage-Driven Systematic Test Approach for Simultaneous Localization and Mapping Research Papers | ||
11:40 20mTalk | Property-Based Mutation Testing Research Papers Ezio Bartocci Technische Universität Wien, Leonardo Mariani University of Milano-Bicocca, Dejan Nickovic Austrian Institute of Technology, Drishti Yadav Technische Universität Wien | ||
12:00 20mTalk | Constraint-Guided Automatic Side Object Placement for Steering Control Testing in Virtual Environment Research Papers Baekgyu Kim DGIST |
14:00 - 15:30 | Session 3: Autonomous & Cyberphysical Systems IIPrevious Editions / Industry / Research Papers at Grand canal Chair(s): Fabrizio Pastore University of Luxembourg | ||
14:00 20mTalk | Test Maintenance for Machine Learning Systems: A Case Study in the Automotive Industry Industry Lukas Berglund Chalmers | University of Gothenburg, Tim Grube Chalmers | University of Gothenburg, Gregory Gay Chalmers | University of Gothenburg, Francisco Gomes de Oliveira Neto Chalmers University of Technology, Sweden / University of Gothenburg, Sweden, Dimitrios Platis Zenseact | ||
14:20 20mTalk | Digital Twin-based Anomaly Detection in Cyber-physical Systems Previous Editions DOI | ||
14:40 20mTalk | Quality Metrics and Oracles for Autonomous Vehicles Testing Previous Editions Gunel Jahangirova USI Lugano, Switzerland, Andrea Stocco Technical University of Munich & fortiss, Paolo Tonella USI Lugano DOI | ||
15:00 20mTalk | Simulation-based Test Case Generation for Unmanned Aerial Vehicles in the Neighborhood of Real Flights Research Papers Sajad Khatiri USI-Lugnao & Zurich University of Applied Sciences, Sebastiano Panichella Zurich University of Applied Sciences, Paolo Tonella USI Lugano Pre-print |
14:00 - 15:30 | Session 4: Fault Localization & DebuggingResearch Papers / Tool Demo / Industry / Journal-First Papers at Pearse suite Chair(s): Shin Yoo KAIST | ||
14:00 20mTalk | Flake Aware Culprit Finding Industry Tim A. D. Henderson Google LLC, Bobby Dorward Google, Eric Nickell Google, Collin Johnson Google, Avi Kondareddy Google LLC Pre-print | ||
14:20 20mTalk | An Empirical Characterization of Software Bugs in Open-Source Cyber-Physical Systems Journal-First Papers Fiorella Zampetti University of Sannio, Italy, Ritu Kapur University of Sannio, Massimiliano Di Penta University of Sannio, Italy, Sebastiano Panichella Zurich University of Applied Sciences | ||
14:40 20mTalk | A Case Against Coverage-Based Program Spectra Research Papers Péter Attila Soha Department of Software Engineering, University of Szeged, Tamás Gergely Department of Software Engineering, University of Szeged, Ferenc Horv�th University of Szeged, Department of Software Engineering, Béla Vancsics Department of Software Engineering, University of Szeged, Árpád Beszédes Department of Software Engineering, University of Szeged | ||
15:00 10mTalk | SFLaaS: Software Fault Localization as a Service Tool Demo Qusay Idrees Sarhan Department of Software Engineering, University of Szeged, Hassan Bapeer Hassan University of Duhok, Árpád Beszédes Department of Software Engineering, University of Szeged |
16:00 - 18:00 | Session 5: Testing AI/ML systemsResearch Papers / Previous Editions at Grand canal Chair(s): Jie M. Zhang King's College London | ||
16:00 20mTalk | Robustness assessment and improvement of a neural network for blood oxygen pressure estimation Previous Editions Paolo Arcaini National Institute of Informatics
, Andrea Bombarda University of Bergamo, Silvia Bonfanti University of Bergamo, Angelo Gargantini University of Bergamo, Daniele Gamba AISent S.r.l., Rita Pedercini AISent S.r.l. DOI | ||
16:20 20mTalk | An Empirical Evaluation of Mutation Operators for Deep Learning Systems Previous Editions DOI | ||
16:40 20mTalk | Distributed Repair of Deep Neural Networks Research Papers Davide Li Calsi Politecnico di Milano, Matias Duran National Institute of Informatics, Xiao-Yi Zhang School of Computer and Communication Engineering, University of Science and Technology Beijing, Paolo Arcaini National Institute of Informatics
, Fuyuki Ishikawa National Institute of Informatics | ||
17:00 20mTalk | Mutation Testing of Deep Reinforcement Learning Based on Real Faults Research Papers Florian Tambon Polytechnique Montréal, Vahid Majdinasab Polytechnique Montréal, Amin Nikanjam École Polytechnique de Montréal, Foutse Khomh Polytechnique Montréal, Giuliano Antoniol Polytechnique Montréal Pre-print | ||
17:20 20mTalk | Repairing DNN Architecture: Are We There Yet? Research Papers Jinhan Kim KAIST, Nargiz Humbatova USI Lugano, Gunel Jahangirova King's College London, Paolo Tonella USI Lugano, Shin Yoo KAIST Pre-print |
16:00 - 18:00 | Session 7: Testing for Safery & Security Industry / Research Papers / Journal-First Papers / Previous Editions at Hanover Chair(s): Eric Bodden Heinz Nixdorf Institut, Paderborn University and Fraunhofer IEM | ||
16:00 20mTalk | Learning Non-robustness using Simulation-based Testing: a Network Traffic-shaping Case Study Industry Baharin Aliashrafi Jodat University of Ottawa, Shiva Nejati University of Ottawa, Mehrdad Sabetzadeh University of Ottawa, Patricio Saavedra RabbitRun Technologies Inc Pre-print | ||
16:20 20mTalk | Test environments for large-scale software systems – an industrial study of intrinsic and extrinsic success factors Journal-First Papers | ||
16:40 20mTalk | Assessing the Effectiveness of Input and Output Coverage Criteria for Testing Quantum Programs Previous Editions Shaukat Ali Simula Research Laboratory, Paolo Arcaini National Institute of Informatics
, Xinyi Wang , Tao Yue Simula Research Laboratory DOI | ||
17:00 20mTalk | Heap Fuzzing: Automatic Garbage Collection Testing with Directed Random Events Research Papers Guillermo Polito Inria, Cristal, UMR 9189, Université de Lille, Pablo Tesone Univ. Lille, Inria, CNRS, Centrale Lille, UMR 9189 CRIStAL, Pharo Consortium, Jean Privat Université du Québec à Montréal (UQAM), Nahuel Palumbo Université Lille, CNRS, Centrale Lille, Inria, UMR 9189 - CRIStAL, Stéphane Ducasse Inria; University of Lille; CNRS; Centrale Lille; CRIStAL | ||
17:20 20mTalk | MagicMirror: Towards High-Coverage Fuzzing of Smart Contracts Research Papers Huadong Feng University of Texas at Arlington, Xiaolei Ren University of Texas at Arlington, Qiping Wei University of Texas at Arlington, Jeff Yu Lei University of Texas at Arlington, Raghu Kacker National Institute of Standards and Technology, Richard Kuhn National Institute of Standards and Technology, Dimitris Simos SBA Research |
Tue 18 AprDisplayed time zone: Dublin change
11:00 - 12:30 | Session 8: Mutation TestingJournal-First Papers / Research Papers at Grand canal Chair(s): Renzo Degiovanni SnT, University of Luxembourg | ||
11:00 20mTalk | Mutation testing of PL/SQL programs Journal-First Papers | ||
11:20 20mTalk | How Closely are Common Mutation Operators Coupled to Real Faults? Research Papers | ||
11:40 20mTalk | Batching Non-Conflicting Mutations for Efficient, Safe, Parallel Mutation Analysis in Rust Research Papers | ||
12:00 20mTalk | MUTAGEN: Reliable Coverage-Guided, Property-Based Testing using Exhaustive Mutations Research Papers Agustín Mista Chalmers University of Technology, Alejandro Russo Chalmers University of Technology, Sweden |
14:00 - 15:30 | Session 10: Program RepairResearch Papers / Previous Editions / Posters at Grand canal Chair(s): Gunel Jahangirova USI Lugano, Switzerland | ||
14:00 20mTalk | Exploring True Test Overfitting in Dynamic Automated Program Repair using Formal Methods Previous Editions Amirfarhad Nilizadeh University of Central Florida, Gary T. Leavens University of Central Florida, Xuan Bach D. Le The University of Melbourne, Corina S. Păsăreanu Carnegie Mellon University, David Cok Safer Software Consulting, LLC DOI | ||
14:20 20mTalk | Embedding Context as Code Dependencies for Neural Program Repair Research Papers Noor Nashid University of British Columbia, Mifta Sintaha University of British Columbia, Ali Mesbah University of British Columbia (UBC) | ||
14:40 20mTalk | CorCA: An Automatic Program Repair Tool for Checking and Removing Effectively C Flaws Research Papers João Inácio LASIGE, Faculdade de Ciências da Universidade de Lisboa, Ibéria Medeiros LaSIGE, Faculdade de Ciências da Universidade de Lisboa | ||
15:00 20mTalk | Set the right example when teaching programming: Test Informed Learning with Examples (TILE) Research Papers Niels Doorn Open Universiteit and NHL Stenden University of Applied Sciences, Tanja E. J. Vos Universitat Politècnica de València and Open Universiteit, Beatriz Marín Universitat Politècnica de València, Erik Barendsen Open Universiteit | ||
15:20 5mTalk | Poster: Software Fault Localization as a Service (SFLaaS) Posters Qusay Idrees Sarhan Department of Software Engineering, University of Szeged, Hassan Bapeer Hassan University of Duhok, Árpád Beszédes Department of Software Engineering, University of Szeged | ||
15:25 5mTalk | Poster: Improving Spectrum Based Fault Localization For Python Programs Using Weighted Code Elements Posters Qusay Idrees Sarhan Department of Software Engineering, University of Szeged, Árpád Beszédes Department of Software Engineering, University of Szeged |
14:00 - 15:30 | Session 11: Test GenerationJournal-First Papers / Previous Editions / Research Papers / Tool Demo at Pearse suite Chair(s): Gregory Gay Chalmers | University of Gothenburg | ||
14:00 20mTalk | Automatic Creation of Acceptance Tests by Extracting Conditionals from Requirements: NLP Approach and Case Study Journal-First Papers Jannik Fischbach Netlight GmbH / fortiss GmbH, Julian Frattini Blekinge Institute of Technology, Andreas Vogelsang University of Cologne, Daniel Mendez Blekinge Institute of Technology, Michael Unterkalmsteiner Blekinge Institute of Technology, Andreas Wehrle Allianz Deutschland AG, Pablo Restrepo Henao Technical University of Munich, Parisa Yousefi Ericsson, Tedi Juricic Ericsson, Jeannette Radduenz Allianz Deutschland, Carsten Wiecher Kostal Automobil Elektrik GmbH & Co. KG | ||
14:20 20mTalk | Scaling Test Case Generation For Expressive Decision Tables Previous Editions Supriya Agrawal Tata Consultancy Services Ltd. (TCS), R. Venkatesh , Ulka Shrotri Tata Consultancy Services Ltd. (TCS), Amey Zare TCS Research, Sagar Verma Tata Consultancy Services Ltd. (TCS) DOI | ||
14:40 20mTalk | Spectacular: Finding Laws from 25 Trillion Programs Research Papers Matthías Páll Gissurarson Chalmers University of Technology, Sweden, Diego Roque Dark Forest Technologies, James Koppel Massachusetts Institute of Technology, USA | ||
15:00 20mTalk | Pairwise Testing Revisited for Structured Data With Constraints Research Papers | ||
15:20 10mTalk | RICK: Generating Mocks from Production Data Tool Demo Deepika Tiwari KTH Royal Institute of Technology, Martin Monperrus KTH Royal Institute of Technology, Benoit Baudry KTH |
Wed 19 AprDisplayed time zone: Dublin change
11:00 - 12:30 | Session 13: Testing with Static Analysis MethodsResearch Papers / Previous Editions at Pearse suite Chair(s): Paolo Arcaini National Institute of Informatics | ||
11:00 20mTalk | Address-Aware Query Caching for Symbolic Execution Previous Editions DOI | ||
11:20 20mTalk | Automating Differential Testing with Overapproximate Symbolic Execution Previous Editions DOI | ||
11:40 20mTalk | Two Sparsification Strategies for Accelerating Demand-Driven Pointer Analysis Research Papers Pre-print Media Attached | ||
12:00 20mTalk | Model Generation For Java Frameworks Research Papers Linghui Luo Amazon Web Services, Goran Piskachev Amazon Web Services, Ranjith Krishnamurthy Fraunhofer IEM, Julian Dolby IBM Research, Eric Bodden Heinz Nixdorf Institut, Paderborn University and Fraunhofer IEM, Martin Schäf Amazon Web Services |
14:00 - 15:40 | Session 14: Web TestingJournal-First Papers / Industry / Research Papers at Grand canal Chair(s): Thomas Laurent JSPS@National Institute of Informatics, Japan | ||
14:00 20mTalk | Challenges of End-to-End Testing with Selenium WebDriver and How to Face Them: A Survey Industry Maurizio Leotta DIBRIS, University of Genova, Italy, Boni Garcia Universidad Carlos III de Madrid, Filippo Ricca Università di Genova, Jim Whitehead University of California, Santa Cruz | ||
14:20 20mTalk | E2E-Loader: A Framework to Support Performance Testing of Web Applications Industry Ermanno Battista Fervento srl, Sergio Di Martino Università degli Studi di Napoli Federico II, Sergio Di Meglio Department of Electrical Engineering and Information Technology Università degli Studi di Napoli Federico II, Fabio Scippacercola Fervento srl, Luigi Libero Lucio Starace Università degli Studi di Napoli Federico II | ||
14:40 20mTalk | Automated Web Application Testing driven by Pre-recorded Test Cases Journal-First Papers | ||
15:00 20mTalk | Robust web element identification for evolving applications by considering visual overlaps Research Papers Michel Nass Blekinge Institute of Technology, Riccardo Coppola Politecnico di Torino, Emil Alégroth Blekinge Institute of Technology, Robert Feldt Chalmers University of Technology, Sweden | ||
15:20 20mTalk | Automating Endurance Test for Flash-based Storage Devices in Samsung Electronics Industry Jinkook Kim Samsung Electronics Co., Minseok Jeon Korea University, Sejeong Jang Samsung Electronics Co., Hakjoo Oh Korea University |
14:00 - 15:40 | Session 15: Flaky TestsPrevious Editions / Research Papers at Pearse suite Chair(s): John Micco VMware | ||
14:00 20mTalk | Evaluating Features for Machine Learning Detection of Order- and Non-Order-Dependent Flaky Tests Previous Editions Owain Parry The University of Sheffield, Gregory Kapfhammer Allegheny College, Michael Hilton Carnegie Mellon University, Phil McMinn University of Sheffield DOI | ||
14:20 20mTalk | An Empirical Study of Flaky Tests in Python Previous Editions Martin Gruber BMW Group, University of Passau, Stephan Lukasczyk University of Passau, Florian Kroiß , Gordon Fraser University of Passau DOI | ||
14:40 20mTalk | A Survey on How Test Flakiness Affects Developers and What Support They Need To Address It Previous Editions DOI | ||
15:00 20mTalk | Practical Flaky Test Prediction using Common Code Evolution and Test History Data Research Papers Martin Gruber BMW Group, University of Passau, Michael Heine BMW Group; Friedrich-Alexander Universität Erlangen-Nürnberg (FAU), Programming Systems Group, Norbert Oster Friedrich-Alexander Universität Erlangen-Nürnberg (FAU), Programming Systems Group, Michael Philippsen Friedrich-Alexander Universität Erlangen-Nürnberg (FAU), Programming Systems Group, Gordon Fraser University of Passau Pre-print | ||
15:20 20mTalk | A Qualitative Study on the Sources, Impacts, and Mitigation Strategies of Flaky Tests Previous Editions Sarra Habchi Ubisoft, Guillaume Haben University of Luxembourg, Mike Papadakis University of Luxembourg, Luxembourg, Maxime Cordy University of Luxembourg, Luxembourg, Yves Le Traon University of Luxembourg, Luxembourg DOI |
Accepted Papers
ICST 2023 Demonstrations Track Call for Papers
The Demonstrations track seeks to bridge the gap between research and practice, focusing on software testing, verification, and validation. Its main goal is to advance both the state of the art and the state of the practice.
The submission details:
- Demonstrations of testing-related tools and prototypes with an extended abstract (max 2 pages + references) and a demo video of 3-5 mins
Accepted extended abstracts will be published in ICST conference proceedings.
Demonstrations should show and discuss advances in Verification and Validation Research through running demos. The demonstrations should clearly communicate the following information to the audience:
- The envisioned users of the tool
- The software engineering challenge the tool addresses
- The testing process or testing technique implemented by the tool
- The implied use case scenario for its users
- The potential ways that the tool could be extended (by the authors or the community) in the future.
Highlighting scientific contributions through concrete artifacts is an important supplement to the traditional ICST research papers. A demonstration provides the opportunity to communicate how the scientific approach has been implemented or how a specific hypothesis has been assessed, including details such as implementation and usage issues, data models and representations, or APIs for tool and data access. Authors of regular research papers are thus also encouraged to submit an accompanying demonstration paper.
The tool itself should be made available, if not already, for researchers or practitioners to use. At minimum, the tool should be accessible (either free to download or for purchase). If possible, the source code of the tool should also be available as well. Exceptions will be made to this rule if a valid reason is provided for why the tool cannot be released (e.g., organizational rules or it was developed for internal use in a company and cannot be made public).
Areas of interest:
The Demonstrations Track invites submissions related, but not limited, to the following areas:
- Design and development of novel tools for software testing, verification, and validation;
- Working prototypes for novel research approaches, reporting on the technical challenges and early feedback obtained from practitioners;
- Strategies and challenges for tool adoption as well as lessons learned from technology transfer of academic approaches into open source or commercial tools;
- Case studies and/or experiments involving tools in particular contexts, reporting on lessons learned and negative results (if any).
Evaluation:
Each submission will be evaluated based on:
- The relevance and significance of the addressed problem.
- The innovation element of the approach.
- The availability, maturity, and adoption of the tool.
- The presence of lessons learned from developing or using the tool.
- The quality of the presentation.
Accepted extended abstracts will appear in the conference or companion proceedings and the IEEE digital library.
Relationship to Testing Tools Track:
Authors of tool papers will be asked to give a demonstration of the tool, in the same way as the authors of demonstration papers (extended abstracts). The extra page space for tool papers compared to demonstration papers is to provide more insight about the tool and how it can be used (e.g., about the Software Engineering challenges and lessons learned behind building such tools). Therefore, authors of tool papers are not allowed to submit a demonstration paper for the same tool. In the case in which the insights provided by a tool paper are not considered of sufficient scientific and engineering quality, the paper could still be accepted as a (shorter) Demonstrations abstract instead (if the authors agree).
Submission:
Submissions will be handled via Easychair (ICST2023 / Demonstrations Track):
The Demonstrations Track of ICST 2023 uses single-anonymous reviewing, which means that authors and tools do not have to be anonymized. All submitted papers must conform to the two column IEEE conference publication format. Templates for Latex and Word are available at: http://www.ieee.org/conferences_events/conferences/publishing/templates.html;
- It must conform to the IEEE Conference Proceedings Formatting Guidelines (please use the letter format template and conference option).
- Extended abstracts must be submitted as PDF.
- Extended abstracts of demonstrations must not exceed 2 pages, including all text, figures, tables, and appendices; one additional page containing only references is permitted.
- Demo submissions must have a demo video with length of 3 - 5 minutes, and the submission must have a link to an online location that allows reviewers to watch it.
The submission must also comply with the ACM plagiarism policy and procedures. In particular, the same content must not have been published elsewhere and must not be under review elsewhere while under review for ICST. The submission must also comply with the IEEE Policy on Authorship.