ICST 2023
Sun 16 - Thu 20 April 2023 Dublin, Ireland

ICST 2023 invites high quality submissions in all areas of software testing, verification, and validation. The industry track has the same topics and requires the same rigorousness as the research track. However, the industry track papers should prioritize impact and realism over novelty. We expect that industry track publications report useful testing tools, successful stories and experiences when applying software testing theories and methodologies into practice, or promising prototypes that have the potential to impact practice in the next few years. Ideally they should be evaluated on real-world, industrial data, and discuss non-functional aspects such as scalability and reliability, and also compare to pre-existing industrial best practices. The industry track papers will be published in the ICST 2023 proceedings and listed by IEEE databases.

Supporters
Diamond Sponsor
Platinum Sponsor
Platinum Sponsor
Gold Sponsor
Silver Sponsor
Bronze Sponsor
Dates
Tracks
Plenary
You're viewing the program in a time zone which is different from your device's time zone change time zone

Mon 17 Apr

Displayed time zone: Dublin change

10:30 - 11:00
Coffee break Social
10:30
30m
Coffee break
Break
Social

11:00 - 12:30
Session 1: Automated Testing Journal-First Papers / Research Papers / Previous Editions / Testing Tools / Tool Demo at Grand canal
Chair(s): Gilles Perrouin Fonds de la Recherche Scientifique - FNRS & University of Namur
11:00
20m
Talk
JUGE: An infrastructure for benchmarking Java unit test generators
Journal-First Papers
Xavier Devroey University of Namur, Alessio Gambi IMC University of Applied Sciences Krems, Juan Pablo Galeotti University of Buenos Aires, René Just University of Washington, Fitsum Kifetew Fondazione Bruno Kessler, Annibale Panichella Delft University of Technology, Sebastiano Panichella Zurich University of Applied Sciences
DOI Authorizer link Pre-print
11:20
20m
Talk
Metamorphic Testing with Causal Graphs
Research Papers
Andrew Graham Clark University of Sheffield, Michael Foster University of Sheffield, Neil Walkinshaw University of Sheffield, Robert Hierons University of Sheffield
11:40
20m
Talk
QEX: Automated Testing Observability and QA Developer Experience Framework
Testing Tools
Luohua Huang Shopee, Joseph Chu Shopee, Keshia Yap Shopee, Hock Yao Chua Shopee
12:00
10m
Talk
ASDF - A Differential Testing Framework for Automatic Speech Recognition Systems
Tool Demo
Daniel Hao Xian Yuen School of Information Technology, Monash University Malaysia, Andrew Yong Chen Pang School of Information Technology, Monash University Malaysia, Zhou Yang Singapore Management University, Chun Yong Chong Monash University Malaysia, Mei Kuan Lim Monash University Malaysia, David Lo Singapore Management University
12:10
20m
Talk
A Framework for Automated API Fuzzing at Enterprise Scale
Previous Editions
Riyadh Mahmood The Aerospace Corporation, Jay Pennington The Aerospace Corporation, Danny Tsang The Aerospace Corporation, Tan Tran The Aerospace Corporation, Andrea Bogle The Aerospace Corporation
DOI
11:00 - 12:30
Session 2: Autonomous & Cyberphysical Systems IResearch Papers / Journal-First Papers at Pearse suite
Chair(s): Shaukat Ali Simula Research Laboratory
11:00
20m
Talk
A Robustness-Based Confidence Measure for Hybrid System Falsification
Journal-First Papers
Toru Takisaka University of Electronic Science and Technology of China, Zhenya Zhang Kyushu University, Paolo Arcaini National Institute of Informatics , Ichiro Hasuo National Institute of Informatics, Japan
Link to publication DOI
11:20
20m
Talk
A Coverage-Driven Systematic Test Approach for Simultaneous Localization and Mapping
Research Papers
Philip Tasche University of Twente, Paula Herber University of Munster, Germany
11:40
20m
Talk
Property-Based Mutation Testing
Research Papers
Ezio Bartocci Technische Universität Wien, Leonardo Mariani University of Milano-Bicocca, Dejan Nickovic Austrian Institute of Technology, Drishti Yadav Technische Universität Wien
12:00
20m
Talk
Constraint-Guided Automatic Side Object Placement for Steering Control Testing in Virtual Environment
Research Papers
12:30 - 14:00
LunchSocial
12:30
90m
Lunch
Lunch
Social

14:00 - 15:30
Session 3: Autonomous & Cyberphysical Systems IIPrevious Editions / Industry / Research Papers at Grand canal
Chair(s): Fabrizio Pastore University of Luxembourg
14:00
20m
Talk
Test Maintenance for Machine Learning Systems: A Case Study in the Automotive Industry
Industry
Lukas Berglund Chalmers | University of Gothenburg, Tim Grube Chalmers | University of Gothenburg, Gregory Gay Chalmers | University of Gothenburg, Francisco Gomes de Oliveira Neto Chalmers University of Technology, Sweden / University of Gothenburg, Sweden, Dimitrios Platis Zenseact
14:20
20m
Talk
Digital Twin-based Anomaly Detection in Cyber-physical Systems
Previous Editions
Xu Qinghua , Shaukat Ali Simula Research Laboratory, Tao Yue Simula Research Laboratory
DOI
14:40
20m
Talk
Quality Metrics and Oracles for Autonomous Vehicles Testing
Previous Editions
Gunel Jahangirova USI Lugano, Switzerland, Andrea Stocco Technical University of Munich & fortiss, Paolo Tonella USI Lugano
DOI
15:00
20m
Talk
Simulation-based Test Case Generation for Unmanned Aerial Vehicles in the Neighborhood of Real Flights
Research Papers
Sajad Khatiri USI-Lugnao & Zurich University of Applied Sciences, Sebastiano Panichella Zurich University of Applied Sciences, Paolo Tonella USI Lugano
Pre-print
14:00 - 15:30
Session 4: Fault Localization & DebuggingResearch Papers / Tool Demo / Industry / Journal-First Papers at Pearse suite
Chair(s): Shin Yoo KAIST
14:00
20m
Talk
Flake Aware Culprit Finding
Industry
Tim A. D. Henderson Google LLC, Bobby Dorward Google, Eric Nickell Google, Collin Johnson Google, Avi Kondareddy Google LLC
Pre-print
14:20
20m
Talk
An Empirical Characterization of Software Bugs in Open-Source Cyber-Physical Systems
Journal-First Papers
Fiorella Zampetti University of Sannio, Italy, Ritu Kapur University of Sannio, Massimiliano Di Penta University of Sannio, Italy, Sebastiano Panichella Zurich University of Applied Sciences
14:40
20m
Talk
A Case Against Coverage-Based Program Spectra
Research Papers
Péter Attila Soha Department of Software Engineering, University of Szeged, Tamás Gergely Department of Software Engineering, University of Szeged, Ferenc Horv�th University of Szeged, Department of Software Engineering, Béla Vancsics Department of Software Engineering, University of Szeged, Árpád Beszédes Department of Software Engineering, University of Szeged
15:00
10m
Talk
SFLaaS: Software Fault Localization as a Service
Tool Demo
Qusay Idrees Sarhan Department of Software Engineering, University of Szeged, Hassan Bapeer Hassan University of Duhok, Árpád Beszédes Department of Software Engineering, University of Szeged
15:30 - 16:00
Coffee break Social
15:30
30m
Coffee break
Break
Social

16:00 - 18:00
Session 5: Testing AI/ML systemsResearch Papers / Previous Editions at Grand canal
Chair(s): Jie M. Zhang King's College London
16:00
20m
Talk
Robustness assessment and improvement of a neural network for blood oxygen pressure estimation
Previous Editions
Paolo Arcaini National Institute of Informatics , Andrea Bombarda University of Bergamo, Silvia Bonfanti University of Bergamo, Angelo Gargantini University of Bergamo, Daniele Gamba AISent S.r.l., Rita Pedercini AISent S.r.l.
DOI
16:20
20m
Talk
An Empirical Evaluation of Mutation Operators for Deep Learning Systems
Previous Editions
Gunel Jahangirova King's College London, Paolo Tonella USI Lugano
DOI
16:40
20m
Talk
Distributed Repair of Deep Neural Networks
Research Papers
Davide Li Calsi Politecnico di Milano, Matias Duran National Institute of Informatics, Xiao-Yi Zhang School of Computer and Communication Engineering, University of Science and Technology Beijing, Paolo Arcaini National Institute of Informatics , Fuyuki Ishikawa National Institute of Informatics
17:00
20m
Talk
Mutation Testing of Deep Reinforcement Learning Based on Real Faults
Research Papers
Florian Tambon Polytechnique Montréal, Vahid Majdinasab Polytechnique Montréal, Amin Nikanjam École Polytechnique de Montréal, Foutse Khomh Polytechnique Montréal, Giuliano Antoniol Polytechnique Montréal
Pre-print
17:20
20m
Talk
Repairing DNN Architecture: Are We There Yet?
Research Papers
Jinhan Kim KAIST, Nargiz Humbatova USI Lugano, Gunel Jahangirova King's College London, Paolo Tonella USI Lugano, Shin Yoo KAIST
Pre-print
16:00 - 18:00
Session 7: Testing for Safery & Security Industry / Research Papers / Journal-First Papers / Previous Editions at Hanover
Chair(s): Eric Bodden Heinz Nixdorf Institut, Paderborn University and Fraunhofer IEM
16:00
20m
Talk
Learning Non-robustness using Simulation-based Testing: a Network Traffic-shaping Case Study
Industry
Baharin Aliashrafi Jodat University of Ottawa, Shiva Nejati University of Ottawa, Mehrdad Sabetzadeh University of Ottawa, Patricio Saavedra RabbitRun Technologies Inc
Pre-print
16:20
20m
Talk
Test environments for large-scale software systems – an industrial study of intrinsic and extrinsic success factors
Journal-First Papers
Torvald Mårtensson Saab AB, Göran Ancher Saab AB, Daniel Ståhl Ericsson AB
16:40
20m
Talk
Assessing the Effectiveness of Input and Output Coverage Criteria for Testing Quantum Programs
Previous Editions
Shaukat Ali Simula Research Laboratory, Paolo Arcaini National Institute of Informatics , Xinyi Wang , Tao Yue Simula Research Laboratory
DOI
17:00
20m
Talk
Heap Fuzzing: Automatic Garbage Collection Testing with Directed Random Events
Research Papers
Guillermo Polito Inria, Cristal, UMR 9189, Université de Lille, Pablo Tesone Univ. Lille, Inria, CNRS, Centrale Lille, UMR 9189 CRIStAL, Pharo Consortium, Jean Privat Université du Québec à Montréal (UQAM), Nahuel Palumbo Université Lille, CNRS, Centrale Lille, Inria, UMR 9189 - CRIStAL, Stéphane Ducasse Inria; University of Lille; CNRS; Centrale Lille; CRIStAL
17:20
20m
Talk
MagicMirror: Towards High-Coverage Fuzzing of Smart Contracts
Research Papers
Huadong Feng University of Texas at Arlington, Xiaolei Ren University of Texas at Arlington, Qiping Wei University of Texas at Arlington, Jeff Yu Lei University of Texas at Arlington, Raghu Kacker National Institute of Standards and Technology, Richard Kuhn National Institute of Standards and Technology, Dimitris Simos SBA Research
16:00 - 18:00
Session 6: GUI/API testingTesting Tools / Previous Editions / Research Papers / Posters at Pearse suite
Chair(s): Phil McMinn University of Sheffield
16:00
20m
Talk
Repairing Fragile GUI Test Cases Using Word and Layout Embedding
Previous Editions
Juyeon Yoon Korea Advanced Institute of Science and Technology, Seungjun Chung , Kihyuck Shin Samsung Electronics, Jinhan Kim KAIST, Shin Hong Handong Global University, Shin Yoo KAIST
DOI
16:20
20m
Talk
RIDA: Cross-App Record and Replay for Android
Research Papers
Jiayuan Liang Southern University of Science and Technology, Sinan Wang Southern University of Science and Technology, Xiangbo Deng Southern University of Science and Technology, Yepang Liu Southern University of Science and Technology
16:40
20m
Talk
EMB: A Curated Corpus of Web/Enterprise Applications And Library Support for Software Testing Research
Testing Tools
Andrea Arcuri Kristiania University College and Oslo Metropolitan University, Man Zhang Kristiania University College, Norway, Amid Golmohammadi Kristiania University College, Asma Belhadi Kristiania University College, Juan Pablo Galeotti University of Buenos Aires, Bogdan Marculescu Kristiania University College, Norway, Susruthan Seran Kristiania University College
17:00
20m
Talk
LatteArt: A Platform for Recording and Analyzing Exploratory Testing
Testing Tools
Hiroyuki Kirinuki NTT, Masaki Tajima NTT Software Innovation Center, Haruto Tanno NTT
17:20
20m
Talk
POWER: Program Option-Aware Fuzzer for High Bug Detection Ability
Previous Editions
Ahcheong Lee KAIST, Irfan Ariq , Yunho Kim Hanyang University, Moonzoo Kim KAIST / VPlusLab Inc.
DOI
17:40
5m
Talk
Poster: Link between Bias, Node Sensitivity and Long-Tail Distribution in trained DNNs
Posters
Mahum Naseer Technische Universität Wien, Muhammad Shafique New York University Abu Dhabi

Tue 18 Apr

Displayed time zone: Dublin change

11:00 - 12:30
Session 8: Mutation TestingJournal-First Papers / Research Papers at Grand canal
Chair(s): Renzo Degiovanni SnT, University of Luxembourg
11:00
20m
Talk
Mutation testing of PL/SQL programs
Journal-First Papers
Arzu Behiye Tarimci Turkcell Technology, Hasan Sozer Ozyegin University
11:20
20m
Talk
How Closely are Common Mutation Operators Coupled to Real Faults?
Research Papers
Gregory Gay Chalmers | University of Gothenburg, Alireza Salahirad University of South Carolina
11:40
20m
Talk
Batching Non-Conflicting Mutations for Efficient, Safe, Parallel Mutation Analysis in Rust
Research Papers
Zalán Lévai University of Sheffield, Phil McMinn University of Sheffield
12:00
20m
Talk
MUTAGEN: Reliable Coverage-Guided, Property-Based Testing using Exhaustive Mutations
Research Papers
Agustín Mista Chalmers University of Technology, Alejandro Russo Chalmers University of Technology, Sweden
11:00 - 12:30
Session 9: FuzzingPrevious Editions / Posters / Industry / Research Papers at Pearse suite
Chair(s): Xavier Devroey University of Namur
11:00
20m
Talk
Industrial Deployment of Compiler Fuzzing Techniques for Two GPU Shading Languages
Industry
Alastair F. Donaldson Imperial College London, Ben Clayton Google, Ryan Harrison Google, Hasan Mohsin Imperial College London, David Neto Google, Vasyl Teliman National Technical University of Ukraine, Hana Watson Imperial College London
11:20
20m
Talk
Metamorphic Fuzzing of C++ Libraries
Previous Editions
Andrei Lascu Imperial College London, Alastair F. Donaldson Imperial College London, Tobias Grosser University of Edinburgh, Torsten Hoefler ETH Zurich
DOI
11:40
20m
Talk
Android Fuzzing: Balancing User-Inputs and Intents
Research Papers
Michael Auer University of Passau, Andreas Stahlbauer University of Passau, Gordon Fraser University of Passau
12:00
20m
Talk
Homo in Machina: Improving Fuzz Testing Coverage via Compartment Analysis
Research Papers
Joshua Bundt Northeastern University, Andrew Fasano Northeastern University, Brendan Dolan-Gavitt New York University, William Robertson Northeastern University, USA, Tim Leek MIT Lincoln Laboratory
12:20
5m
Talk
Poster: BugOSS: A Regression Bug Benchmark for Empirical Study of Regression Fuzzing Techniques
Posters
Jeewoong Kim Handong Global University, Shin Hong Handong Global University
14:00 - 15:30
Session 10: Program RepairResearch Papers / Previous Editions / Posters at Grand canal
Chair(s): Gunel Jahangirova USI Lugano, Switzerland
14:00
20m
Talk
Exploring True Test Overfitting in Dynamic Automated Program Repair using Formal Methods
Previous Editions
Amirfarhad Nilizadeh University of Central Florida, Gary T. Leavens University of Central Florida, Xuan Bach D. Le The University of Melbourne, Corina S. Păsăreanu Carnegie Mellon University, David Cok Safer Software Consulting, LLC
DOI
14:20
20m
Talk
Embedding Context as Code Dependencies for Neural Program Repair
Research Papers
Noor Nashid University of British Columbia, Mifta Sintaha University of British Columbia, Ali Mesbah University of British Columbia (UBC)
14:40
20m
Talk
CorCA: An Automatic Program Repair Tool for Checking and Removing Effectively C Flaws
Research Papers
João Inácio LASIGE, Faculdade de Ciências da Universidade de Lisboa, Ibéria Medeiros LaSIGE, Faculdade de Ciências da Universidade de Lisboa
15:00
20m
Talk
Set the right example when teaching programming: Test Informed Learning with Examples (TILE)
Research Papers
Niels Doorn Open Universiteit and NHL Stenden University of Applied Sciences, Tanja E. J. Vos Universitat Politècnica de València and Open Universiteit, Beatriz Marín Universitat Politècnica de València, Erik Barendsen Open Universiteit
15:20
5m
Talk
Poster: Software Fault Localization as a Service (SFLaaS)
Posters
Qusay Idrees Sarhan Department of Software Engineering, University of Szeged, Hassan Bapeer Hassan University of Duhok, Árpád Beszédes Department of Software Engineering, University of Szeged
15:25
5m
Talk
Poster: Improving Spectrum Based Fault Localization For Python Programs Using Weighted Code Elements
Posters
Qusay Idrees Sarhan Department of Software Engineering, University of Szeged, Árpád Beszédes Department of Software Engineering, University of Szeged
14:00 - 15:30
Session 11: Test GenerationJournal-First Papers / Previous Editions / Research Papers / Tool Demo at Pearse suite
Chair(s): Gregory Gay Chalmers | University of Gothenburg
14:00
20m
Talk
Automatic Creation of Acceptance Tests by Extracting Conditionals from Requirements: NLP Approach and Case Study
Journal-First Papers
Jannik Fischbach Netlight GmbH / fortiss GmbH, Julian Frattini Blekinge Institute of Technology, Andreas Vogelsang University of Cologne, Daniel Mendez Blekinge Institute of Technology, Michael Unterkalmsteiner Blekinge Institute of Technology, Andreas Wehrle Allianz Deutschland AG, Pablo Restrepo Henao Technical University of Munich, Parisa Yousefi Ericsson, Tedi Juricic Ericsson, Jeannette Radduenz Allianz Deutschland, Carsten Wiecher Kostal Automobil Elektrik GmbH & Co. KG
14:20
20m
Talk
Scaling Test Case Generation For Expressive Decision Tables
Previous Editions
Supriya Agrawal Tata Consultancy Services Ltd. (TCS), R. Venkatesh , Ulka Shrotri Tata Consultancy Services Ltd. (TCS), Amey Zare TCS Research, Sagar Verma Tata Consultancy Services Ltd. (TCS)
DOI
14:40
20m
Talk
Spectacular: Finding Laws from 25 Trillion Programs
Research Papers
Matthías Páll Gissurarson Chalmers University of Technology, Sweden, Diego Roque Dark Forest Technologies, James Koppel Massachusetts Institute of Technology, USA
15:00
20m
Talk
Pairwise Testing Revisited for Structured Data With Constraints
Research Papers
Luca V. Sartori LAAS-CNRS, Helene Waeselynck LAAS-CNRS, Jérémie Guiochet LAAS-CNRS
15:20
10m
Talk
RICK: Generating Mocks from Production Data
Tool Demo
Deepika Tiwari KTH Royal Institute of Technology, Martin Monperrus KTH Royal Institute of Technology, Benoit Baudry KTH

Wed 19 Apr

Displayed time zone: Dublin change

10:30 - 11:00
Coffee break Social
10:30
30m
Coffee break
Break
Social

11:00 - 12:30
Session 12: Regression Testing Previous Editions / Industry / Testing Tools / Journal-First Papers at Grand canal
Chair(s): Yvan Labiche Carleton University
11:00
20m
Talk
Severity-Aware Prioritization of System-Level Regression Tests in Automotive Software
Industry
Roland Würsching Technical University of Munich, Daniel Elsner TU Munich, Fabian Leinen Technical University of Munich, Alexander Pretschner Technical University of Munich, Georg Grueneissl MAN Truck & Bus SE, Thomas Neumeyr MAN Truck & Bus SE, Tobias Vosseler MAN Truck & Bus SE
11:20
20m
Talk
BinaryRTS: Cross-language Regression Test Selection for C++ Binaries in CI
Industry
Daniel Elsner TU Munich, Severin Kacianka Technical University of Munich, Stephan Lipp Technical University of Munich, Alexander Pretschner TU Munich, Axel Habermann IVU Traffic Technologies, Maria Graber IVU Traffic Technologies, Silke Reimer IVU Traffic Technologies
Pre-print
11:40
20m
Talk
An Overview of Regression Test Prioritization by Delta Displacement in Test Coverage
Journal-First Papers
Shouvick Mondal IIT Gandhinagar, India, Rupesh Nasre IIT Madras, India
Link to publication DOI Media Attached
12:00
20m
Talk
Patterns of Code-to-Test Co-evolution for Automated Test Suite Maintenance
Previous Editions
Samiha Shimmi Northern Illinois University, Mona Rahimi Northern Illinois University
DOI
12:20
20m
Talk
DIRTS: Dependency Injection Aware Regression Test Selection
Testing Tools
Simon Hundsdorfer Technical University of Munich, Daniel Elsner TU Munich, Alexander Pretschner TU Munich
Pre-print
11:00 - 12:30
Session 13: Testing with Static Analysis MethodsResearch Papers / Previous Editions at Pearse suite
Chair(s): Paolo Arcaini National Institute of Informatics
11:00
20m
Talk
Address-Aware Query Caching for Symbolic Execution
Previous Editions
David Trabish Tel Aviv University, Shachar Itzhaky Technion, Noam Rinetzky Tel Aviv University
DOI
11:20
20m
Talk
Automating Differential Testing with Overapproximate Symbolic Execution
Previous Editions
Richard Rutledge Georgia Institute of Technology, Alessandro Orso Georgia Tech
DOI
11:40
20m
Talk
Two Sparsification Strategies for Accelerating Demand-Driven Pointer Analysis
Research Papers
Kadiray Karakaya Heinz Nixdorf Institut, Paderborn University, Eric Bodden
Pre-print Media Attached
12:00
20m
Talk
Model Generation For Java Frameworks
Research Papers
Linghui Luo Amazon Web Services, Goran Piskachev Amazon Web Services, Ranjith Krishnamurthy Fraunhofer IEM, Julian Dolby IBM Research, Eric Bodden Heinz Nixdorf Institut, Paderborn University and Fraunhofer IEM, Martin Schäf Amazon Web Services
12:30 - 14:00
LunchSocial
12:30
90m
Lunch
Lunch
Social

14:00 - 15:40
Session 14: Web TestingJournal-First Papers / Industry / Research Papers at Grand canal
Chair(s): Thomas Laurent JSPS@National Institute of Informatics, Japan
14:00
20m
Talk
Challenges of End-to-End Testing with Selenium WebDriver and How to Face Them: A Survey
Industry
Maurizio Leotta DIBRIS, University of Genova, Italy, Boni Garcia Universidad Carlos III de Madrid, Filippo Ricca Università di Genova, Jim Whitehead University of California, Santa Cruz
14:20
20m
Talk
E2E-Loader: A Framework to Support Performance Testing of Web Applications
Industry
Ermanno Battista Fervento srl, Sergio Di Martino Università degli Studi di Napoli Federico II, Sergio Di Meglio Department of Electrical Engineering and Information Technology Università degli Studi di Napoli Federico II, Fabio Scippacercola Fervento srl, Luigi Libero Lucio Starace Università degli Studi di Napoli Federico II
14:40
20m
Talk
Automated Web Application Testing driven by Pre-recorded Test Cases
Journal-First Papers
Hasan Sozer Ozyegin University, Nezih Sunman Siemens, Yigit Soydan Ozyegin University
15:00
20m
Talk
Robust web element identification for evolving applications by considering visual overlaps
Research Papers
Michel Nass Blekinge Institute of Technology, Riccardo Coppola Politecnico di Torino, Emil Alégroth Blekinge Institute of Technology, Robert Feldt Chalmers University of Technology, Sweden
15:20
20m
Talk
Automating Endurance Test for Flash-based Storage Devices in Samsung Electronics
Industry
Jinkook Kim Samsung Electronics Co., Minseok Jeon Korea University, Sejeong Jang Samsung Electronics Co., Hakjoo Oh Korea University
14:00 - 15:40
Session 15: Flaky TestsPrevious Editions / Research Papers at Pearse suite
Chair(s): John Micco VMware
14:00
20m
Talk
Evaluating Features for Machine Learning Detection of Order- and Non-Order-Dependent Flaky Tests
Previous Editions
Owain Parry The University of Sheffield, Gregory Kapfhammer Allegheny College, Michael Hilton Carnegie Mellon University, Phil McMinn University of Sheffield
DOI
14:20
20m
Talk
An Empirical Study of Flaky Tests in Python
Previous Editions
Martin Gruber BMW Group, University of Passau, Stephan Lukasczyk University of Passau, Florian Kroiß , Gordon Fraser University of Passau
DOI
14:40
20m
Talk
A Survey on How Test Flakiness Affects Developers and What Support They Need To Address It
Previous Editions
Martin Gruber BMW Group, University of Passau, Gordon Fraser University of Passau
DOI
15:00
20m
Talk
Practical Flaky Test Prediction using Common Code Evolution and Test History Data
Research Papers
Martin Gruber BMW Group, University of Passau, Michael Heine BMW Group; Friedrich-Alexander Universität Erlangen-Nürnberg (FAU), Programming Systems Group, Norbert Oster Friedrich-Alexander Universität Erlangen-Nürnberg (FAU), Programming Systems Group, Michael Philippsen Friedrich-Alexander Universität Erlangen-Nürnberg (FAU), Programming Systems Group, Gordon Fraser University of Passau
Pre-print
15:20
20m
Talk
A Qualitative Study on the Sources, Impacts, and Mitigation Strategies of Flaky Tests
Previous Editions
Sarra Habchi Ubisoft, Guillaume Haben University of Luxembourg, Mike Papadakis University of Luxembourg, Luxembourg, Maxime Cordy University of Luxembourg, Luxembourg, Yves Le Traon University of Luxembourg, Luxembourg
DOI
15:30 - 16:00
Coffee breakSocial
15:30
30m
Coffee break
Break
Social

Accepted Papers

Title
Automating Endurance Test for Flash-based Storage Devices in Samsung Electronics
Industry
BinaryRTS: Cross-language Regression Test Selection for C++ Binaries in CI
Industry
Pre-print
Challenges of End-to-End Testing with Selenium WebDriver and How to Face Them: A Survey
Industry
E2E-Loader: A Framework to Support Performance Testing of Web Applications
Industry
Flake Aware Culprit Finding
Industry
Pre-print
Industrial Deployment of Compiler Fuzzing Techniques for Two GPU Shading Languages
Industry
Learning Non-robustness using Simulation-based Testing: a Network Traffic-shaping Case Study
Industry
Pre-print
Severity-Aware Prioritization of System-Level Regression Tests in Automotive Software
Industry
Test Maintenance for Machine Learning Systems: A Case Study in the Automotive Industry
Industry

Call for Papers

ICST 2023 invites high quality submissions in all areas of software testing, verification, and validation. The industry track has the same topics and requires the same rigorousness as the research track. However, the industry track papers should prioritize impact and realism over novelty. We expect that industry track publications report useful testing tools, successful stories and experiences when applying software testing theories and methodologies into practice, or promising prototypes that have the potential to impact practice in the next few years. Ideally they should be evaluated on real-world, industrial data, and discuss non-functional aspects such as scalability and reliability, and also compare to pre-existing industrial best practices. The industry track papers will be published in the ICST 2023 proceedings and listed by IEEE databases.

Topics of interest include, but are not limited to, the following:

  • Applications of software testing theories in practice
  • Applying machine learning techniques to testing
  • Empirical studies
  • Experience reports
  • Formal verification
  • Mobile testing
  • Model based testing
  • Model checking
  • Privacy testing
  • Search based software testing
  • Security testing
  • Software reliability
  • Test automation
  • Testability and design
  • Testing / debugging frameworks and tools
  • Testing AI products
  • Testing and development processes
  • Testing education
  • Domain specific testing, such as: Autonomous vehicles, advanced control systems, mobile or web app testing, embedded systems testing, concurrent systems, GUI testing, etc.

Each submission will be reviewed by at least three members of the ICST Program Committee.

Format

Industry papers must conform to the two-column IEEE conference publication format. Templates for LaTeX and Microsoft Word are available from http://www.ieee.org/conferences_events/conferences/publishing/templates.html: please use the letter format template and conference option.

Papers should be submitted in the PDF format: they must not exceed ten pages, plus up to two pages of references. Submissions will be handled via EasyChair . Make sure when submitting to submit to your intended track as all tracks share the same link.

Papers must neither have been previously accepted for publication nor be under submission in another conference or journal.

For your paper to be published in the ICST 2023 conference proceedings, at least one of the authors of the paper must register for the conference and confirm that she/he will present the paper in person.

Double Anonymous

Industry track papers do not need to be double-anonymyzed. Since industry papers typically rely heavily on the industrial/practical context in which the work was carried out it would be too much to ask to require this context to be anonymized. Please contact the industry track chairs if you have more questions.

FAQ

Q: Does the industry track accept abstracts or short papers?

A: No. The industry track has the same rigorousness as the research papers, therefore, the industry track papers need to present the motivation, methodology, design of experiments / case studies, results, discussions, and related work in a clear and comprehensive way. Abstracts or short papers don’t have enough room to incorporate these required and important contents.

Q: My paper has less than 10 pages, can I submit it to the industry track?

A: Yes, as long as your paper clearly presents the motivation, methodology, design of experiments / case studies, results, discussions, and related work.

Q: Do I need to supply replication material?

A: No. We understand that industry track publications might report experiments using confidential industry data or closed source tools. Therefore the submission is not required to supply all information that is needed to replicate the results.