ICST 2024
Mon 27 - Fri 31 May 2024 Canada

The Journal-First Track will give authors of selected papers an opportunity to present and discuss their work with the ICST 2024 audience. The details are presented in the Call for Journal-First Presentations.

Supporters
Gold
Bronze
Dates
Tracks
You're viewing the program in a time zone which is different from your device's time zone change time zone

Wed 29 May

Displayed time zone: Eastern Time (US & Canada) change

11:00 - 12:40
Test Generation and Test CausalityIndustry / Research Papers / Journal-First Papers at Room 1
Chair(s): Jonathan Bell Northeastern University
11:00
20m
Research paper
Automatically Removing Unnecessary Stubbings from Test Suites
Research Papers
Mengzhen Li University of Minnesota, Mattia Fazzini University of Minnesota
11:20
20m
Industry talk
BugOut: Automated Test Generation and Bug Detection for Low-Code
Industry
11:40
20m
Research paper
Enhanced Fast and Reliable Statistical Vulnerability Root Cause Analysis with Sanitizer
Research Papers
Zhuo Yan Ocean University of China, Haipeng Qu Ocean University of China, Lingyun Ying Qi An Xin Group Corp., Ke Liu , Chao Qu
12:00
20m
Research paper
Causal Test Adequacy
Research Papers
Michael Foster The University of Sheffield, Christopher Wild , Neil Walkinshaw University of Sheffield, Robert Hierons The University of Sheffield
12:20
20m
Long-paper
Summary of Automated Code-based Test Case Reuse for Software Product Line Testing
Journal-First Papers
Pilsu Jung Gyeongsang National Unviersity, Seonah Lee Gyeongsang National University, Uicheon Lee
11:00 - 12:40
Testing and LLMsIndustry / Research Papers at Room 2 & 3
Chair(s): Sudipta Chattopadhyay Singapore University of Technology and Design
11:00
20m
Industry talk
Are We Testing or Being Tested? Exploring the Practical Applications of Large Language Models in Software Testing
Industry
Robson T. de Souza Santos , Italo Santos Northern Arizona University, Cleyton V. C. de Magalhaes CESAR School, Ronnie de Souza Santos University of Calgary
11:20
20m
Research paper
Improving Patch Correctness Analysis via Random Testing and Large Language Models
Research Papers
Facundo Molina IMDEA Software Institute, Juan Manuel Copia IMDEA Software Institute; Universidad Politécnica de Madrid, Alessandra Gorla IMDEA Software Institute
Pre-print
11:40
20m
Research paper
Intent-Driven Mobile GUI Testing with Autonomous Large Language Model Agents
Research Papers
Juyeon Yoon Korea Advanced Institute of Science and Technology, Robert Feldt Chalmers University of Technology, Sweden, Shin Yoo Korea Advanced Institute of Science and Technology
Pre-print
12:00
20m
Research paper
KAT: Dependency-aware Automated API Testing with Large Language Models
Research Papers
Tri Le , Thien Tran , Duy Cao , Vy Le , Vu Nguyen Head of Research, Katalon Inc.; University of Science, VNU-HCM, Vietnam, Tien N. Nguyen University of Texas at Dallas
12:20
20m
Research paper
Quantizing Large-Language Models for Predicting Flaky Tests
Research Papers
Shanto Rahman The University of Texas at Austin, Abdelrahman Baz , Sasa Misailovic University of Illinois at Urbana-Champaign, August Shi The University of Texas at Austin
15:30 - 17:00
15:30
20m
Research paper
MSGFuzzer: Message Sequence Guided Industrial Robot Protocol Fuzzing
Research Papers
Yang Zhang Beijing Key Laboratory of IOT Information Security Technology, Institute of Information Engineering, CAS, Beijing, China; School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China, Dongliang Fang Beijing Key Laboratory of IOT Information Security Technology, Institute of Information Engineering, CAS, China; School of Cyber Security, University of Chinese Academy of Sciences, China, Puzhuo Liu Beijing Key Laboratory of IOT Information Security Technology, Institute of Information Engineering, CAS, Beijing, China; School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China;, Laile Xi , Xiao Lu , Xin Chen , Shuaizong Si , Limin Sun Institute of Information Engineering, Chinese Academy of Sciences, School of Cyber Security, University of Chinese Academy of Sciences,
15:50
20m
Research paper
U-Fuzz: Stateful Fuzzing of IoT Protocols on COTS Devices
Research Papers
Shang Zewen , Matheus Eduardo Garbelini , Sudipta Chattopadhyay Singapore University of Technology and Design
16:10
20m
Research paper
Formatted Stateful Greybox Fuzzing of TLS server
Research Papers
Fan Hu , Jiangan Ji , Hui Shu , Zheming Li Tsinghua University, Tieming Liu , Chao Zhang Tsinghua University
16:30
20m
Long-paper
A Fuzzing-Based Test-Creation Approach for Evaluating Digital TV Receivers via Transport Streams
Journal-First Papers
Fabrício Izumi Banceira , Eddie Lima Samsung Electronics, Brazil, Lucas C. Cordeiro University of Manchester, UK and Federal University of Amazonas, Brazil, Orlewilson Maia , Rômulo Fabrício , Bruno Farias University of Manchester, UK, Aguinaldo Silva
16:50
10m
Demonstration
MOTIF: A tool for Mutation Testing with Fuzzing
Testing Tools and Demonstration
Jaekwon Lee University of Ottawa & University of Luxembourg, Enrico Viganò University of Luxembourg, Fabrizio Pastore University of Luxembourg, Lionel Briand University of Ottawa, Canada; Lero centre, University of Limerick, Ireland
15:30 - 17:00
Testing Autonomous Driving SystemsResearch Papers / Testing Tools and Demonstration at Room 2 & 3
Chair(s): Nargiz Humbatova USI Lugano
15:30
20m
Research paper
Adversarial Testing with Reinforcement Learning: A Case Study on Autonomous Driving
Research Papers
Andréa Doreste , Matteo Biagiola Università della Svizzera italiana, Paolo Tonella USI Lugano
15:50
20m
Research paper
Assessing Quality Metrics for Neural Reality Gap Input Mitigation in Autonomous Driving Testing
Research Papers
Stefano Carlo Lambertenghi Technische Universität München, fortiss GmbH, Andrea Stocco Technical University of Munich, fortiss
Pre-print
16:10
20m
Research paper
Predicting Safety Misbehaviours in Autonomous Driving Systems using Uncertainty Quantification
Research Papers
Ruben Grewal , Paolo Tonella USI Lugano, Andrea Stocco Technical University of Munich, fortiss
Pre-print
16:30
20m
Research paper
AURORA: Navigating UI Tarpits via Automated Neural Screen Understanding
Research Papers
Safwat Ali Khan George Mason University, Wenyu Wang University of Illinois Urbana-Champaign, Yiran Ren , Bin Zhu , Jiangfan Shi , Wing Lam George Mason University, Kevin Moran University of Central Florida
Pre-print
16:50
10m
Demonstration
U-Fuzz: A Tool for Stateful Fuzzing of IoT Protocols on COTS Devices
Testing Tools and Demonstration
Shang Zewen , Matheus Eduardo Garbelini , Sudipta Chattopadhyay Singapore University of Technology and Design

Thu 30 May

Displayed time zone: Eastern Time (US & Canada) change

11:00 - 12:40
Test FlakinessJournal-First Papers / Research Papers / Industry at Room 1
Chair(s): Andrea Stocco Technical University of Munich, fortiss
11:00
20m
Long-paper
Test Code Flakiness in Mobile Apps: The Developer's Perspective
Journal-First Papers
Valeria Pontillo Vrije Universiteit Brussel, Fabio Palomba University of Salerno, Filomena Ferrucci University of Salerno
Link to publication
11:20
20m
Long-paper
Flakiness goes live: Insights from an In Vivo testing simulation study
Journal-First Papers
Morena Barboni University of Camerino, Antonia Bertolino National Research Council, Italy, Guglielmo De Angelis CNR-IASI
11:40
20m
Research paper
262,447 Test Failures Later: An Empirical Evaluation of Flaky Failure Classifiers
Research Papers
Abdulrahman Alshammari George Mason University, Paul Ammann George Mason University, USA, Michael Hilton Carnegie Mellon University, Jonathan Bell Northeastern University
12:00
20m
Research paper
Automatically Reproducing Timing-Dependent Flaky-Test Failures
Research Papers
Shanto Rahman The University of Texas at Austin, Aaron Massey George Mason University, Wing Lam George Mason University, August Shi The University of Texas at Austin, Jonathan Bell Northeastern University
12:20
20m
Industry talk
Cost of Flaky Tests in CI: An Industrial Case Study
Industry
Fabian Leinen Technical University of Munich, Daniel Elsner TU Munich, Alexander Pretschner TU Munich, Andreas Stahlbauer , Michael Sailer , Elmar Juergens CQSE GmbH
Pre-print
11:00 - 12:40
Mutation Testing and Test PrioritizaitonResearch Papers / Journal-First Papers / Industry at Room 2 & 3
Chair(s): Facundo Molina IMDEA Software Institute
11:00
20m
Industry talk
Towards Mutation-guided Test Suites for Smart Contracts
Industry
Pre-print
11:20
20m
Research paper
On the Coupling between Vulnerabilities and LLM-generated Mutants: A Study on Vul4J dataset
Research Papers
Aayush Garg Luxembourg Institute of Science and Technology, Renzo Degiovanni Luxembourg Institute of Science and Technology, Mike Papadakis University of Luxembourg, Yves Le Traon University of Luxembourg, Luxembourg
Pre-print
11:40
20m
Long-paper
Mutation Testing Optimisations using the Clang Front-end
Journal-First Papers
Sten Vercammen , Serge Demeyer University of Antwerp; Flanders Make, Markus Borg CodeScene, Niklas Pettersson , Görel Hedin Lund University
12:00
20m
Research paper
MACS: Multi-agent Adversarial Reinforcement Learning for Finding Diverse Critical Driving Scenarios
Research Papers
Shuting Kang University of Chinese Academy of Sciences, Qian Dong Institute of Software Chinese Academy of Sciences, Yunzhi Xue Institute of Software Chinese Academy of Sciences, Yanjun Wu Institute of Software, Chinese Academy of Sciences
12:20
20m
Long-paper
Lessons learned from replicating a study on information-retrieval-based test case prioritization
Journal-First Papers
Nasir Mehmood Minhas Mälardalen University, Mohsin Irshad , Kai Petersen University of Applied Sciences Flensburg, Germany / Blekinge Institute of Technology, Sweden, Jürgen Börstler Blekinge Institute of Technology
14:00 - 15:00
Software Testing Education and Fault LocalizationResearch Papers / Journal-First Papers at Room 1
Chair(s): Shin Yoo Korea Advanced Institute of Science and Technology
14:00
20m
Research paper
State of the Practice in Software Testing Teaching in Four European Countries
Research Papers
Porfirio Tramontana Department of Electrical Engineering and Information Technologies, University of Naples Federico II, Italy, Beatriz Marín Universitat Politècnica de València, Ana Paiva Faculty of Engineering, University of Porto and INESC TEC, Alexandra Mendes Faculty of Engineering, University of Porto & INESC TEC, Tanja E. J. Vos Universitat Politècnica de València and Open Universiteit, Domenico Amalfitano University of Naples Federico II, Felix Cammaerts KU Leuven, Monique Snoeck Katholieke Universiteit Leuven, Anna Rita Fasolino Federico II University of Naples
14:20
20m
Long-paper
Can gamification help in software testing education? Findings from an empirical study
Journal-First Papers
Raquel Blanco , Manuel Trinidad , María José Suárez-Cabal , Alejandro Calderón Sánchez , Mercedes Ruiz University of Cadiz, Javier Tuya Computer Science Department, University of Oviedo
14:40
20m
Research paper
FusionFL: A statement-level feature fusion based fault localization approach
Research Papers
Yanbo Zhang , Yawen Wang State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Dongming Zhu , Wenjing Liu
14:00 - 15:00
Verification and TestingIndustry / Journal-First Papers / Research Papers at Room 2 & 3
Chair(s): Franz Wotawa Graz University of Technology
14:00
20m
Industry talk
Randomised Testing of the Compiler for a Verification-Aware Programming Language
Industry
Alastair F. Donaldson Imperial College London, Dilan Sheth Imperial College London, Jean-Baptiste Tristan Amazon Web Services, Alex Usher Imperial College London
14:20
20m
Long-paper
Automatically Generating Test Cases for Safety-Critical Software via Symbolic Execution
Journal-First Papers
Elson Kurian University of Milano Bicocca, Daniela Briola University of Milano Bicocca, Pietro Braione University of Milano-Bicocca, Giovanni Denaro University of Milano - Bicocca
14:40
20m
Research paper
EmoSTL: Formal Spatial-Temporal Verification of Emotion Specifications in Computer Games
Research Papers
Saba Gholizadeh Ansari Utrecht University, Wishnu Prasetya Utrecht University, Frank Dignum Umea University, Mehdi Dastani , Gabriele Keller Utrecht University
Pre-print

Fri 31 May

Displayed time zone: Eastern Time (US & Canada) change

11:00 - 12:20
Testing and ApplicationsResearch Papers / Testing Tools and Demonstration / Industry at Room 1
Chair(s): Jeremy Bradbury Ontario Tech University
11:00
20m
Industry talk
In industrial embedded software, are some compilation errors easier to localize and fix than others?
Industry
Han Fu , Sigrid Eldh Ericsson AB, Mälardalen University, Carleton Unviersity, Kristian Wiklund Ericsson AB, Andreas Ermedahl , Philipp Haller KTH Royal Institute of Technology, Cyrille Artho KTH Royal Institute of Technology, Sweden
11:20
20m
Research paper
Brewing Up Reliability: Espresso Test Generation for Android Apps
Research Papers
Iván Arcuschin Moreno University of Buenos Aires, Argentina, Lisandro Di Meo , Michael Auer University of Passau, Juan Pablo Galeotti University of Buenos Aires, Gordon Fraser University of Passau
DOI Pre-print
11:40
20m
Research paper
Differential Optimization Testing of Gremlin-Based Graph Database Systems
Research Papers
Yingying Zheng Institute of Software Chinese Academy of Sciences, Wensheng Dou Institute of Software Chinese Academy of Sciences, Lei Tang Institute of Software Chinese Academy of Sciences, Ziyu Cui Institute of Software Chinese Academy of Sciences, Jiansen Song Institute of Software Chinese Academy of Sciences, Ziyue Cheng , Wei Wang , Jun Wei Institute of Software at Chinese Academy of Sciences; University of Chinese Academy of Sciences; University of Chinese Academy of Sciences Chongqing School, Hua Zhong , Tao Huang Institute of Software Chinese Academy of Sciences
12:00
10m
Demonstration
MLHCBugs: A Framework to Reproduce Real Faults in Healthcare Machine Learning Applications
Testing Tools and Demonstration
Guna Sekaran Jaganathan , Nazmul Kazi , Indika Kahanda University of North Florida, Upulee Kanewala University of North Florida
12:10
10m
Demonstration
The GitHub Recent Bugs Dataset for Evaluating LLM-based Debugging Applications
Testing Tools and Demonstration
Jae Yong Lee , Sungmin Kang , Juyeon Yoon Korea Advanced Institute of Science and Technology, Shin Yoo Korea Advanced Institute of Science and Technology
11:00 - 12:20
Testing with and for Deep and Reinforcement LearningResearch Papers / Industry at Room 2 & 3
Chair(s): Paolo Arcaini National Institute of Informatics
11:00
20m
Research paper
METAL: Metamorphic Testing Framework for Analyzing Large-Language Model Qualities
Research Papers
Sangwon Hyun University of Adelaide, Mingyu Guo , Muhammad Ali Babar School of Computer Science, The University of Adelaide
11:20
20m
Industry talk
End-to-end RPA-like testing using reinforcement learning
Industry
Ciprian Paduraru University of Bucharest, Rares Cristea University of Bucharest, Alin Stefanescu University of Bucharest
11:40
20m
Research paper
Spectral Analysis of the Relation between Deep Learning Faults and Neural Activation Values
Research Papers
Nargiz Humbatova USI Lugano, Gunel Jahangirova King's College London, Paolo Tonella USI Lugano
12:00
20m
Research paper
Learning Environment Models with Continuous Stochastic Dynamics - with an Application to Deep RL Testing
Research Papers
Martin Tappler TU Wien, Austria, Edi Muskardin , Bernhard Aichernig Graz University of Technology, Bettina Könighofer
13:30 - 14:30
Testing and RepairResearch Papers / Industry at Room 1
Chair(s): August Shi The University of Texas at Austin
13:30
20m
Research paper
Evolutionary Testing for Program Repair
Research Papers
Haifeng Ruan , Hoang Lam Nguyen Humboldt-Universität zu Berlin, Ridwan Salihin Shariffdeen National University of Singapore, Yannic Noller Singapore University of Technology and Design, Abhik Roychoudhury National University of Singapore
13:50
20m
Research paper
Does Going Beyond Branch Coverage Make Program Repair Tools More Reliable?
Research Papers
Amirfarhad Nilizadeh University of Central Florida, Gary T. Leavens University of Central Florida, Corina Pasareanu CMU, NASA, KBR, Xuan-Bach D. Le The University of Melbourne, David Cok CEA, LIST, Software Safety and Security Laboratory
14:10
20m
Industry talk
SafeRevert: When Can Breaking Changes be Automatically Reverted?
Industry
Pre-print
13:30 - 14:30
Metamorphic and Combinatorial TestingIndustry / Journal-First Papers at Room 2 & 3
Chair(s): Wishnu Prasetya Utrecht University
13:30
20m
Industry talk
Metamorphic Testing of an Autonomous Delivery Robots Scheduler
Industry
Thomas Laurent Lero@Trinity College Dublin, Paolo Arcaini National Institute of Informatics , Xiao-Yi Zhang , Fuyuki Ishikawa National Institute of Informatics
Pre-print
13:50
20m
Long-paper
Design, implementation, and validation of a benchmark generator for combinatorial interaction testing tools
Journal-First Papers
Andrea Bombarda University of Bergamo, Angelo Gargantini University of Bergamo

Accepted Papers

Title
A Fuzzing-Based Test-Creation Approach for Evaluating Digital TV Receivers via Transport Streams
Journal-First Papers
Automatically Generating Test Cases for Safety-Critical Software via Symbolic Execution
Journal-First Papers
Can gamification help in software testing education? Findings from an empirical study
Journal-First Papers
Design, implementation, and validation of a benchmark generator for combinatorial interaction testing tools
Journal-First Papers
Flakiness goes live: Insights from an In Vivo testing simulation study
Journal-First Papers
Lessons learned from replicating a study on information-retrieval-based test case prioritization
Journal-First Papers
Mutation Testing Optimisations using the Clang Front-end
Journal-First Papers
Summary of Automated Code-based Test Case Reuse for Software Product Line Testing
Journal-First Papers
Test Code Flakiness in Mobile Apps: The Developer's Perspective
Journal-First Papers
Link to publication

Call for Journal-First Presentations

ICST 2024 invites submissions for presentations of journal-first papers published recently in prestigious software engineering journals in all areas of software testing, verification, and validation. This will both enrich the ICST program as well as offer the authors an opportunity to present their work to the community.


Eligibility

Applications for journal papers published in:

  • Journal of Systems and Software (JSS)
  • Software Quality Journal (SQJ)
  • Information and Software Technology (IST)
  • Software Testing, Verification and Reliability (STVR)

Other journals may be considered on a case-by-case basis. A journal-first presentation submitted to ICST 2024 must adhere to the following criteria:

  • The paper was accepted for publication (i.e., the date of the acceptance notification) no earlier than 1 January 2022.
  • The paper is in the scope of the conference, as defined in the call for ICST 2023 research papers.
  • The paper reports completely new research results or presents novel contributions that were not previously reported in prior published work.
  • The paper has not been presented and is not under consideration for journal-first programs of other conferences.


Topics of Interest

Topics of interest include, but are not limited to, the following:

  • Automated test generation, e.g., fuzz testing, search-based test generation, (dynamic) symbolic execution
  • Manual testing practices and techniques
  • Security testing
  • Model-based testing
  • Test automation
  • Static analysis and symbolic execution
  • Formal verification and model checking
  • Software reliability
  • Social aspects of the software testing process
  • Testability and design
  • Testing and development processes
  • Testing education
  • Testing in specific domains, such as mobile, web, embedded/cyber-physical systems, concurrent, - distributed, cloud, GUI, and real-time systems
  • Testing for learning-enabled software, including deep learning and Large Language Models (LLMs)
  • Testing video games, augmented reality
  • Testing/debugging tools
  • Theory of software testing
  • Empirical studies
  • Experience reports


Submission of Proposals

If you have a journal publication that meets the aforementioned criteria, you can submit a one-page proposal via EasyChair.

The submission should consist of:

  • the paper title;
  • names of all authors;
  • an extended abstract using the two-column IEEE conference publication format (same as the research track submission);
  • a short statement on how the work satisfies the journal-first criteria;
  • a pointer to the original journal paper.

Please list all authors of the original journal paper as authors of your submission in EasyChair.


Important Dates

  • Journal First Submissions Deadline: January 1st 2024
  • Journal First Acceptance Notification: January 5th 2024
  • Submissions close at 23:59 AoE (Anywhere on Earth, UTC-12)


Publication

The JF manuscripts have already been, or will be, published through the journals and therefore will NOT be part of the ICST proceedings. The journal-first papers will be listed in the conference program and the proceeding will only contain the abstract and a pointer to the journal publication.


Presentation

If a submission is accepted for the journal-first program, at least one author of the associated journal paper must register and attend the conference to present the paper in person.