ICST 2024, 17th IEEE International Conference on Software Testing, Verification and Validation, is thrilled to invite you to participate in our prestigious Industry Track. At ICST 2024, we recognize the crucial role that industry professionals play in shaping the landscape of software testing, verification, and validation. The Industry Track at ICST 2024 aligns seamlessly with the conference’s overarching themes of innovation, quality, and practicality. We extend our warmest invitation to high-caliber submissions that transcend boundaries and bring a fresh perspective to the software testing field.
Supporters
Dates
Tracks
Wed 29 MayDisplayed time zone: Eastern Time (US & Canada) change
Wed 29 May
Displayed time zone: Eastern Time (US & Canada) change
11:00 - 12:40 | Testing and LLMsIndustry / Research Papers at Room 2 & 3 Chair(s): Sudipta Chattopadhyay Singapore University of Technology and Design | ||
11:00 20mIndustry talk | Are We Testing or Being Tested? Exploring the Practical Applications of Large Language Models in Software Testing Industry Robson T. de Souza Santos , Italo Santos Northern Arizona University, Cleyton V. C. de Magalhaes CESAR School, Ronnie de Souza Santos University of Calgary | ||
11:20 20mResearch paper | Improving Patch Correctness Analysis via Random Testing and Large Language Models Research Papers Facundo Molina IMDEA Software Institute, Juan Manuel Copia IMDEA Software Institute; Universidad Politécnica de Madrid, Alessandra Gorla IMDEA Software Institute Pre-print | ||
11:40 20mResearch paper | Intent-Driven Mobile GUI Testing with Autonomous Large Language Model Agents Research Papers Juyeon Yoon Korea Advanced Institute of Science and Technology, Robert Feldt Chalmers University of Technology, Sweden, Shin Yoo Korea Advanced Institute of Science and Technology Pre-print | ||
12:00 20mResearch paper | KAT: Dependency-aware Automated API Testing with Large Language Models Research Papers Tri Le , Thien Tran , Duy Cao , Vy Le , Vu Nguyen Head of Research, Katalon Inc.; University of Science, VNU-HCM, Vietnam, Tien N. Nguyen University of Texas at Dallas | ||
12:20 20mResearch paper | Quantizing Large-Language Models for Predicting Flaky Tests Research Papers Shanto Rahman The University of Texas at Austin, Abdelrahman Baz , Sasa Misailovic University of Illinois at Urbana-Champaign, August Shi The University of Texas at Austin | ||
15:30 - 17:00 | FuzzingJournal-First Papers / Research Papers / Testing Tools and Demonstration at Room 1 Chair(s): Sahar Tahvili Ericsson AB | ||
15:30 20mResearch paper | MSGFuzzer: Message Sequence Guided Industrial Robot Protocol Fuzzing Research Papers Yang Zhang Beijing Key Laboratory of IOT Information Security Technology, Institute of Information Engineering, CAS, Beijing, China; School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China, Dongliang Fang Beijing Key Laboratory of IOT Information Security Technology, Institute of Information Engineering, CAS, China; School of Cyber Security, University of Chinese Academy of Sciences, China, Puzhuo Liu Beijing Key Laboratory of IOT Information Security Technology, Institute of Information Engineering, CAS, Beijing, China； School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China；, Laile Xi , Xiao Lu , Xin Chen , Shuaizong Si , Limin Sun Institute of Information Engineering, Chinese Academy of Sciences, School of Cyber Security, University of Chinese Academy of Sciences, | ||
15:50 20mResearch paper | U-Fuzz: Stateful Fuzzing of IoT Protocols on COTS Devices Research Papers Shang Zewen , Matheus Eduardo Garbelini , Sudipta Chattopadhyay Singapore University of Technology and Design | ||
16:10 20mResearch paper | Formatted Stateful Greybox Fuzzing of TLS server Research Papers Fan Hu , Jiangan Ji , Hui Shu , Zheming Li Tsinghua University, Tieming Liu , Chao Zhang Tsinghua University | ||
16:30 20mLong-paper | A Fuzzing-Based Test-Creation Approach for Evaluating Digital TV Receivers via Transport Streams Journal-First Papers Fabrício Izumi Banceira , Eddie Lima Samsung Electronics, Brazil, Lucas C. Cordeiro University of Manchester, UK and Federal University of Amazonas, Brazil, Orlewilson Maia , Rômulo Fabrício , Bruno Farias University of Manchester, UK, Aguinaldo Silva | ||
16:50 10mDemonstration | MOTIF: A tool for Mutation Testing with Fuzzing Testing Tools and Demonstration Jaekwon Lee University of Ottawa & University of Luxembourg, Enrico Viganò University of Luxembourg, Fabrizio Pastore University of Luxembourg, Lionel Briand University of Ottawa, Canada; Lero centre, University of Limerick, Ireland | ||
Thu 30 MayDisplayed time zone: Eastern Time (US & Canada) change
Thu 30 May
Displayed time zone: Eastern Time (US & Canada) change
11:00 - 12:40 | Test FlakinessJournal-First Papers / Research Papers / Industry at Room 1 Chair(s): Andrea Stocco Technical University of Munich, fortiss | ||
11:00 20mLong-paper | Test Code Flakiness in Mobile Apps: The Developer's Perspective Journal-First Papers Valeria Pontillo Vrije Universiteit Brussel, Fabio Palomba University of Salerno, Filomena Ferrucci University of Salerno Link to publication | ||
11:20 20mLong-paper | Flakiness goes live: Insights from an In Vivo testing simulation study Journal-First Papers Morena Barboni University of Camerino, Antonia Bertolino National Research Council, Italy, Guglielmo De Angelis CNR-IASI | ||
11:40 20mResearch paper | 262,447 Test Failures Later: An Empirical Evaluation of Flaky Failure Classifiers Research Papers Abdulrahman Alshammari George Mason University, Paul Ammann George Mason University, USA, Michael Hilton Carnegie Mellon University, Jonathan Bell Northeastern University | ||
12:00 20mResearch paper | Automatically Reproducing Timing-Dependent Flaky-Test Failures Research Papers Shanto Rahman The University of Texas at Austin, Aaron Massey George Mason University, Wing Lam George Mason University, August Shi The University of Texas at Austin, Jonathan Bell Northeastern University | ||
12:20 20mIndustry talk | Cost of Flaky Tests in CI: An Industrial Case Study Industry Fabian Leinen Technical University of Munich, Daniel Elsner TU Munich, Alexander Pretschner TU Munich, Andreas Stahlbauer , Michael Sailer , Elmar Juergens CQSE GmbH Pre-print | ||
11:00 - 12:40 | Mutation Testing and Test PrioritizaitonResearch Papers / Journal-First Papers / Industry at Room 2 & 3 Chair(s): Facundo Molina IMDEA Software Institute | ||
11:00 20mIndustry talk | Towards Mutation-guided Test Suites for Smart Contracts Industry Pre-print | ||
11:20 20mResearch paper | On the Coupling between Vulnerabilities and LLM-generated Mutants: A Study on Vul4J dataset Research Papers Aayush Garg Luxembourg Institute of Science and Technology, Renzo Degiovanni Luxembourg Institute of Science and Technology, Mike Papadakis University of Luxembourg, Yves Le Traon University of Luxembourg, Luxembourg Pre-print | ||
11:40 20mLong-paper | Mutation Testing Optimisations using the Clang Front-end Journal-First Papers Sten Vercammen , Serge Demeyer University of Antwerp; Flanders Make, Markus Borg CodeScene, Niklas Pettersson , Görel Hedin Lund University | ||
12:00 20mResearch paper | MACS: Multi-agent Adversarial Reinforcement Learning for Finding Diverse Critical Driving Scenarios Research Papers Shuting Kang University of Chinese Academy of Sciences, Qian Dong Institute of Software Chinese Academy of Sciences, Yunzhi Xue Institute of Software Chinese Academy of Sciences, Yanjun Wu Institute of Software, Chinese Academy of Sciences | ||
12:20 20mLong-paper | Lessons learned from replicating a study on information-retrieval-based test case prioritization Journal-First Papers Nasir Mehmood Minhas Mälardalen University, Mohsin Irshad , Kai Petersen University of Applied Sciences Flensburg, Germany / Blekinge Institute of Technology, Sweden, Jürgen Börstler Blekinge Institute of Technology | ||
14:00 - 15:00 | Software Testing Education and Fault LocalizationResearch Papers / Journal-First Papers at Room 1 Chair(s): Shin Yoo Korea Advanced Institute of Science and Technology | ||
14:00 20mResearch paper | State of the Practice in Software Testing Teaching in Four European Countries Research Papers Porfirio Tramontana Department of Electrical Engineering and Information Technologies, University of Naples Federico II, Italy, Beatriz Marín Universitat Politècnica de València, Ana Paiva Faculty of Engineering, University of Porto and INESC TEC, Alexandra Mendes Faculty of Engineering, University of Porto & INESC TEC, Tanja E. J. Vos Universitat Politècnica de València and Open Universiteit, Domenico Amalfitano University of Naples Federico II, Felix Cammaerts KU Leuven, Monique Snoeck Katholieke Universiteit Leuven, Anna Rita Fasolino Federico II University of Naples | ||
14:20 20mLong-paper | Can gamification help in software testing education? Findings from an empirical study Journal-First Papers Raquel Blanco , Manuel Trinidad , María José Suárez-Cabal , Alejandro Calderón Sánchez , Mercedes Ruiz University of Cadiz, Javier Tuya Computer Science Department, University of Oviedo | ||
14:40 20mResearch paper | FusionFL: A statement-level feature fusion based fault localization approach Research Papers Yanbo Zhang , Yawen Wang State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Dongming Zhu , Wenjing Liu | ||
Fri 31 MayDisplayed time zone: Eastern Time (US & Canada) change
Fri 31 May
Displayed time zone: Eastern Time (US & Canada) change
11:00 - 12:20 | Testing and ApplicationsResearch Papers / Testing Tools and Demonstration / Industry at Room 1 Chair(s): Jeremy Bradbury Ontario Tech University | ||
11:00 20mIndustry talk | In industrial embedded software, are some compilation errors easier to localize and fix than others? Industry Han Fu , Sigrid Eldh Ericsson AB, Mälardalen University, Carleton Unviersity, Kristian Wiklund Ericsson AB, Andreas Ermedahl , Philipp Haller KTH Royal Institute of Technology, Cyrille Artho KTH Royal Institute of Technology, Sweden | ||
11:20 20mResearch paper | Brewing Up Reliability: Espresso Test Generation for Android Apps Research Papers Iván Arcuschin Moreno University of Buenos Aires, Argentina, Lisandro Di Meo , Michael Auer University of Passau, Juan Pablo Galeotti University of Buenos Aires, Gordon Fraser University of Passau DOI Pre-print | ||
11:40 20mResearch paper | Differential Optimization Testing of Gremlin-Based Graph Database Systems Research Papers Yingying Zheng Institute of Software Chinese Academy of Sciences, Wensheng Dou Institute of Software Chinese Academy of Sciences, Lei Tang Institute of Software Chinese Academy of Sciences, Ziyu Cui Institute of Software Chinese Academy of Sciences, Jiansen Song Institute of Software Chinese Academy of Sciences, Ziyue Cheng , Wei Wang , Jun Wei Institute of Software at Chinese Academy of Sciences; University of Chinese Academy of Sciences; University of Chinese Academy of Sciences Chongqing School, Hua Zhong , Tao Huang Institute of Software Chinese Academy of Sciences | ||
12:00 10mDemonstration | MLHCBugs: A Framework to Reproduce Real Faults in Healthcare Machine Learning Applications Testing Tools and Demonstration Guna Sekaran Jaganathan , Nazmul Kazi , Indika Kahanda University of North Florida, Upulee Kanewala University of North Florida | ||
12:10 10mDemonstration | The GitHub Recent Bugs Dataset for Evaluating LLM-based Debugging Applications Testing Tools and Demonstration Jae Yong Lee , Sungmin Kang , Juyeon Yoon Korea Advanced Institute of Science and Technology, Shin Yoo Korea Advanced Institute of Science and Technology | ||
11:00 - 12:20 | Testing with and for Deep and Reinforcement LearningResearch Papers / Industry at Room 2 & 3 Chair(s): Paolo Arcaini National Institute of Informatics | ||
11:00 20mResearch paper | METAL: Metamorphic Testing Framework for Analyzing Large-Language Model Qualities Research Papers Sangwon Hyun University of Adelaide, Mingyu Guo , Muhammad Ali Babar School of Computer Science, The University of Adelaide | ||
11:20 20mIndustry talk | End-to-end RPA-like testing using reinforcement learning Industry Ciprian Paduraru University of Bucharest, Rares Cristea University of Bucharest, Alin Stefanescu University of Bucharest | ||
11:40 20mResearch paper | Spectral Analysis of the Relation between Deep Learning Faults and Neural Activation Values Research Papers | ||
12:00 20mResearch paper | Learning Environment Models with Continuous Stochastic Dynamics - with an Application to Deep RL Testing Research Papers Martin Tappler TU Wien, Austria, Edi Muskardin , Bernhard Aichernig Graz University of Technology, Bettina Könighofer | ||
13:30 - 14:30 | Testing and RepairResearch Papers / Industry at Room 1 Chair(s): August Shi The University of Texas at Austin | ||
13:30 20mResearch paper | Evolutionary Testing for Program Repair Research Papers Haifeng Ruan , Hoang Lam Nguyen Humboldt-Universität zu Berlin, Ridwan Salihin Shariffdeen National University of Singapore, Yannic Noller Singapore University of Technology and Design, Abhik Roychoudhury National University of Singapore | ||
13:50 20mResearch paper | Does Going Beyond Branch Coverage Make Program Repair Tools More Reliable? Research Papers Amirfarhad Nilizadeh University of Central Florida, Gary T. Leavens University of Central Florida, Corina Pasareanu CMU, NASA, KBR, Xuan-Bach D. Le The University of Melbourne, David Cok CEA, LIST, Software Safety and Security Laboratory | ||
14:10 20mIndustry talk | SafeRevert: When Can Breaking Changes be Automatically Reverted? Industry Pre-print | ||
13:30 - 14:30 | Metamorphic and Combinatorial TestingIndustry / Journal-First Papers at Room 2 & 3 Chair(s): Wishnu Prasetya Utrecht University | ||
13:30 20mIndustry talk | Metamorphic Testing of an Autonomous Delivery Robots Scheduler Industry Thomas Laurent Lero@Trinity College Dublin, Paolo Arcaini National Institute of Informatics
, Xiao-Yi Zhang , Fuyuki Ishikawa National Institute of Informatics Pre-print | ||
13:50 20mLong-paper | Design, implementation, and validation of a benchmark generator for combinatorial interaction testing tools Journal-First Papers | ||
Accepted Papers
Call for Papers
ICST 2024 - Industry Track
ICST 2024, 17th IEEE International Conference on Software Testing, Verification and Validation, is thrilled to invite you to participate in our prestigious Industry Track. At ICST 2024, we recognize the crucial role that industry professionals play in shaping the landscape of software testing, verification, and validation.
About ICST 2024 - Industry Track
The Industry Track at ICST 2024 aligns seamlessly with the conference’s overarching themes of innovation, quality, and practicality. We extend our warmest invitation to high-caliber submissions that transcend boundaries and bring a fresh perspective to the software testing field.
Our Expectations
While the Industry Track shares the same overarching topics as the Research Track, we emphasize real-world impact and practicality over novelty. We are looking for submissions that:
- Highlight Useful Testing Tools: Present groundbreaking testing tools that have demonstrable real-world utility.
- Share Success Stories: Showcase your experiences and success stories in applying software testing theories and methodologies to practical scenarios.
- Unveil Promising Prototypes: Introduce prototypes with the potential to revolutionize software testing practices in the near future.
Real-World Evaluation and Beyond
We believe in the power of practicality. Industry Track submissions should ideally be grounded in real-world, industrial data. In addition, we encourage discussions on non-functional aspects such as scalability and reliability, as well as comparisons to established industrial best practices.
Publication and Recognition
Selected Industry Track papers will be published in the ICST 2024 proceedings and prominently listed in the esteemed IEEE databases. This is your opportunity to contribute to the global knowledge repository and gain recognition for your valuable insights.
Topics of Interest
Topics for submission to the Industry Track include, but are not limited to:
- Applications of software testing theories in practice
- Applying machine learning techniques to testing
- Empirical studies
- Experience reports
- Formal verification
- Mobile testing
- Model-based testing
- Model checking
- Privacy testing
- Search-based software testing
- Security testing
- Software reliability
- Test automation
- Testability and design
- Testing/debugging frameworks and tools
- Testing AI products
- Testing and development processes
- Testing education
- Domain-specific testing (e.g., autonomous vehicles, IoT applications, advanced control systems, mobile or web app testing, embedded systems testing, blockchain applications, concurrent systems, GUI testing, etc.)
Peer Review Process
Each submission will undergo a rigorous evaluation process, reviewed by at least three members of the ICST Program Committee. Don’t miss this opportunity to be a driving force in the advancement of software testing and validation. Join us at ICST 2024’s Industry Track and be part of the transformation.
Important Dates
- Abstract Deadline: Wednesday, October 18, 2023
- Full Paper Submission: Wednesday, October 25, 2023
- Author Notification: Wednesday, December 6, 2023
- Camera Ready: Friday, January 19, 2024
For comprehensive submission guidelines and further information, please visit our conference website: https://conf.researchr.org/home/icst-2024
Double Anonymous
Industry track papers do not need to be double-anonymized. Since industry papers typically rely heavily on the industrial/practical context in which the work was carried out it would be too much to ask to require this context to be anonymized. Please contact the industry track chairs if you have more questions.
Together, let’s shape the future of software testing! We eagerly anticipate your contributions to ICST 2024 - Industry Track
Submission Format
Industry papers must conform to the two-column IEEE conference publication format. Templates for LaTeX and Microsoft Word are available from http://www.ieee.org/conferences_events/conferences/publishing/templates.html: please use the letter format template and conference option.
Papers should be submitted in PDF format: they must not exceed ten pages, plus up to two pages of references. Submissions will be handled via https://easychair.org/my/conference?conf=icst2024industrytrac.
Papers must neither have been previously accepted for publication nor be under submission in another conference or journal.
For your paper to be published in the ICST 2024 conference proceedings, at least one of the authors of the paper must register for the conference and confirm that she/he will present the paper in person.
The abstract submission is an optional step. You are welcome to submit the full paper directly without submitting an abstract. We look forward to receiving your contributions!
Important Dates AoE (UTC-12h)
| Fri 19 Jan 2024 Camera Ready |
| Wed 6 Dec 2023 Author Notification |
| Wed 25 Oct 2023 Full Paper Submission |
| Wed 18 Oct 2023 Abstract Deadline |
Submission Link
Annibale PanichellaCo-chair
Delft University of Technology
Netherlands
Sahar TahviliCo-chair
Ericsson AB
Sweden
Andrea ArcuriPC Member
Kristiania University College and Oslo Metropolitan University
Norway
Pouria DerakhshanfarPC Member
JetBrains Research
Netherlands
Ankush DesaiPC Member
University of California, Berkeley
Thomas DurieuxPC Member
TU Delft
Netherlands
Sigrid EldhPC Member
Ericsson AB, Mälardalen University, Carleton Unviersity
Sweden
Michael FeldererPC Member
German Aerospace Center (DLR) & University of Cologne
Germany
Gordon FraserPC Member
University of Passau
Francisco Gomes de Oliveira NetoPC Member
Chalmers | University of Gothenburg
Sweden
Mark HarmanPC Member
Meta Platforms, Inc. and UCL
United Kingdom
Joseph HejderupPC Member
Endor Labs & Delft University of Technology
Netherlands
Yu JiangPC Member
Tsinghua university
China
Maria KechagiaPC Member
University College London
United Kingdom
Teng LongPC Member
Apple
United States
Daniel SundmarkPC Member
Mälardalen University