Write a Blog >>
AST 2021
Fri 28 - Sat 29 May 2021
co-located with ICSE 2021

The 2nd ACM/IEEE International Conference on Automation of Software Test AST 2021

Software testing of modern distributed and pervasive systems remains a paramount and costly activity,
and its automation still poses many difficult challenges for both researchers and practitioners.

After a successful event at ICSE 2020, we repropose AST as a collocated event with ICSE 2021 in Madrid, Spain.

The conference theme for AST 2021 is “Automatic Software Testing from the Trenches”.

For more details, take a look to the CFP

The list of accepted papers for 2021 edition is available at this link

Recent news!

We will have a special issue in the Journal of Software: Evolution and Process focused on AST 2021 theme: “Automatic Software Testing from the Trenches”.

We will invite extended versions of the best papers from the conference as well as new submissions on the topic.

Announcing the winner of the AST 2021 Industrial Competition

After the jury’s deliberation, it is our pleasure to inform you that the winner of the AST 2021 Industrial Competition was the paper entitled:

Model-based Automation of Test Scripts Generation Across Product Variants: a Railway Perspective

presented by Alessio Bucaioni and co-authored with Fabio Di Silvestro, Inderjeet Singh, Mehrdad Saadatmand, Henry Muccini and Thorvaldur Jochumsson.

The jury selected this paper mainly due to the potential of this research to be transferred to industry, especially in the context of Industry 4.0.

We want to thank the participation of the authors and the jury which was composed by Dr. Laura García-Borgoñon from ITAInnova, Nicolás Sánchez from G7 Innovation and Dr. Gustavo Aragón from Deloitte.

Dates
You're viewing the program in a time zone which is different from your device's time zone change time zone

Thu 20 May

Displayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

11:30 - 12:00
Conference OpeningAST 2021 at AST Room
Chair(s): María José Escalona University of Seville

YT video: https://www.youtube.com/watch?v=u9O9YgoNZAs

12:00 - 13:15
Test GenerationAST 2021 at AST Room
Chair(s): Francisco Gortázar Universidad Rey Juan Carlos
12:00
30m
Long-paper
Automatic Property-based Testing of GraphQL APIs
AST 2021
Stefan Karlsson ABB AB, Mälardalen University, Adnan Causevic Mälardalen University, Daniel Sundmark Mälardalen University
Pre-print Media Attached
12:30
30m
Long-paper
Automated Performance Testing Based on Active Deep Learning
AST 2021
Ali Sedaghatbaf RISE Research Institutes of Sweden, Mahshid Helali Moghadam RISE Research Institutes of Sweden , Mehrdad Saadatmand RISE Research Institutes of Sweden
Pre-print Media Attached
13:00
15m
Short-paper
Towards exhaustive branch coverage with PathCrawler
AST 2021
Nicky Williams CEA - LIST
Pre-print Media Attached
15:00 - 16:15
Mutation Testing and DebuggingAST 2021 at AST Room
Chair(s): Francisco Gortázar Universidad Rey Juan Carlos
15:00
30m
Long-paper
SuMo: A Mutation Strategy for Solidity Smart Contracts
AST 2021
Pre-print Media Attached
15:30
15m
Short-paper
Extreme mutation testing in practice: An industrial case study
AST 2021
Maik Betka University of Stuttgart, Stefan Wagner University of Stuttgart
Pre-print Media Attached
15:45
30m
Long-paper
Extending Hierarchical Delta Debugging with Hoisting
AST 2021
Daniel Vince , Renáta Hodován University of Szeged, Hungary, Daniella Bársony University of Szeged, Department of Software Engineering, Ákos Kiss University of Szeged, Hungary
Pre-print Media Attached
17:00 - 18:00
KeynoteAST 2021 at AST Room
Chair(s): María José Escalona University of Seville

YT Video: https://www.youtube.com/watch?v=NqzwJHPq9aU

Fri 21 May

Displayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

12:00 - 13:15
Testing for Specific Domains - 1AST 2021 at AST Room
Chair(s): Antonia Bertolino CNR-ISTI
12:00
30m
Long-paper
An Evolutionary Approach to Adapt Tests Across Mobile Apps
AST 2021
Leonardo Mariani University of Milano Bicocca, Mauro Pezze USI Lugano, Switzerland, Valerio Terragni The University of Auckland, Daniele Zuddas Università della Svizzera italiana (USI)
Pre-print Media Attached
12:30
15m
Short-paper
A framework for the automation of testing computer vision systems
AST 2021
Franz Wotawa , Ledio Jahaj Technische Universitaet Graz, Lorenz Klampfl Graz University of Technology, Austria
Pre-print Media Attached
12:45
30m
Long-paper
Multimodal Surprise Adequacy Analysis of Inputs for Natural Language Processing DNN Models
AST 2021
Seah Kim KAIST, Shin Yoo Korea Advanced Institute of Science and Technology
Pre-print Media Attached
15:00 - 16:15
Testing for Specific Domains - 2AST 2021 at AST Room
Chair(s): Alejandra Garrido LIFIA, University of La Plata & CONICET, Argentina
15:00
15m
Short-paper
Automated User Experience Testing through Multi-Dimensional Performance Impact Analysis
AST 2021
Chidera Biringa University of Massachusetts, Gokhan Kul University of Massachusetts Dartmouth
Pre-print Media Attached
15:15
30m
Long-paper
A Survey of Video Game Testing
AST 2021
Cristiano Politowski Concordia University, Fabio Petrillo Université du Québec à Chicoutimi, Canada, Yann-Gaël Guéhéneuc Concordia University and Polytechnique Montréal
Pre-print Media Attached
15:45
30m
Long-paper
Test suites as a source of training data for static analysis alert classifiers
AST 2021
Pre-print Media Attached
16:30 - 18:15
Industrial case studies & Doctoral StudentsAST 2021 at AST Room
Chair(s): Breno Miranda Federal University of Pernambuco
16:30
15m
Short-paper
Continuous Testing Improvement Model
AST 2021
Maximiliano Agustin Mascheroni Universidad Nacional de La Plata, Emanuel Agustin Irrazábal Universidad Nacional del Nordeste, Gustavo Rossi Universidad Nacional de La Plata, LIFIA-Fac. Informatica, La Plata, Argentina
Pre-print Media Attached
16:45
30m
Long-paper
Model-based Automation of Test Scripts Generation Across Product Variants: a Railway Perspective
AST 2021
Alessio Bucaioni Mälardalen University, Fabio Di Silvestro Bombardier Railway Transportation, Inderjeet Singh Bombardier Railway Transportation, Mehrdad Saadatmand RISE Research Institutes of Sweden, Henry Muccini University of L'Aquila, Italy, Thorvaldur Jochumsson Bombardier Railway Transportation
Pre-print Media Attached
17:15
30m
Long-paper
Using Machine Learning to Build Test Oracles: an Industrial Case Study on Elevators Dispatching Algorithms
AST 2021
Aitor Arrieta University of Mondragon, Jon Ayerdi Mondragon Unibertsitatea, Miren Illarramendi Mondragon University, Aitor Agirre IKERLAN-IK4, Goiuria Sagardui University of Mondragon , Maite Arratibel Orona
Pre-print Media Attached
17:45
30m
Long-paper
Automatically Assessing and Extending Code Coverage for NPM Packages
AST 2021
Haiyang Sun Università della Svizzera italiana, Andrea Rosà University of Lugano, Switzerland, Daniele Bonetta Oracle Labs, Walter Binder University of Lugano, Switzerland
Media Attached
18:15 - 18:30
Conference closingAST 2021 at AST Room
Chair(s): María José Escalona University of Seville

YT Video: https://www.youtube.com/watch?v=WlZz47rRylY

Accepted Papers

Title
A framework for the automation of testing computer vision systems
AST 2021
Pre-print Media Attached
An Evolutionary Approach to Adapt Tests Across Mobile Apps
AST 2021
Pre-print Media Attached
A Survey of Video Game Testing
AST 2021
Pre-print Media Attached
Automated Performance Testing Based on Active Deep Learning
AST 2021
Pre-print Media Attached
Automated User Experience Testing through Multi-Dimensional Performance Impact Analysis
AST 2021
Pre-print Media Attached
Automatically Assessing and Extending Code Coverage for NPM Packages
AST 2021
Media Attached
Automatic Property-based Testing of GraphQL APIs
AST 2021
Pre-print Media Attached
Continuous Testing Improvement Model
AST 2021
Pre-print Media Attached
Extending Hierarchical Delta Debugging with Hoisting
AST 2021
Pre-print Media Attached
Extreme mutation testing in practice: An industrial case study
AST 2021
Pre-print Media Attached
Model-based Automation of Test Scripts Generation Across Product Variants: a Railway Perspective
AST 2021
Pre-print Media Attached
Multimodal Surprise Adequacy Analysis of Inputs for Natural Language Processing DNN Models
AST 2021
Pre-print Media Attached
SuMo: A Mutation Strategy for Solidity Smart Contracts
AST 2021
Pre-print Media Attached
Test suites as a source of training data for static analysis alert classifiers
AST 2021
Pre-print Media Attached
Towards exhaustive branch coverage with PathCrawler
AST 2021
Pre-print Media Attached
Using Machine Learning to Build Test Oracles: an Industrial Case Study on Elevators Dispatching Algorithms
AST 2021
Pre-print Media Attached

Call for Papers

The increasing complexity, pervasiveness and inter-connection of software systems on the one hand, and the ever-shrinking development cycles and time-to-market on the other, make the automation of software test (AST) an urgent requirement today more than ever. Despite significant achievements both in theory and practice, AST remains a challenging research area.

Conference Theme

The AST 2021 conference theme is “Automatic Software Testing from the Trenches”.

Software testing automation is a discipline that has produced noteworthy research in the last decade. The search for solutions to automatically test any concept of software is critical, mainly to fulfill the ever-increasing demands from the industry. Many times, however, industry needs differ from the research agenda, as companies need to prioritize reducing cost and time-to-market. Moreover, practitioners may have a hard time choosing a particular testing method or technology, since the real challenges that influence the decision are usually hidden.

AST 2021 continues with a long record of workshops and recently, conferences, oriented to look for solutions to automated testing needs. In this edition, our goal is to go one step further in creating a bridge between theory and practice. AST 2021 will offer a forum of discussion about solutions, experiences or conclusions obtained from the trenches.

We invite contributions that focus on: i) lessons learned about experiments of automatic testing in practice; ii) experiences of the adoption of testing tools, methods and techniques; iii) best practices to follow in testing and their measurable consequences; and iv) theorical approaches that are applicable to the industry in the context of AST.

Topics of Interest

Submissions on the AST 2021 theme are especially encouraged, but papers on other topics relevant to the automation of software test are also welcome.

Topics of interest include, but are not limited to the following:

  • Test automation of large, complex system
  • Metrics for testing - test efficiency, test coverage
  • Tools for model-based V&V
  • Test-driven development
  • Standardization of test tools
  • Test coverage metrics and criteria
  • Product line testing
  • Formal methods and theories for testing and test automation
  • Test case generation based on formal and semi-formal models
  • Testing with software usage models
  • Testing of reactive and object-oriented systems
  • Software simulation by models, forecasts of behavior and properties
  • Application of model checking in testing
  • Tools for security specification, models, protocols, testing and evaluation
  • Theoretical foundations of test automation
  • Models as test oracles; test validation with models
  • Testing anomaly detectors
  • Testing cyber physical systems
  • Automated usability and user experience testing

We are interested in the following aspects related to AST:

  1. Problem identification. Analysis and specification of requirements for AST, and elicitation of problems that hamper wider adoption of AST
  2. Methodology. Novel methods and approaches for AST in the context of up-to-date software development methodologies
  3. Technology. Automation of various test techniques and methods for test-related activities, as well as for testing various types of software
  4. Tools and Environments. Issues and solutions in the development, operation, maintenance and evolution of tools and environments for AST, and their integration with other types of tools and runtime support platforms
  5. Empirical studies, and Experience reports, and Industrial Contributions. Real experiences in using automated testing techniques, methods and tools in industry
  6. Visions of the future. Foresight and thought-provoking ideas for AST that can inspire new powerful research trends.

Submission

Three types of submissions are invited:

  • Regular Papers (up to 10 pages)
    • Research Paper
    • Industrial Case Study
  • Short Papers (up to 4 pages)
    • Research Paper
    • Industrial Case Study
    • Doctoral Student Research
  • Industrial Abstracts (up to 2 pages)

Regular papers include both Research papers that present research in the area of software test automation, and Industrial Case Studies that report on practical applications of test automation.

Regular papers must not exceed 10 pages for all materials (including the main text, appendices, figures, tables, and references).

Short papers also include both Research papers and Industrial Case Studies.

Short papers must not exceed 4 pages for all materials.

As short papers, doctoral students working on software testing are encouraged to submit their work. AST will have an independent session to bring doctoral students working on software testing, with experts assigned to each paper together, to discuss their research in a constructive and international atmosphere, and to prepare for defense exam. The first author in a submission must be the doctoral student and the second author the advisor. Authors of selected submissions will be invited to make a brief presentation followed by a constructive discussion in a session dedicated to doctoral students.

Industrial abstract talks are specifically conceived to promote industrial participation: We require the first author of such papers to come from industry. Authors of accepted papers get invited to give a talk with same time length and within same sessions as regular papers. Industrial abstracts must not exceed 2 pages for all materials.

The submission website is: https://easychair.org/conferences/?conf=ast2021

All submissions must be in PDF. The page limit is strict, and it will not be possible to purchase additional pages at any point in the process (including after the paper is accepted).

Submissions must conform to the IEEE formatting instructions IEEE Conference Proceedings Formatting Guidelines (title in 24pt font and full text in 10pt type, LaTeX users must use \documentclass[10pt,conference]{IEEEtran} without including the compsoc or compsocconf options).

Submissions must be unpublished original work and should not be under review or submitted elsewhere while being under consideration. AST 2021 will follow a double-blind review process, so author names, affiliations and funding sources should be removed from the PDF, as well as any references that may reveal the identity of the authors in the text.

By submitting to AST, authors acknowledge that they are aware of and agree to be bound by the ACM Policy and Procedures on Plagiarism and the IEEE Plagiarism FAQ. The authors also acknowledge that they conform to the authorship policy of the ACM, and the authorship policy of the IEEE.

The accepted regular and short papers, case studies, and industrial abstracts will be published in the ICSE 2021 Co-located Event Proceedings and included in the IEEE and ACM Digital Libraries. Authors of accepted papers are required to register and present their accepted paper at the conference in order for the paper to be included in the proceedings and the Digital Libraries.

The official publication date is the date the proceedings are made available in the ACM or IEEE Digital Libraries.

This date may be up to two weeks prior to the first day of ICSE 2021. The official publication date affects the deadline for any patent filings related to published work.