AST 2023
Mon 15 - Tue 16 May 2023 Melbourne, Australia
co-located with ICSE 2023
Tue 16 May 2023 14:07 - 14:30 at Meeting Room 107 - Test Prioritization

Diversity-based techniques (DBT) have been cost-effective by prioritizing the most dissimilar test cases to detect faults at earlier stages of test execution. Diversity is measured on test specifications to convey how different test cases are from one another. However, there is little research on the trade-off of diversity measures based on different types of text-based specification (lexicographical or semantics). Particularly because the text content in test scripts vary widely from unit (e.g., code) to system-level (e.g., natural language). This paper compares and evaluates the cost-effectiveness in coverage and failures of different text-based diversity measures for different levels of tests. We perform an experiment on the test suites of 7 open source projects on the unit level, and 2 industry projects on the integration and system level. Our results show that test suites prioritised using semantic-based diversity measures causes a small improvement in requirements coverage, as opposed to lexical diversity that performed worse than random for system-level artefacts. In contrast, using lexical-based measures such as Jaccard or Levenshtein to prioritise code artefacts yield better failure coverage across all levels of tests. We summarise our findings into a list of recommendations for using semantic or lexical diversity on different levels of testing.

Tue 16 May

Displayed time zone: Hobart change

13:45 - 15:15
Test PrioritizationAST 2023 at Meeting Room 107
13:45
22m
Talk
Orchestration Strategies for Regression Test Suites
AST 2023
Renan Greca Gran Sasso Science Institute, ISTI-CNR, Breno Miranda Federal University of Pernambuco, Antonia Bertolino National Research Council, Italy
Pre-print
14:07
22m
Talk
Evaluating the Trade-offs of Text-based Diversity in Test Prioritization
AST 2023
Ranim Khojah Chalmers | University of Gothenburg, Chi Hong Chao Chalmers | University of Gothenburg, Francisco Gomes de Oliveira Neto Chalmers University of Technology, Sweden / University of Gothenburg, Sweden
14:30
22m
Talk
MuTCR: Test Case Recommendation via Multi-Level Signature Matching
AST 2023
Weisong Sun Nanjing University, Weidong Qian China Ship Scientific Research Center, Bin Luo Nanjing University, Zhenyu Chen Nanjing University
14:52
22m
Talk
Test Case Prioritization using Transfer Learning in Continuous Integration Environments
AST 2023
Rezwana Mamata Ontario Tech University, Akramul Azim Ontario Tech University, Ramiro Liscano Ontario Tech University, Kevin Smith International Business Machines Corporation (IBM), Yee-Kang Chang International Business Machines Corporation (IBM), Gkerta Seferi International Business Machines Corporation (IBM), Qasim Tauseef International Business Machines Corporation (IBM)