Using Natural Language Processing Techniques to Improve Manual Test Case Descriptions
Thu 12 May 2022 20:00 - 20:05 at ICSE room 5-even hours - Software Testing 9 Chair(s): Jonathan Sillito
Despite the recent advancements in test automation, software testing often remains a manual, and costly, activity in many industries. Manual test cases, often described only in natural language, consist of one or more test steps, which are instructions that must be performed to achieve the testing objective. Having different employees specifying test cases might results in redundant, unclear, or incomplete test cases. Manually reviewing and validating newly-specified test cases is time-consuming and becomes impractical in a scenario with a large test suite. Therefore, in this paper, we propose an automated framework to automatically analyze test cases that are specified in natural language and provide actionable recommendations on how to improve the test cases. Our framework consists of configurable components and modules for analysis, which are capable of recommending improvements to the following: (1) the terminology of a new test case through language modeling, (2) potentially missing test steps for a new test case through frequent itemset and association rule mining, and (3) recommendation of similar test cases that already exist in the test suite through text embedding and clustering. We thoroughly evaluated the three modules on data from our industry partner. Our framework can provide actionable recommendations, which is an important challenge given the widespread occurrence of test cases that are described only in natural language in the software industry (in particular, the game industry).
Wed 11 MayDisplayed time zone: Eastern Time (US & Canada) change
12:00 - 13:00 | Software Testing 12Technical Track / Journal-First Papers / SEIP - Software Engineering in Practice at ICSE room 4-even hours Chair(s): Frank Tip Northeastern University | ||
12:00 5mTalk | Mutation Analysis for Cyber-Physical Systems: Scalable Solutions and Results in the Space Domain Journal-First Papers Oscar Cornejo SnT Centre, University of Luxembourg, Fabrizio Pastore University of Luxembourg, Lionel Briand University of Luxembourg; University of Ottawa Link to publication DOI Pre-print Media Attached | ||
12:05 5mTalk | Using Natural Language Processing Techniques to Improve Manual Test Case Descriptions SEIP - Software Engineering in Practice Markos Viggiato University of Alberta, Dale Paas Prodigy Education, Chris Buzon Prodigy Education, Cor-Paul Bezemer University of Alberta Pre-print Media Attached | ||
12:10 5mTalk | Repairing Order-Dependent Flaky Tests via Test Generation Technical Track Chengpeng Li University of Texas at Austin, Chenguang Zhu University of Texas at Austin, Wenxi Wang University of Texas at Austin, August Shi University of Texas at Austin Link to publication DOI Media Attached | ||
12:15 5mTalk | One Fuzzing Strategy to Rule Them All Technical Track Mingyuan Wu Southern University of Science and Technology, Ling Jiang Southern University of Science and Technology, Jiahong Xiang Southern University of Science and Technology, Yanwei Huang Zhejiang University, Heming Cui University of Hong Kong, Lingming Zhang University of Illinois at Urbana-Champaign, Yuqun Zhang Southern University of Science and Technology Pre-print Media Attached | ||
12:20 5mTalk | BeDivFuzz: Integrating Behavioral Diversity into Generator-based Fuzzing Technical Track Pre-print Media Attached | ||
12:25 5mTalk | Preempting Flaky Tests via Non-Idempotent-Outcome Tests Technical Track Anjiang Wei Stanford University, Pu Yi Peking University, Zhengxi Li University of Illinois Urbana-Champaign, Tao Xie Peking University, Darko Marinov University of Illinois at Urbana-Champaign, Wing Lam University of Illinois at Urbana-Champaign Pre-print Media Attached | ||
Thu 12 MayDisplayed time zone: Eastern Time (US & Canada) change
20:00 - 21:00 | Software Testing 9SEIP - Software Engineering in Practice / Technical Track at ICSE room 5-even hours Chair(s): Jonathan Sillito Brigham Young University | ||
20:00 5mTalk | Using Natural Language Processing Techniques to Improve Manual Test Case Descriptions SEIP - Software Engineering in Practice Markos Viggiato University of Alberta, Dale Paas Prodigy Education, Chris Buzon Prodigy Education, Cor-Paul Bezemer University of Alberta Pre-print Media Attached | ||
20:05 5mTalk | Combinatorial Testing of RESTful APIs Technical Track Huayao Wu Nanjing University, Lixin Xu Nanjing University, Xintao Niu Nanjing University, Changhai Nie Nanjing University Pre-print Media Attached | ||
20:10 5mTalk | History-Driven Test Program Synthesis for JVM Testing Technical Track Yingquan Zhao Tianjin University, Zan Wang Tianjin University, China, Junjie Chen Tianjin University, Mengdi Liu College of Intelligence and Computing, Tianjin University, Mingyuan Wu Southern University of Science and Technology, Yuqun Zhang Southern University of Science and Technology, Lingming Zhang University of Illinois at Urbana-Champaign Pre-print Media Attached | ||
20:15 5mTalk | FADATest: Fast and Adaptive Performance Regression Testing of Dynamic Binary Translation Systems Technical Track Jin Wu Harbin Institute of Technology, Jian Dong Harbin Institute Of Technology, Ruili Fang University of Georgia, Wen Zhang University of Georgia, Wenwen Wang University of Georgia, Decheng Zuo Harbin Institute of Technology Pre-print Media Attached | ||
20:20 5mTalk | Preempting Flaky Tests via Non-Idempotent-Outcome Tests Technical Track Anjiang Wei Stanford University, Pu Yi Peking University, Zhengxi Li University of Illinois Urbana-Champaign, Tao Xie Peking University, Darko Marinov University of Illinois at Urbana-Champaign, Wing Lam University of Illinois at Urbana-Champaign Pre-print Media Attached | ||
20:25 5mTalk | One Fuzzing Strategy to Rule Them All Technical Track Mingyuan Wu Southern University of Science and Technology, Ling Jiang Southern University of Science and Technology, Jiahong Xiang Southern University of Science and Technology, Yanwei Huang Zhejiang University, Heming Cui University of Hong Kong, Lingming Zhang University of Illinois at Urbana-Champaign, Yuqun Zhang Southern University of Science and Technology Pre-print Media Attached | ||