Write a Blog >>
ICSE 2022
Sun 8 - Fri 27 May 2022
Wed 11 May 2022 12:10 - 12:15 at ICSE room 4-even hours - Software Testing 12 Chair(s): Frank Tip
Thu 12 May 2022 03:15 - 03:20 at ICSE room 5-odd hours - Software Testing 4 Chair(s): Aldeida Aleti
Thu 26 May 2022 11:20 - 11:25 at Room 301+302 - Papers 15: Software Testing 2 Chair(s): Rohan Padhye

Flaky tests are tests that pass or fail nondeterministically on the same version of code. These tests can mislead developers concerning the quality of their code changes during regression testing. A common kind of flaky test are order-dependent tests whose pass/- fail outcomes depend on the order which the tests are run, and test order is not guaranteed. Such tests have different outcomes because they share state with other tests that pollute that shared state when run beforehand. Prior work has proposed repairing order-dependent tests by searching for tests, known as “cleaners” that can make the order-dependent tests to pass when run right before, effectively resetting the shared state. While the code within these cleaners represent a patch that can repair the order-dependent test, this repair technique depends heavily on developers having already written such test code.

We propose ODRepair, a technique to repair order-dependent tests even without the presence of existing cleaners. The idea is to first determine what exactly is the polluted shared state that results in the order-dependent test to fail and then generate code that can modify and reset the shared state so the order-dependent test can pass. We focus on shared state through internal heap memory, in particular shared state reachable from static fields. Once we know which static field leads to the pollution, we search for reset-methods in the codebase that can potentially access and modify state reachable from that static field. We then apply a test generation tool to generate method sequences, including any necessary arguments as inputs, that target these potential reset-methods. Our evaluation on 327 order-dependent tests from a publicly available dataset showed that we can automatically identify the polluted static field for 181 tests, and of those we can generate patches for 141 of them. Compared against state-of-the-art iFixFlakies, we can generate patches for 24 tests that iFixFlakies cannot.

Wed 11 May

Displayed time zone: Eastern Time (US & Canada) change

12:00 - 13:00
12:00
5m
Talk
Mutation Analysis for Cyber-Physical Systems: Scalable Solutions and Results in the Space Domain
Journal-First Papers
Oscar Cornejo SnT Centre, University of Luxembourg, Fabrizio Pastore University of Luxembourg, Lionel Briand University of Luxembourg; University of Ottawa
Link to publication DOI Pre-print Media Attached
12:05
5m
Talk
Using Natural Language Processing Techniques to Improve Manual Test Case Descriptions
SEIP - Software Engineering in Practice
Markos Viggiato University of Alberta, Dale Paas Prodigy Education, Chris Buzon Prodigy Education, Cor-Paul Bezemer University of Alberta
Pre-print Media Attached
12:10
5m
Talk
Repairing Order-Dependent Flaky Tests via Test Generation
Technical Track
Chengpeng Li University of Texas at Austin, Chenguang Zhu University of Texas at Austin, Wenxi Wang University of Texas at Austin, August Shi University of Texas at Austin
Media Attached
12:15
5m
Talk
One Fuzzing Strategy to Rule Them All
Technical Track
Mingyuan Wu Southern University of Science and Technology, Ling Jiang Southern University of Science and Technology, Jiahong Xiang Southern University of Science and Technology, Yanwei Huang Zhejiang University, Heming Cui University of Hong Kong, Lingming Zhang University of Illinois at Urbana-Champaign, Yuqun Zhang Southern University of Science and Technology
Pre-print Media Attached
12:20
5m
Talk
BeDivFuzz: Integrating Behavioral Diversity into Generator-based Fuzzing
Technical Track
Hoang Lam Nguyen Humboldt-Universität zu Berlin, Lars Grunske Humboldt-Universität zu Berlin
Pre-print Media Attached
12:25
5m
Talk
Preempting Flaky Tests via Non-Idempotent-Outcome Tests
Technical Track
Anjiang Wei Stanford University, Pu Yi Peking University, Zhengxi Li University of Illinois Urbana-Champaign, Tao Xie Peking University, Darko Marinov University of Illinois at Urbana-Champaign, Wing Lam University of Illinois at Urbana-Champaign
Pre-print Media Attached

Thu 12 May

Displayed time zone: Eastern Time (US & Canada) change

03:00 - 04:00
03:00
5m
Talk
Mutation Analysis for Cyber-Physical Systems: Scalable Solutions and Results in the Space Domain
Journal-First Papers
Oscar Cornejo SnT Centre, University of Luxembourg, Fabrizio Pastore University of Luxembourg, Lionel Briand University of Luxembourg; University of Ottawa
Link to publication DOI Pre-print Media Attached
03:05
5m
Talk
Automated Visual Testing for Mobile Apps in an Industrial Setting
SEIP - Software Engineering in Practice
Dezhi Ran Peking University, Zongyang Li Peking University, Chenxu Liu Peking University, Wenyu Wang University of Illinois Urbana-Champaign, Weizhi Meng Alibaba Group, Xionglin Wu Alibaba Group, Hui Jin Alibaba Group, Jing Cui Alibaba Group, Xing Tang Alibaba Group, Tao Xie Peking University
DOI Pre-print
03:10
5m
Talk
FADATest: Fast and Adaptive Performance Regression Testing of Dynamic Binary Translation Systems
Technical Track
Jin Wu Harbin Institute of Technology, Jian Dong Harbin Institute Of Technology, Ruili Fang , Wen Zhang University of Georgia, Wenwen Wang University of Georgia, Decheng Zuo Harbin Institute of Technology
Pre-print Media Attached
03:15
5m
Talk
Repairing Order-Dependent Flaky Tests via Test Generation
Technical Track
Chengpeng Li University of Texas at Austin, Chenguang Zhu University of Texas at Austin, Wenxi Wang University of Texas at Austin, August Shi University of Texas at Austin
Media Attached
03:20
5m
Talk
Controlled Concurrency Testing via Periodical Scheduling
Technical Track
Cheng Wen Shenzhen University, Mengda He SCEDT, Teesside University, Bohao Wu CSSE, Shenzhen University, Zhiwu Xu Shenzhen University, Shengchao Qin Huawei Hong Kong Research Centre, China
DOI Pre-print Media Attached
03:25
5m
Talk
BeDivFuzz: Integrating Behavioral Diversity into Generator-based Fuzzing
Technical Track
Hoang Lam Nguyen Humboldt-Universität zu Berlin, Lars Grunske Humboldt-Universität zu Berlin
Pre-print Media Attached

Thu 26 May

Displayed time zone: Eastern Time (US & Canada) change

11:00 - 12:30
Papers 15: Software Testing 2Technical Track / SEIP - Software Engineering in Practice at Room 301+302
Chair(s): Rohan Padhye Carnegie Mellon University
11:00
5m
Talk
CONFETTI: Amplifying Concolic Guidance for Fuzzers
Technical Track
James Kukucka George Mason University, Luís Pina University of Illinois at Chicago, Paul Ammann George Mason University, USA, Jonathan Bell Northeastern University
Pre-print Media Attached
11:05
5m
Talk
Surveying the Developer Experience of Flaky Tests
SEIP - Software Engineering in Practice
Owain Parry The University of Sheffield, Gregory Kapfhammer Allegheny College, Michael Hilton Carnegie Mellon University, USA, Phil McMinn University of Sheffield
Pre-print Media Attached
11:10
5m
Talk
Natural Attack for Pre-trained Models of Code
Technical Track
Zhou Yang Singapore Management University, Jieke SHI Singapore Management University, Junda He Singapore Management University, David Lo Singapore Management University
DOI Pre-print Media Attached
11:15
5m
Talk
FADATest: Fast and Adaptive Performance Regression Testing of Dynamic Binary Translation Systems
Technical Track
Jin Wu Harbin Institute of Technology, Jian Dong Harbin Institute Of Technology, Ruili Fang , Wen Zhang University of Georgia, Wenwen Wang University of Georgia, Decheng Zuo Harbin Institute of Technology
Pre-print Media Attached
11:20
5m
Talk
Repairing Order-Dependent Flaky Tests via Test Generation
Technical Track
Chengpeng Li University of Texas at Austin, Chenguang Zhu University of Texas at Austin, Wenxi Wang University of Texas at Austin, August Shi University of Texas at Austin
Media Attached
11:25
5m
Talk
BeDivFuzz: Integrating Behavioral Diversity into Generator-based Fuzzing
Technical Track
Hoang Lam Nguyen Humboldt-Universität zu Berlin, Lars Grunske Humboldt-Universität zu Berlin
Pre-print Media Attached
11:30
5m
Talk
Nessie: Automatically Testing JavaScript APIs with Asynchronous Callbacks
Technical Track
Ellen Arteca Northeastern University, Sebastian Harner University of Stuttgart, Michael Pradel University of Stuttgart, Frank Tip Northeastern University
Pre-print Media Attached
11:35
5m
Talk
EAGLE: Creating Equivalent Graphs to Test Deep Learning Libraries
Technical Track
Jiannan Wang Purdue University, Thibaud Lutellier University of Waterloo, Shangshu Qian Purdue University, Viet Hung Pham University of Waterloo, Lin Tan Purdue University
Pre-print Media Attached
Hide past events

Information for Participants
Wed 11 May 2022 12:00 - 13:00 at ICSE room 4-even hours - Software Testing 12 Chair(s): Frank Tip
Info for room ICSE room 4-even hours:

Click here to go to the room on Midspace

Thu 12 May 2022 03:00 - 04:00 at ICSE room 5-odd hours - Software Testing 4 Chair(s): Aldeida Aleti
Info for room ICSE room 5-odd hours:

Click here to go to the room on Midspace