Write a Blog >>
ICSE 2022
Sun 8 - Fri 27 May 2022
Thu 12 May 2022 04:20 - 04:25 at ICSE room 3-even hours - Testing and Analysis Chair(s): Jie Zhang
Thu 12 May 2022 11:25 - 11:30 at ICSE room 4-odd hours - Software Testing 13 Chair(s): Peter C. Rigby

Given a program where none of our fuzzers finds any bugs, how do we know which fuzzer is better? In practice, we often look to code coverage as a proxy measure of fuzzer effectiveness and consider the fuzzer which achieves more coverage as the better one.

Indeed, evaluating 10 fuzzers for 23 hours on 24 programs, we find that a fuzzer that covers more code also finds more bugs. There is a very strong correlation between the coverage achieved and the number of bugs found by a fuzzer. Hence, it might seem reasonable to compare fuzzers in terms of coverage achieved, and from that derive empirical claims about a fuzzer’s superiority at finding bugs.

Curiously enough, however, we find no strong agreement on which fuzzer is superior if we compared multiple fuzzers in terms of coverage achieved instead of the number of bugs found. The fuzzer best at achieving coverage, may not be best at finding bugs.

Thu 12 May

Displayed time zone: Eastern Time (US & Canada) change

04:00 - 05:00
Testing and AnalysisTechnical Track at ICSE room 3-even hours
Chair(s): Jie Zhang UCL
04:00
5m
Talk
Static Stack-Preserving Intra-Procedural Slicing of WebAssembly BinariesBest Artifact Award
Technical Track
Quentin Stiévenart Vrije Universiteit Brussel, David Binkley Loyola University Maryland, Coen De Roover Vrije Universiteit Brussel
DOI Pre-print Media Attached
04:05
5m
Talk
Automated Assertion Generation via Information Retrieval and Its Integration with Deep Learning
Technical Track
Hao Yu Peking University, Yiling Lou Purdue University, Ke Sun , Dezhi Ran Peking University, Tao Xie Peking University, Dan Hao Peking University, Ying Li School of Software and Microelectronics, Peking University, Beijing, China, Ge Li Peking University, Qianxiang Wang Huawei Technologies Co. Ltd
DOI Pre-print Media Attached
04:10
5m
Talk
Combinatorial Testing of RESTful APIs
Technical Track
Huayao Wu Nanjing University, Lixin Xu Nanjing University, Xintao Niu Nanjing University, Changhai Nie Nanjing University
Pre-print Media Attached
04:15
5m
Talk
History-Driven Test Program Synthesis for JVM Testing
Technical Track
Yingquan Zhao College of Intelligence and Computing, Tianjin University, Zan Wang Tianjin University, China, Junjie Chen Tianjin University, Mengdi Liu College of Intelligence and Computing, Tianjin University, Mingyuan Wu Southern University of Science and Technology, Yuqun Zhang Southern University of Science and Technology, Lingming Zhang University of Illinois at Urbana-Champaign
Pre-print Media Attached
04:20
5m
Talk
On the Reliability of Coverage-Based Fuzzer Benchmarking
Technical Track
Marcel Böhme MPI-SP, Germany and Monash University, Australia, Laszlo Szekeres Google, Jonathan Metzman Google
DOI Pre-print Media Attached
11:00 - 12:00
11:00
5m
Talk
Software Batch Testing to Save Build Test Resources and to Reduce Feedback Time
Journal-First Papers
Mohammad Javad Beheshtian Concordia University, Amir Bavand Concordia University, Peter Rigby Concordia University, Montreal, Canada
Link to publication DOI Media Attached
11:05
5m
Talk
A Family of Experiments on Test-Driven Development
Journal-First Papers
Adrian Santos Parrilla University of Oulu, Sira Vegas Universidad Politecnica de Madrid, Oscar Dieste Universidad Politécnica de Madrid, Fernando Uyaguari ETAPA Telecommunications Company, Ayse Tosun Istanbul Technical University, Davide Fucci Blekinge Institute of Technology, Burak Turhan University of Oulu, Giuseppe Scanniello University of Basilicata, Simone Romano University of Bari, Itir Karac University of Oulu, Marco Kuhrmann Reutlingen University, Vladimir Mandić Faculty of Technical Sciences, University of Novi Sad, Robert Ramač Faculty of Technical Sciences, University of Novi Sad, Dietmar Pfahl University of Tartu, Christian Engblom Ericsson, Jarno Kyykka Ericsson, Kerli Rungi Testlio, Carolina Palomeque ETAPA Telecommunications Company, Jaroslav Spisak PAF, Markku Oivo University of Oulu, Natalia Juristo Universidad Politecnica de Madrid
Link to publication DOI Pre-print Media Attached
11:10
5m
Talk
Prioritizing Mutants to Guide Mutation Testing
Technical Track
Samuel Kaufman University of Washington, Ryan Featherman University of Washington, Justin Alvin University of Massachusetts Amherst, Bob Kurtz George Mason University, USA, Paul Ammann George Mason University, USA, René Just University of Washington
DOI Pre-print Media Attached
11:15
5m
Talk
Automated Testing of Software that Uses Machine Learning APIs
Technical Track
Chengcheng Wan The University of Chicago, Shicheng Liu University of Chicago, Sophie Xie University of California, Berkeley, Yifan Liu University of Chicago, Henry Hoffmann University of Chicago, Michael Maire University of Chicago, Shan Lu University of Chicago
Pre-print Media Attached
11:20
5m
Talk
CONFETTI: Amplifying Concolic Guidance for Fuzzers
Technical Track
James Kukucka George Mason University, Luís Pina University of Illinois at Chicago, Paul Ammann George Mason University, USA, Jonathan Bell Northeastern University
Pre-print Media Attached
11:25
5m
Talk
On the Reliability of Coverage-Based Fuzzer Benchmarking
Technical Track
Marcel Böhme MPI-SP, Germany and Monash University, Australia, Laszlo Szekeres Google, Jonathan Metzman Google
DOI Pre-print Media Attached

Information for Participants
Thu 12 May 2022 04:00 - 05:00 at ICSE room 3-even hours - Testing and Analysis Chair(s): Jie Zhang
Info for room ICSE room 3-even hours:

Click here to go to the room on Midspace

Thu 12 May 2022 11:00 - 12:00 at ICSE room 4-odd hours - Software Testing 13 Chair(s): Peter C. Rigby
Info for room ICSE room 4-odd hours:

Click here to go to the room on Midspace