Thu 12 May 2022 04:20 - 04:25 at ICSE room 3-even hours - Testing and Analysis Chair(s): Jie Zhang
Thu 12 May 2022 11:25 - 11:30 at ICSE room 4-odd hours - Software Testing 13 Chair(s): Peter C. Rigby
Thu 12 May 2022 11:25 - 11:30 at ICSE room 4-odd hours - Software Testing 13 Chair(s): Peter C. Rigby
Given a program where none of our fuzzers finds any bugs, how do we know which fuzzer is better? In practice, we often look to code coverage as a proxy measure of fuzzer effectiveness and consider the fuzzer which achieves more coverage as the better one.
Indeed, evaluating 10 fuzzers for 23 hours on 24 programs, we find that a fuzzer that covers more code also finds more bugs. There is a very strong correlation between the coverage achieved and the number of bugs found by a fuzzer. Hence, it might seem reasonable to compare fuzzers in terms of coverage achieved, and from that derive empirical claims about a fuzzer’s superiority at finding bugs.
Curiously enough, however, we find no strong agreement on which fuzzer is superior if we compared multiple fuzzers in terms of coverage achieved instead of the number of bugs found. The fuzzer best at achieving coverage, may not be best at finding bugs.
Thu 12 MayDisplayed time zone: Eastern Time (US & Canada) change
Thu 12 May
Displayed time zone: Eastern Time (US & Canada) change
04:00 - 05:00 | |||
04:00 5mTalk | Static Stack-Preserving Intra-Procedural Slicing of WebAssembly BinariesBest Artifact Award Technical Track Quentin Stiévenart Vrije Universiteit Brussel, David Binkley Loyola University Maryland, Coen De Roover Vrije Universiteit Brussel DOI Pre-print Media Attached | ||
04:05 5mTalk | Automated Assertion Generation via Information Retrieval and Its Integration with Deep Learning Technical Track Hao Yu Peking University, Yiling Lou Purdue University, Ke Sun , Dezhi Ran Peking University, Tao Xie Peking University, Dan Hao Peking University, Ying Li School of Software and Microelectronics, Peking University, Beijing, China, Ge Li Peking University, Qianxiang Wang Huawei Technologies Co. Ltd DOI Pre-print Media Attached | ||
04:10 5mTalk | Combinatorial Testing of RESTful APIs Technical Track Huayao Wu Nanjing University, Lixin Xu Nanjing University, Xintao Niu Nanjing University, Changhai Nie Nanjing University Pre-print Media Attached | ||
04:15 5mTalk | History-Driven Test Program Synthesis for JVM Testing Technical Track Yingquan Zhao Tianjin University, Zan Wang Tianjin University, China, Junjie Chen Tianjin University, Mengdi Liu College of Intelligence and Computing, Tianjin University, Mingyuan Wu Southern University of Science and Technology, Yuqun Zhang Southern University of Science and Technology, Lingming Zhang University of Illinois at Urbana-Champaign Pre-print Media Attached | ||
04:20 5mTalk | On the Reliability of Coverage-Based Fuzzer Benchmarking Technical Track Marcel Böhme MPI-SP, Germany and Monash University, Australia, Laszlo Szekeres Google, Jonathan Metzman Google DOI Pre-print Media Attached | ||
11:00 - 12:00 | Software Testing 13Journal-First Papers / Technical Track at ICSE room 4-odd hours Chair(s): Peter C. Rigby | ||
11:00 5mTalk | Software Batch Testing to Save Build Test Resources and to Reduce Feedback Time Journal-First Papers Mohammad Javad Beheshtian Concordia University, Amir Bavand Concordia University, Peter Rigby Concordia University, Montreal, Canada Link to publication DOI Media Attached | ||
11:05 5mTalk | A Family of Experiments on Test-Driven Development Journal-First Papers Adrian Santos Parrilla University of Oulu, Sira Vegas Universidad Politecnica de Madrid, Oscar Dieste Universidad Politécnica de Madrid, Fernando Uyaguari ETAPA Telecommunications Company, Ayse Tosun Istanbul Technical University, Davide Fucci Blekinge Institute of Technology, Burak Turhan University of Oulu, Giuseppe Scanniello University of Basilicata, Simone Romano University of Bari, Itir Karac University of Oulu, Marco Kuhrmann Reutlingen University, Vladimir Mandić Faculty of Technical Sciences, University of Novi Sad, Robert Ramač Faculty of Technical Sciences, University of Novi Sad, Dietmar Pfahl University of Tartu, Christian Engblom Ericsson, Jarno Kyykka Ericsson, Kerli Rungi Testlio, Carolina Palomeque ETAPA Telecommunications Company, Jaroslav Spisak PAF, Markku Oivo University of Oulu, Natalia Juristo Universidad Politecnica de Madrid Link to publication DOI Pre-print Media Attached | ||
11:10 5mTalk | Prioritizing Mutants to Guide Mutation Testing Technical Track Samuel Kaufman University of Washington, Ryan Featherman University of Washington, Justin Alvin University of Massachusetts Amherst, Bob Kurtz George Mason University, USA, Paul Ammann George Mason University, USA, René Just University of Washington DOI Pre-print Media Attached | ||
11:15 5mTalk | Automated Testing of Software that Uses Machine Learning APIs Technical Track Chengcheng Wan The University of Chicago, Shicheng Liu University of Chicago, Sophie Xie University of California, Berkeley, Yifan Liu University of Chicago, Henry Hoffmann University of Chicago, Michael Maire University of Chicago, Shan Lu University of Chicago Pre-print Media Attached | ||
11:20 5mTalk | CONFETTI: Amplifying Concolic Guidance for Fuzzers Technical Track James Kukucka George Mason University, Luís Pina University of Illinois at Chicago, Paul Ammann George Mason University, USA, Jonathan Bell Northeastern University Pre-print Media Attached | ||
11:25 5mTalk | On the Reliability of Coverage-Based Fuzzer Benchmarking Technical Track Marcel Böhme MPI-SP, Germany and Monash University, Australia, Laszlo Szekeres Google, Jonathan Metzman Google DOI Pre-print Media Attached | ||
Information for Participants
Thu 12 May 2022 04:00 - 05:00 at ICSE room 3-even hours - Testing and Analysis Chair(s): Jie Zhang
Info for room ICSE room 3-even hours:
Thu 12 May 2022 11:00 - 12:00 at ICSE room 4-odd hours - Software Testing 13 Chair(s): Peter C. Rigby
Info for room ICSE room 4-odd hours: