Write a Blog >>
Fri 29 May 2020 15:20 - 15:40 at TBD1 - Machine Learning and Model

Property-based testing is a popular approach for validating the logic of a program. An effective property-based test quickly generates many diverse valid test inputs and runs them through a parameterized test driver. However, when the test driver requires strict validity constraints on the inputs, completely random input generation fails to generate enough valid inputs. Existing approaches to solving this problem rely on whitebox or greybox information collected by instrumenting the input generator and/or test driver. However, collecting such information reduces the speed at which tests can be executed.In this paper, we propose and study a blackbox approach for generating valid test inputs. We first formalize the problem of guiding random input generators towards producing a diverse set of valid inputs. This formalization highlights the role of a \emph{guide} which governs the space of choices within a random input generator. We then propose a solution based on reinforcement learning (RL), using a tabular, on-policy RL approach to guide the generator. We evaluate this approach, RLCheck, against pure random input generation as well as a state-of-the-art greybox evolutionary algorithm, on four real-world benchmarks. We find that in the same time budget, RLCheck generates an order of magnitude more diverse valid inputs than the baselines.

Fri 29 May

14:00 - 15:40: Paper Presentations - Machine Learning and Model at TBD1
icse-2020-Journal-First14:00 - 14:15
Zhe YuNORTH CAROLINA STATE UNIVERSITY, Chris TheisenMicrosoft, Laurie WilliamsNorth Carolina State University, Tim MenziesNorth Carolina State University
icse-2020-Journal-First14:15 - 14:30
Xu WangCollege of Engineering & Computer ScienceAustralian National University, Canberra, Australia, Chunyang ChenMonash University, Zhenchang XingAustralia National University
icse-2020-Journal-First14:30 - 14:45
Gema Rodríguez-PérezUniversity of Waterloo, Canada, Gregorio RoblesUniversidad Rey Juan Carlos, Alexander SerebrenikEindhoven University of Technology, Andy ZaidmanTU Delft, Daniel M. GermanUniversity of Victoria, Jesus M. Gonzalez-BarahonaUniversidad Rey Juan Carlos
icse-2020-Journal-First14:45 - 15:00
Amritanshu AgrawalWayfair, Wei FuLanding AI, Di ChenNorth Carolina State University, USA, Xipeng ShenNorth Carolina State University, Tim MenziesNorth Carolina State University
icse-2020-papers15:00 - 15:20
Cody WatsonWashington and Lee University, Michele TufanoMicrosoft, Kevin MoranCollege of William & Mary, Gabriele BavotaUniversità della Svizzera italiana, Denys PoshyvanykWilliam and Mary
icse-2020-papers15:20 - 15:40
Sameer ReddyUniversity of California, Berkeley, Caroline LemieuxUniversity of California, Berkeley, Rohan PadhyeUniversity of California, Berkeley, Koushik SenUniversity of California, Berkeley