DeepTest 2021
Tue 1 Jun 2021
co-located with ICSE 2021
Tue 1 Jun 2021 14:00 - 14:20 at DeepTest Room - Session 2 Chair(s): Vincenzo Riccio

Machine learning (ML) is widely deployed in safety-critical systems (e.g. self-driving cars). Failures can have disastrous consequences in these systems, and so ensuring the reliability of its operations is important. Mutation testing is a popular method for assessing the dependability of applications and tools have recently been developed for ML frameworks. However, the focus has been on improving the quality of test data. We present an open source data mutation tool, TensorFlow Data Mutator (TF-DM), which targets different kinds of data faults for any ML program written in TensorFlow 2. TF-DM supports different types of data mutators so users can study model resilience to data faults. We explain how different fault models are mapped to mutators in TF-DM, and present a detailed evaluation and resiliency analysis of 6 ML models and 3 datasets.

Tue 1 Jun

Displayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

13:00 - 15:00
Session 2deeptest2021 at DeepTest Room
Chair(s): Vincenzo Riccio USI Lugano, Switzerland
Testing Facebook's WW Simulation System, a Cyber-Cyber Digital Twin of the Facebook WWW Platform
Mark Harman Facebook, Inc., Natalija Gucevska Facebook
TF-DM: Tool for Studying ML Model Resilience to Data Faults
Niranjhana Narayanan The University of British Columbia, Karthik Pattabiraman University of British Columbia
Machine Learning Model Drift Detection Via Weak Data Slices
Orna Raz IBM Research, Samuel Ackerman IBM Corporation, Israel, Parijat Dube IBM, USA, Eitan Farchi IBM Haifa Research Lab, Marcel Zalmanovici
Live Q&A
Open Discussion & Q/A

Information for Participants
Tue 1 Jun 2021 13:00 - 15:00 at DeepTest Room - Session 2 Chair(s): Vincenzo Riccio
Info for room DeepTest Room:

Go directly to this room on Clowdr