Write a Blog >>
ICSE 2021
Sun 16 May - Sat 5 June 2021

This program is tentative and subject to change.

The widespread adoption of Deep Neural Networks (DNNs) in important domains raises questions about the trustworthiness of DNN outputs. Even a highly accurate DNN will make mistakes some of the time, and in settings like self-driving vehicles these mistakes must be quickly detected and properly dealt with \emph{in deployment}.

Just as our community has developed effective techniques and mechanisms to monitor and check programmed components, we believe it is now necessary to do the same for DNNs. In this paper we present DNN self-checking as a process by which internal DNN layer features are used to check DNN predictions. We detail \emph{SelfChecker}, a self-checking system that monitors DNN outputs and triggers an alarm if the internal layer features of the model are inconsistent with the final prediction. SelfChecker also provides \emph{advice} in the form of an alternative prediction.

We evaluated SelfChecker on four popular image datasets and three DNN models and found that SelfChecker triggers correct alarms on 60.56% of wrong DNN predictions, and false alarms on 2.04% of correct DNN predictions. This is a substantial improvement over prior work (SelfOracle, Dissector, and ConfidNet). In experiments with self-driving car scenarios, SelfChecker triggers more correct alarms than SelfOracle for two DNN models (DAVE-2 and Chauffeur) with comparable false alarms. Our implementation is available as open source.

This program is tentative and subject to change.

Tue 25 May
Times are displayed in time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

10:30 - 11:30
1.1.3. Deep Neural Networks: Validation #1Technical Track at Blended Sessions Room 3 +12h
Chair(s): Oscar DiesteUniversidad Politécnica de Madrid
10:30
20m
Paper
Operation is the hardest teacher: estimating DNN accuracy looking for mispredictionsTechnical Track
Technical Track
Antonio GuerrieroUniversità di Napoli Federico II, Roberto PietrantuonoUniversità di Napoli Federico II, Stefano RussoUniversità di Napoli Federico II
Pre-print
10:50
20m
Paper
AUTOTRAINER: An Automatic DNN Training Problem Detection and Repair SystemTechnical Track
Technical Track
Xiaoyu ZhangXi'an Jiaotong University, Juan ZhaiRutgers University, Shiqing MaRutgers University, Chao ShenXi'an Jiaotong University
Pre-print
11:10
20m
Paper
Self-Checking Deep Neural Networks in DeploymentTechnical Track
Technical Track
Yan XiaoNational University of Singapore, Ivan BeschastnikhUniversity of British Columbia, David S. RosenblumGeorge Mason University, Changsheng SunNational University of Singapore, Sebastian ElbaumUniversity of Virginia, Yun LinNational University of Singapore, Jin Song DongNational University of Singapore
Pre-print
22:30 - 23:30
1.1.3. Deep Neural Networks: Validation #1Technical Track at Blended Sessions Room 3
22:30
20m
Paper
Operation is the hardest teacher: estimating DNN accuracy looking for mispredictionsTechnical Track
Technical Track
Antonio GuerrieroUniversità di Napoli Federico II, Roberto PietrantuonoUniversità di Napoli Federico II, Stefano RussoUniversità di Napoli Federico II
Pre-print
22:50
20m
Paper
AUTOTRAINER: An Automatic DNN Training Problem Detection and Repair SystemTechnical Track
Technical Track
Xiaoyu ZhangXi'an Jiaotong University, Juan ZhaiRutgers University, Shiqing MaRutgers University, Chao ShenXi'an Jiaotong University
Pre-print
23:10
20m
Paper
Self-Checking Deep Neural Networks in DeploymentTechnical Track
Technical Track
Yan XiaoNational University of Singapore, Ivan BeschastnikhUniversity of British Columbia, David S. RosenblumGeorge Mason University, Changsheng SunNational University of Singapore, Sebastian ElbaumUniversity of Virginia, Yun LinNational University of Singapore, Jin Song DongNational University of Singapore
Pre-print

Information for Participants
Info for Blended Sessions Room 3: