Deep Neural Network (DNN) testing is one of the most widely-used techniques to guarantee the quality of DNNs. However, DNN testing typically requires the ground truth of test inputs, which is time-consuming and labor-intensive to obtain. To relieve the labeling-cost problem of DNN testing, we propose TDPR, a test input prioritization technique for DNNs based on training dynamics. The key insight of TDPR is that bug-revealing samples exhibit different learning trajectories compared to normal ones. Based on this, TDPR constructs a learning trajectory for each test input, which characterizes the evolving learning behavior of DNNs. Then, TDPR extracts features from these learning trajectories and applies learning-to-rank techniques to build a ranking model, which can intelligently utilize the generated features to prioritize test inputs. To evaluate TDPR, we conduct extensive experiments on 8 diverse subjects, considering various domains of test inputs, different DNN architectures, and diverse types of test inputs. The evaluation results demonstrate that TDPR outperforms 7 baseline approaches in both prioritizing test inputs and guiding the retraining of DNNs.
Wed 30 OctDisplayed time zone: Pacific Time (US & Canada) change
15:30 - 16:30 | |||
15:30 15mTalk | UFront: Toward A Unified MLIR Frontend for Deep Learning Research Papers Guoqing Bao Shanghai Enflame Technology Co., Ltd., Heng Shi Shanghai Jiao Tong University, Shanghai Enflame Technology Co., Ltd., Chengyi Cui Shanghai Enflame Technology Co., Ltd., Yalin Zhang Shanghai Enflame Technology Co., Ltd., Jianguo Yao Shanghai Jiao Tong University; Shanghai Enflame Technology | ||
15:45 15mTalk | FIPSER: Improving Fairness Testing of DNN by Seed Prioritization Research Papers Junwei Chen East China Normal University, Yueling Zhang East China Normal University, Lingfeng Zhang East China Normal University, Min Zhang East China Normal University, Chengcheng Wan East China Normal University, Ting Su East China Normal University, Geguang Pu East China Normal University, China | ||
16:00 15mTalk | Prioritizing Test Inputs for DNNs Using Training Dynamics Research Papers Jian Shen Nanjing University, Zhong Li , Minxue Pan Nanjing University, Xuandong Li Nanjing University | ||
16:15 15mTalk | Learning DNN Abstractions using Gradient DescentRecorded Talk NIER Track Diganta Mukhopadhyay TCS Research, Pune, India, Sanaa Siddiqui Indian Institute of Technology Delhi, New Delhi, India, Hrishikesh Karmarkar TCS Research, Kumar Madhukar Indian Institute of Technologiy Delhi, New Delhi, India, Guy Katz The Hebrew University of Jerusalem |