Revisiting Process versus Product Metrics: a Large Scale Analysi
Thu 12 May 2022 13:05 - 13:10 at ICSE room 4-odd hours - Machine Learning with and for SE 12 Chair(s): Wei Yang
Numerous methods can build predictive models from software data. However, what methods and conclusions should we endorse as we move from analytics in-the-small (dealing with a handful of projects) to analytics in-the-large (dealing with hundreds of projects)?
To answer this question, we recheck prior small-scale results (about process versus product metrics for defect prediction and the granularity of metrics) using 722,471 commits from 700 Github projects. We find that some analytics in-the-small conclusions still hold when scaling up to analytics in-the-large. For example, like prior work, we see that process metrics are better predictors for defects than product metrics (best process/product-based learners respectively achieve recalls of 98%/44% and AUCs of 95%/54%, median values).
That said, we warn that it is unwise to trust metric importance results from analytics in-the-small studies since those change dramatically when moving to analytics in-the-large. Also, when reasoning in-the-large about hundreds of projects, it is better to use predictions from multiple models (since single model predictions can become confused and exhibit a high variance).
|Revisiting Process versus Product Metrics: a Large Scale Analysis (Revisiting Process versus Product Metrics- a Large Scale Analysis.pdf)||2.3MiB|
Mon 9 MayDisplayed time zone: Eastern Time (US & Canada) change
20:00 - 21:00
Machine Learning with and for SE 4Journal-First Papers / Technical Track / SEIP - Software Engineering in Practice at ICSE room 1-even hours
Chair(s): Gias Uddin University of Calgary, Canada
|Revisiting Process versus Product Metrics: a Large Scale Analysi|
Suvodeep Majumder North Carolina State University, Pranav Mody North Carolina State University, Tim Menzies North Carolina State UniversityLink to publication DOI Pre-print Media Attached File Attached
|Learning to Recognize Actionable Static Code Warnings (is Intrinsically Easy)|
Xueqi Yang NCSU, Jianfeng Chen North Carolina State University, Rahul Yedida North Carolina State University, Zhe Yu , Tim Menzies North Carolina State UniversityLink to publication DOI Pre-print Media Attached
|Mining Root Cause Knowledge from Cloud Service Incident Investigations for AIOps|
SEIP - Software Engineering in Practice
Amrita Saha Salesforce Research Asia, Steven C.H. Hoi Salesforce Research AsiaPre-print Media Attached
|FairNeuron: Improving Deep Neural Network Fairness with Adversary Games on Selective Neurons|
Xuanqi Gao Xi'an Jiaotong University, Juan Zhai Rutgers University, Shiqing Ma Rutgers University, Chao Shen Xi'an Jiaotong University, Yufei Chen Xi'an Jiaotong University, Qian Wang Wuhan UniversityDOI Pre-print Media Attached
|EREBA: Black-box Energy Testing of Adaptive Neural Networks|
Mirazul Haque UT Dallas, Yaswanth Yadlapalli University of Texas at Dallas, Wei Yang University of Texas at Dallas, Cong Liu University of Texas at Dallas, USAPre-print Media Attached
|Training Data Debugging for the Fairness of Machine Learning Software|
Yanhui Li Department of Computer Science and Technology, Nanjing University, Linghan Meng Nanjing University, Lin Chen Department of Computer Science and Technology, Nanjing University, Li Yu Nanjing University, Di Wu Momenta, Yuming Zhou Nanjing University, Baowen Xu Nanjing UniversityPre-print Media Attached