ASE 2024
Sun 27 October - Fri 1 November 2024 Sacramento, California, United States
Thu 31 Oct 2024 10:45 - 11:00 at Compagno - Program analysis 3 Chair(s): Xusheng Xiao

Machine learning models make mistakes, yet sometimes it is difficult to identify the systematic problems behind the mistakes. Practitioners engage in various activities, including error analysis, testing, auditing, and red-teaming, to form hypotheses of what can go (or has gone) wrong with their models. To validate these hypotheses, practitioners employ data slicing to identify examples relevant to their hypotheses. However, traditional data slicing is limited by available features and programmatic slicing functions. In this work, we propose SemSlicer, a framework that supports semantic data slicing, which identifies a semantically coherent slice, without the need for existing features. SemSlicer uses Large Language Models (LLMs) to annotate datasets and generate slices from any user-defined slicing criteria. We show that SemSlicer generates accurate slices with low cost, allows flexible trade-offs between different design dimensions, reliably identifies under-performing data slices, and helps practitioners identify useful data slices that reflect systematic problems.

Thu 31 Oct

Displayed time zone: Pacific Time (US & Canada) change

10:30 - 12:00
Program analysis 3Industry Showcase / Research Papers at Compagno
Chair(s): Xusheng Xiao Arizona State University
10:30
15m
Talk
Language-Agnostic Static Analysis of Probabilistic Programs
Research Papers
Markus Böck TU Wien, Michael Schröder TU Wien, Jürgen Cito TU Wien
10:45
15m
Talk
What Is Wrong with My Model? Identifying Systematic Problems with Semantic Data Slicing
Research Papers
Chenyang Yang , Yining Hong Carnegie Mellon University, Grace Lewis Carnegie Mellon Software Engineering Institute, Tongshuang Wu Carnegie Mellon University, Christian Kästner Carnegie Mellon University
11:00
15m
Talk
Interrogation Testing of Program Analyzers for Soundness and Precision Issues
Research Papers
David Kaindlstorfer TU Wien, Austria, Anastasia Isychev TU Wien, Valentin Wüstholz ConsenSys, Maria Christakis TU Wien
11:15
15m
Talk
Constraint-Based Test Oracles for Program Analyzers
Research Papers
Markus Fleischmann TU Wien, Austria, David Kaindlstorfer TU Wien, Austria, Anastasia Isychev TU Wien, Valentin Wüstholz ConsenSys, Maria Christakis TU Wien
11:30
15m
Talk
Program Synthesis Meets Visual What-Comes-Next Puzzles
Research Papers
Sumit Lahiri Indian Institute of Technology Kanpur, Pankaj Kumar Kalita IIT Kanpur, Akshay Kumar Chittora Indian Institute of Technology Kanpur, Varun Vankudre Indian Institute of Technology Kanpur, Subhajit Roy IIT Kanpur
11:45
15m
Talk
Enhancing Compositional Static Analysis with Dynamic Analysis
Industry Showcase