ICSME 2024
Sun 6 - Fri 11 October 2024
Wed 9 Oct 2024 15:45 - 15:55 at Fremont - Session 6: Maintenance of AI-based Systems Chair(s): Sujoy Roychowdhury

In property-based testing (PBT), developers specify properties that they expect the system under test to hold. The PBT tool generates random inputs for the system and tests for each of these inputs whether the given property holds. An advantage of this approach over testing a set of manually defined example inputs is that it enables a higher code coverage.

Machine learning (ML) projects, however, often have to process large amounts of diverse data, both for training a model and afterwards, when the trained model is deployed. Generating a sufficient amount of diverse data for the property-based tests is therefore challenging.

In this paper, we present the results of a preliminary study in which we examined a dataset of 58 open-source ML projects that have dependencies on the popular PBT library Hypothesis, to identify issues faced by developers writing property-based tests. For a subset of 28 open-source ML projects, we study the property-based tests in detail and report on the part of the ML project that is being tested as well as on the adopted data generation strategies. This way, we aim to identify issues in porting current PBT techniques to ML projects so that they can be addressed in the future.

Wed 9 Oct

Displayed time zone: Arizona change

15:30 - 17:00
Session 6: Maintenance of AI-based SystemsResearch Track / Industry Track / New Ideas and Emerging Results Track at Fremont
Chair(s): Sujoy Roychowdhury Ericsson R&D
15:30
15m
A Taxonomy of Self-Admitted Technical Debt in Deep Learning SystemsResearch Track Paper
Research Track
Federica Pepe , Fiorella Zampetti University of Sannio, Italy, Antonio Mastropaolo William and Mary, USA, Gabriele Bavota Software Institute @ Università della Svizzera Italiana, Massimiliano Di Penta University of Sannio, Italy
Pre-print
15:45
10m
Property-based Testing within ML Projects: an Empirical StudyNIER Paper
New Ideas and Emerging Results Track
Cindy Wauters Vrije Universiteit Brussel, Coen De Roover Vrije Universiteit Brussel
Pre-print
15:55
15m
Toward Debugging Deep Reinforcement Learning Programs with RLExplorerResearch Track Paper
Research Track
Rached Bouchoucha Polytechnique Montréal, Ahmed Haj Yahmed École Polytechnique de Montréal, Darshan Patil , Janarthanan Rajendran , Amin Nikanjam École Polytechnique de Montréal, Sarath Chandar Polytechnique Montréal, Foutse Khomh Polytechnique Montréal
16:10
15m
Ghost Echoes Revealed: Benchmarking Maintainability Metrics and Machine Learning Predictions Against Human AssessmentsIndustry Track Paper
Industry Track
Markus Borg CodeScene, Marwa Ezzouhri University of Clermont Auvergne, Adam Tornhill Codescene AB
Pre-print
16:25
10m
RetypeR: Integrated Retrieval-based Automatic Program Repair for Python Type ErrorsVideo presentationResearch Track Paper
Research Track
Sichong Hao Faculty of Computing, Harbin Institute of Technology, Xianjun Shi , Hongwei Liu Faculty of Computing, Harbin Institute of Technology
16:35
10m
OPass: Orchestrating TVM's Passes for Lowering Memory Footprints of Computation GraphsVideo presentationResearch Track Paper
Research Track
Pengbo Nie Shanghai Jiao Tong University, Zihan Wang Shanghai Jiao Tong University, Chengcheng Wan East China Normal University, Ziyi Lin Alibaba Group, He Jiang Dalian University of Technology, Jianjun Zhao Kyushu University, Yuting Chen Shanghai Jiao Tong University