MSR 2023
Dates to be announced Melbourne, Australia
co-located with ICSE 2023
Mon 15 May 2023 11:36 - 11:42 at Meeting Room 110 - SE for ML Chair(s): Sarah Nadi

Due to the cost of developing and training deep learning models from scratch, machine learning engineers have begun to reuse pre-trained models (PTMs) and fine-tune them for downstream tasks. PTM registries known as “model hubs” support engineers in distributing and reusing deep learning models. PTM packages include pre-trained weights, documentation, model architectures, datasets, and metadata. Mining the information in PTM packages will enable the discovery of engineering phenomena and tools to support software engineers. However, accessing this information is difficult — there are many PTM registries, and both the registries and the individual packages may have rate limiting for accessing the data. We present an open-source dataset, PTMTorrent, to facilitate the evaluation and understanding of PTM packages. This paper describes the creation, structure, usage, and limitations of the dataset. The dataset includes a snapshot of 5 model hubs and a total of 15,913 PTM packages. These packages are represented in a uniform data schema for cross-hub mining. We describe prior uses of this data and suggest research opportunities for mining using our dataset. %2F%7E%2F.The PTMTorrent dataset (v1) is available at: https://app.globus.org/file-manager?origin_id=55e17a6e-9d8f-11ed-a2a2-8383522b48d9&origin_path= Our dataset generation tools are available on GitHub: https://doi.org/10.5281/zenodo.7570357.

Mon 15 May

Displayed time zone: Hobart change

11:00 - 11:45
SE for MLData and Tool Showcase Track / Technical Papers at Meeting Room 110
Chair(s): Sarah Nadi University of Alberta
11:00
12m
Talk
AutoML from Software Engineering Perspective: Landscapes and ChallengesDistinguished Paper Award
Technical Papers
Chao Wang Peking University, Zhenpeng Chen University College London, UK, Minghui Zhou Peking University
Pre-print
11:12
12m
Talk
Characterizing and Understanding Software Security Vulnerabilities in Machine Learning Libraries
Technical Papers
Nima Shiri Harzevili York University, Jiho Shin York University, Junjie Wang Institute of Software at Chinese Academy of Sciences; University of Chinese Academy of Sciences, Song Wang York University, Nachiappan Nagappan Facebook
11:24
6m
Talk
DeepScenario: An Open Driving Scenario Dataset for Autonomous Driving System Testing
Data and Tool Showcase Track
Chengjie Lu Simula Research Laboratory and University of Oslo, Tao Yue Simula Research Laboratory, Shaukat Ali Simula Research Laboratory
Pre-print
11:30
6m
Talk
NICHE: A Curated Dataset of Engineered Machine Learning Projects in Python
Data and Tool Showcase Track
Ratnadira Widyasari Singapore Management University, Singapore, Zhou Yang Singapore Management University, Ferdian Thung Singapore Management University, Sheng Qin Sim Singapore Management University, Singapore, Fiona Wee Singapore Management University, Singapore, Camellia Lok Singapore Management University, Singapore, Jack Phan Singapore Management University, Singapore, Haodi Qi Singapore Management University, Singapore, Constance Tan Singapore Management University, Singapore, Qijin Tay Singapore Management University, Singapore, David Lo Singapore Management University
11:36
6m
Talk
PTMTorrent: A Dataset for Mining Open-source Pre-trained Model Packages
Data and Tool Showcase Track
Wenxin Jiang Purdue University, Nicholas Synovic Loyola University Chicago, Purvish Jajal Purdue University, Taylor R. Schorlemmer Purdue University, Arav Tewari Purdue University, Bhavesh Pareek Purdue University, George K. Thiruvathukal Loyola University Chicago and Argonne National Laboratory, James C. Davis Purdue University
Pre-print