PyEvolve: Automating Frequent Code Changes in Python ML Systems
Because of the naturalness of software and the rapid evolution of Machine Learning (ML) techniques, frequently repeated code change patterns (CPATs) occur often. They range from simple API migrations to changes involving several nested control structures such as for loops. While manually performing CPATs is tedious, the current state-of-the-art techniques for inferring transformation rules are not advanced enough to handle unseen variants of complex CPATs, resulting in a low recall rate. In this paper we present a novel, automated workflow that mines CPATs, infers the transformation rules, and then transplants them automatically to new target sites. We designed, implemented, evaluated and released this in a tool, PyEvolve. At its core is a novel data-flow, control-flow aware transformation rule inference engine. Our technique allows us to advance the state-of-the-art for transformation-by-example tools; without it, 70% of the code changes that PyEvolve transforms would not be possible to automate. Our thorough empirical evaluation of over 40,000 transformations shows 97% precision and 94% recall. By accepting 90% of CPATs generated by PyEvolve in famous open-source projects, developers confirmed that our tool is useful.
Wed 17 MayDisplayed time zone: Hobart change
15:45 - 17:15 | Development and evolution of AI-intensive systemsSEIP - Software Engineering in Practice / Technical Track / NIER - New Ideas and Emerging Results at Meeting Room 104 Chair(s): Sebastian Elbaum University of Virginia | ||
15:45 15mTalk | Reusing Deep Neural Network Models through Model Re-engineering Technical Track Binhang Qi Beihang University, Hailong Sun Beihang University, Xiang Gao Beihang University, China, Hongyu Zhang The University of Newcastle, Zhaotian Li Beihang University, Xudong Liu Beihang University | ||
16:00 15mTalk | PyEvolve: Automating Frequent Code Changes in Python ML Systems Technical Track Malinda Dilhara University of Colorado Boulder, USA, Danny Dig JetBrains Research & University of Colorado Boulder, USA, Ameya Ketkar Uber Pre-print | ||
16:15 15mTalk | DeepArc: Modularizing Neural Networks for the Model Maintenance Technical Track xiaoning ren , Yun Lin Shanghai Jiao Tong University; National University of Singapore, Yinxing Xue University of Science and Technology of China, Ruofan Liu National University of Singapore, Jun Sun Singapore Management University, Zhiyong Feng Tianjin University, Jin Song Dong National University of Singapore | ||
16:30 15mTalk | Decomposing a Recurrent Neural Network into Modules for Enabling Reusability and Replacement Technical Track Sayem Mohammad Imtiaz Iowa State University, Fraol Batole Dept. of Computer Science, Iowa State University, Astha Singh Dept. of Computer Science, Iowa State University, Rangeet Pan IBM Research, Breno Dantas Cruz Dept. of Computer Science, Iowa State University, Hridesh Rajan Iowa State University Pre-print | ||
16:45 7mTalk | Safe-DS: A Domain Specific Language to Make Data Science Safe NIER - New Ideas and Emerging Results Pre-print | ||
16:52 7mTalk | Rapid Development of Compositional AI NIER - New Ideas and Emerging Results Lee Martie MIT-IBM Watson AI Lab, Jessie Rosenberg IBM, Veronique Demers MIT-IBM Watson AI Lab, Gaoyuan Zhang IBM, Onkar Bhardwaj MIT-IBM Watson AI Lab, John Henning IBM, Aditya Prasad IBM, Matt Stallone MIT-IBM Watson AI Lab, Ja Young Lee IBM, Lucy Yip IBM, Damilola Adesina IBM, Elahe Paikari IBM, Oscar Resendiz IBM, Sarah Shaw IBM, David Cox IBM Pre-print | ||
17:00 7mTalk | StreamAI: Challenges of Continual Learning Systems in Production for AI Industrialization SEIP - Software Engineering in Practice Mariam Barry BNP Paribas, Albert Bifet University of Waikato, Institut Polytechnique de Paris, Jean Luc Billy BNP Paribas |