Software isn’t created in one dramatic step. It improves bit by bit, one little step at a time — editing, running unit tests, fixing build errors, addressing code reviews, editing some more, appeasing linters, and fixing more errors — until finally it becomes good enough to merge into a code repository. Software engineering isn’t an isolated process, but a dialogue among human developers, code reviewers, bug reporters, software architects and tools, such as compilers, unit tests, linters and static analyzers. I’ll talk about DIDACT(Dynamic Integrated Developer ACTivity), which is a methodology for training large machine learning (ML) models for software development. The novelty of DIDACT is that it uses the process of software development as the source of training data for the model, rather than just the polished end state of that process, the finished code. By exposing the model to the contexts that developers see as they work, paired with the actions they take in response, the model learns about the dynamics of software development and is more aligned with how developers spend their time. We leverage instrumentation of Google’s software development to scale up the quantity and diversity of developer-activity data beyond previous works. Results are promising along two dimensions: usefulness to professional software developers, and as a potential basis for imbuing ML models with general software development skills.

Danny is a Senior Staff Research Scientist at Google DeepMind and a lead of Code AI efforts there. He is also an Adjunct Professor in the Dept of Computer Science at McGill University and a Core Industrial Member at the Mila Quebec AI Institute. He was one of the early people working on Machine Learning for Code and has made a number of contributions to the area over the last 10 years, including DeepCoder, Terpret, generative models of code, and natural language + code modeling. In addition, he has made a number of contributions to core machine learning, including early work on the revival of graph neural networks and a Best Paper Award at NeurIPS for work on sampling. Over the last six years, he has been at Google working on making AI-powered developer tools useful in real software development processes. He holds a PhD in Machine Learning from the University of Toronto and was previously a Researcher at Microsoft Research Cambridge and a Research Fellow at the University of Cambridge.