What’s DAT? Three Case Studies of Measuring Software Development Productivity at Meta With Diff Authoring Time
This paper introduces Diff Authoring Time (DAT), a precise, yet conceptually simple approach to measuring software development productivity that enables rigorous experimentation. DAT is a time-based metric, which assess how much active work time engineers take to develop self-contained changes, so-called diffs. It uses a bespoke telemetry system integrated with version control, the Integrated Development Environments, and the Operating System. We validate DAT through observational studies, surveys, visualizations, and descriptive statistics. At Meta, DAT uplevels the internal tool development workflow to the scientifically grounded, experiment-driven development flow already present for external-facing products. As such, DAT enables rigorous experimentation on long-standing software engineering questions like “do types make development more efficient?” DAT has powered experiments and case studies on more than 20 projects at Meta. Here, we highlight (1) a controlled experiment on introducing mock types, which showed that typed mocks in tests yield a 14% DAT improvement, (2) a case study on the development of automatic memoization in the React compiler, a 33% improvement, and (3) a framework to estimate the thousands of DAT hours saved annually through code sharing (> 50% improvement). Overall, this paper contributes a novel, yet straight-forward way to measure development velocity, DAT. It describes how the use of this metric has enabled rigorous experimentation on development productivity at industry scale, and paves the way toward a more science-based mindset in internal product development.
| What's DAT? Presentation (icse2026_seip_whats_dat.pdf) | 361KiB |
Wed 15 AprDisplayed time zone: Brasilia, Distrito Federal, Brazil change
16:00 - 17:30 | Analytics 2SE In Practice (SEIP) / Research Track at Oceania I Chair(s): Emerson Murphy-Hill Microsoft | ||
16:00 15mTalk | Energy-Efficient Software Development: A Multi-dimensional Empirical Analysis of Stack Overflow Research Track Bihui Jin University of Waterloo, Heng Li Polytechnique Montréal, Pengyu Nie University of Waterloo, Ying Zou Queen's University, Kingston, Ontario Pre-print | ||
16:15 15mTalk | Towards Supporting Open Source Library Maintainers with Community-Based Analytics Research Track | ||
16:30 15mTalk | The Cost vs the Benefit of Adding an Extra Code Reviewer to Mitigate Developer Turnover through Reviewer Recommenders Research Track Mohammadali Sefidi Esfahani Concordia University, Fahimeh Hajari Concordia University, Peter Rigby Concordia University; Meta | ||
16:45 15mTalk | Toward Efficient Package Maintenance: An Empirical Study of Patch Sharing across Four Linux Distributions Research Track Jian Peng Institute of Software, Chinese Academy of Sciences, Jiaxin Zhu Institute of Software at Chinese Academy of Sciences, Yuwei Zhang Institute of Software Chinese Academy of Sciences, Wei Chen Institute of Software at Chinese Academy of Sciences, Guoquan Wu Institute of Software at Chinese Academy of Sciences; University of Chinese Academy of Sciences; University of Chinese Academy of Sciences Nanjing College; China Southern Power Grid, Wei Wang Institute of Software at Chinese Academy of Sciences, Jun Wei Institute of Software at Chinese Academy of Sciences; University of Chinese Academy of Sciences | ||
17:00 15mTalk | Evolving Trends, Patterns, and Hidden Pitfalls: Unveiling JavaScript Feature Usage in the Wild Research Track Dawei Chen Xi'an Jiaotong University, Wuxia Jin Xi'an Jiaotong University, Hui Guo Xi'an Jiaotong University, Guanlin Qiao Xi'an Jiaotong University, Peng Di Ant Group & UNSW Sydney, Ting Liu Xi'an Jiaotong University | ||
17:15 15mTalk | What’s DAT? Three Case Studies of Measuring Software Development Productivity at Meta With Diff Authoring Time SE In Practice (SEIP) Moritz Beller Meta Platforms, Inc., USA, Amanda Park Meta Platforms, Inc., Karim Nakad Meta, Akshay Patel Meta Platforms, Inc., Sarita Mohanty Meta Platforms, Inc., Ford Garberson Meta Platforms, Inc., Henri Verroken Meta Platforms, Inc., Andrew Kennedy Meta Platforms, Inc., Ian G. Malone Meta Platforms, Inc., Vaishali Garg Meta Platforms, Inc., Pavel Avgustinov Meta Platforms, Inc. Pre-print File Attached | ||