Performance Analysis with Bayesian Inference
Statistics are part of any empirical science, and performance analysis is no exception. However, for non-statisticians, picking the right statistical tool to answer a research question can be challenging; each statistical tool comes with a set of assumptions, and it is not clear to researchers what happens when those assumptions are violated. Bayesian statistics offers a framework with more flexibility and with explicit assumptions. In this paper, we present a method to analyse benchmark results using Bayesian inference. We demonstrate how to perform a Bayesian analysis of variance (ANOVA) to estimate what factors matter most for performance, and describe how to investigate what factors affect the impact of optimizations. We find the Bayesian model more flexible, and the Bayesian ANOVA’s output easier to interpret.
Fri 19 MayDisplayed time zone: Hobart change
13:45 - 15:15 | Software performanceDEMO - Demonstrations / NIER - New Ideas and Emerging Results / Technical Track / SEIP - Software Engineering in Practice at Level G - Plenary Room 1 Chair(s): Philipp Leitner Chalmers University of Technology, Sweden / University of Gothenburg, Sweden | ||
13:45 15mTalk | Analyzing the Impact of Workloads on Modeling the Performance of Configurable Software Systems Technical Track Stefan Mühlbauer Leipzig University, Florian Sattler Saarland Informatics Campus, Saarland University, Christian Kaltenecker Saarland University, Germany, Johannes Dorn Leipzig University, Sven Apel Saarland University, Norbert Siegmund Leipzig University Pre-print | ||
14:00 15mTalk | Twins or False Friends? A Study on Energy Consumption and Performance of Configurable Software Technical Track Max Weber Leipzig University, Christian Kaltenecker Saarland University, Germany, Florian Sattler Saarland Informatics Campus, Saarland University, Sven Apel Saarland University, Norbert Siegmund Leipzig University Link to publication | ||
14:15 15mTalk | Auto-tuning elastic applications in production SEIP - Software Engineering in Practice Adalberto R. Sampaio Jr Huawei Canada, Ivan Beschastnikh University of British Columbia, Daryl Maier IBM Canada, Don Bourne IBM Canada, Vijay Sundaresan IBM Canada | ||
14:30 7mTalk | CryptOpt: Automatic Optimization of Straightline Code DEMO - Demonstrations Joel Kuepper University of Adelaide, Andres Erbsen MIT, Jason Gross MIT CSAIL, Owen Conoly MIT, Chuyue Sun Stanford, Samuel Tian MIT, David Wu University of Adelaide, Adam Chlipala Massachusetts Institute of Technology, Chitchanok Chuengsatiansup University of Adelaide, Daniel Genkin Georgia Tech, Markus Wagner Monash University, Australia, Yuval Yarom Ruhr University Bochum Link to publication | ||
14:37 7mTalk | Performance Analysis with Bayesian Inference NIER - New Ideas and Emerging Results Noric Couderc Lund University, Christoph Reichenbach Lund University, Emma Söderberg Lund University | ||
14:45 15mTalk | Runtime Performance Prediction for Deep Learning Models with Graph Neural Network SEIP - Software Engineering in Practice Yanjie Gao Microsoft Research, Xianyu Gu Tsinghua University, Hongyu Zhang The University of Newcastle, Haoxiang Lin Microsoft Research, Mao Yang Microsoft Research Pre-print | ||
15:00 7mTalk | Judging Adam: Studying the Performance of Optimization Methods on ML4SE Tasks NIER - New Ideas and Emerging Results Dmitry Pasechnyuk Mohammed bin Zayed University of Artificial Intelligence, UAE, Anton Prazdnichnykh , Mikhail Evtikhiev JetBrains Research, Timofey Bryksin JetBrains Research | ||
15:07 7mTalk | Who Ate My Memory? Towards Attribution in Memory Management SEIP - Software Engineering in Practice Gunnar Kudrjavets University of Groningen, Ayushi Rastogi University of Groningen, The Netherlands, Jeff Thomas Meta Platforms, Inc., Nachiappan Nagappan Facebook Pre-print | ||