Challenges in Applying Continuous Experimentation: A Practitioners’ Perspective
Thu 12 May 2022 13:10 - 13:15 at ICSE room 2-odd hours - Tools and Environments 6 Chair(s): Domenico Bianculli
Background: Applying Continuous Experimentation on a large scale is not easily achieved. Although the evolution within large tech organisations is well understood, we still lack a good understanding of how to transition a company towards applying more experiments. Objective: This study investigates how practitioners define, value and apply experimentation, the blockers they experience and what to do to solve these. Method: We interviewed and surveyed over one hundred practitioners with regards to experimentation perspectives, from a large financial services and e-commerce organization, based in the Netherlands. Results: Many practitioners have different perspective about experimentation. The value is well understood. We have learned that the practitioners are blocker by a lack of priority, experience and well functioning tooling. Challenges also arise around dependencies between teams and evaluating experiments with the correct metrics. Conclusions: Organisation leaders need to start asking for experiment results and investing in infrastructure and processes to actually enable teams to execute experiments and show the value of their work in terms of value for customers and business.
Wed 11 MayDisplayed time zone: Eastern Time (US & Canada) change
05:00 - 06:00 | Tools and ConfigurationsTechnical Track / SEIP - Software Engineering in Practice at ICSE room 3-odd hours Chair(s): Robert Chatley Imperial College London | ||
05:00 5mTalk | Challenges in Applying Continuous Experimentation: A Practitioners’ Perspective SEIP - Software Engineering in Practice Kevin Anderson Delft University of Technology, Denise Visser bol.com, Jan-Willem Mannen ING, Yuxiang Jiang Delft University of Technology, Arie van Deursen Delft University of Technology, Netherlands DOI Pre-print | ||
05:05 5mTalk | Reflekt: a Library for Compile-Time Reflection in Kotlin SEIP - Software Engineering in Practice Anastasiia Birillo JetBrains Research, Elena Lyulina JetBrains Research, Maria Malysheva JetBrains Research;Saint Petersburg State University, Vladislav Tankov JetBrains, ITMO University, Timofey Bryksin JetBrains Research; HSE University Pre-print Media Attached | ||
05:10 5mTalk | ARCLIN: Automated API Mention Resolution for Unformatted Texts Technical Track Yintong Huo The Chinese University of Hong Kong, Yuxin Su Sun Yat-sen University, Hongming Zhang The Hong Kong University of Science and Technology, Michael Lyu The Chinese University of Hong Kong DOI Pre-print Media Attached | ||
05:15 5mTalk | On the Benefits and Limits of Incremental Build of Software Configurations: An Exploratory Study Technical Track Georges Aaron RANDRIANAINA Université de Rennes 1, IRISA, Xhevahire Tërnava Université de Rennes 1, INRIA/IRISA, Djamel Eddine Khelladi CNRS, France, Mathieu Acher Univ. Rennes 1, Inria, IRISA, Institut Universitaire de France (IUF) Pre-print Media Attached | ||
05:20 5mTalk | Causality in Configurable Software Systems Technical Track Clemens Dubslaff TU Dresden, Kallistos Weis Saarland University, Christel Baier TU Dresden, Germany, Sven Apel Saarland University Pre-print Media Attached | ||
05:25 5mTalk | A Scalable t-wise Coverage Estimator Technical Track Eduard Baranov Université Catholique de Louvain, Belgium, Sourav Chakraborty Indian Statistical Institute (ISI) , Kolkata, India, Axel Legay Université Catholique de Louvain, Belgium, Kuldeep S. Meel National University of Singapore, N. V. Vinodchandran University of Nebraska-Lincoln DOI Pre-print Media Attached |