SEAMS 2024
Mon 15 - Tue 16 April 2024 Lisbon, Portugal
co-located with ICSE 2024

A/B testing is a common approach used in industry to facilitate innovation through the introduction of new features or the modification of existing software. Traditionally, A/B tests are administrated manually and conducted sequentially, with each experiment targeting the entire population that uses the corresponding application. This approach can be time-consuming and costly, particularly when the experiments are not relevant to the entire population. To tackle these problems, we present a new self-adaptive approach called AutoPABS, short for Automated Pipelines of A/B tests using Self-adaptation. AutoPABS contributes: (1) a self-adaptive architecture that enables automated execution of A/B testing pipelines, (2) a population split component that is managed by the managing system of the self-adaptive architecture to enable more efficient A/B testing pipeline execution, and (3) a notation to specify A/B testing pipelines that can be executed using the self-adaptive architecture. We started the evaluation with a small survey to probe the appraisal of the notation and infrastructure of AutoPABS. Then we performed a series of tests to measure the gains obtained by applying a population split in an automated A/B testing pipeline, using an extension of the SEAByTE artifact. The survey results show that the participants appraise the usefulness of automating A/B testing pipelines and population split. The test results show that automatically executing pipelines of A/B tests using the self-adaptation architecture with a population split component accelerates the identification of statistically significant results by exploiting parallel A/B test execution compared to a traditional approach that executes the A/B tests sequentially.

Mon 15 Apr

Displayed time zone: Lisbon change

16:00 - 17:30
Session 4: Testing + Community DebateResearch Track at Luis de Freitas Branco
Chair(s): Siobhán Clarke Trinity College Dublin, Ireland, Bradley Schmerl Carnegie Mellon University, USA
16:00
25m
Talk
Automating Pipelines of A/B Tests with Population Split Using Self-Adaptation and Machine LearningFULL
Research Track
Federico Quin Katholieke Universiteit Leuven, Danny Weyns KU Leuven
16:25
15m
Talk
Generating Executable Test Scenarios from Autonomous Vehicle Disengagements using Natural Language ProcessingSHORT
Research Track
Qunying Song Lund University, Rune Anderberg Lund University, Henrik Olsson Lund University, Per Runeson Lund University
16:40
50m
Panel
Panel: Should the adaptive software systems community re-visit bio-inspired algorithms given advances in ML and more general research attention on bio-diversity and sustainability?COMMUNITY DEBATE
Research Track
Darko Bozhinoski Université Libre de Bruxelles, Rogério de Lemos University of Kent, UK, Sona Ghahremani Hasso Plattner Institute, University of Potsdam, Andrew Jackson Trinity College Dublin, Ireland