Write a Blog >>
ICSE 2020
Mon 5 - Sun 11 October 2020 Yongsan-gu, Seoul, South Korea
Wed 7 Oct 2020 14:30 - 14:40 at TBD2 - Testing 2

Intermittent test failures (test flakiness) is common during continuous integration as modern software systems have become inherently non-deterministic. Understanding the root cause of test flakiness is crucial as intermittent test failures might be the result of real non-deterministic defects in the production code, rather than mere errors in the test code. Existing techniques for root causing test flakiness compare the runtime behaviour of passing and failing executions of the same test. They achieve this by repetitively executing a flaky test on an instrumented version of the system under test. This approach has two fundamental limitations: (i) code instrumentation might disrupt the normal behaviour of the test preventing the manifestation of test flakiness; (ii) passively re-executing a test many times could be insufficient for triggering intermittent test outcomes when test flakiness is rare or difficult to manifest. To address these limitations, we propose a new approach for root causing test flakiness that actively explores the non-deterministic space, without instrumenting code. Our novel idea is to repetitively execute a flaky test, under different execution clusters. Each cluster explores a certain non-deterministic dimension (e.g., concurrency, I/O, and networking) with dedicated software containers and fuzzy-driven resource load generators. The execution cluster that manifests the most balanced (or unbalanced) sets of passing and failing executions is likely to explain the broad type of test flakiness.

Wed 7 Oct

14:00 - 15:40: Paper Presentations - Testing 2 at TBD2
icse-2020-Software-Engineering-in-Practice14:00 - 14:20
Emily KowalczykApple Inc., Karan NairApple, Zebao GaoApple, Leopold SilbersteinApple Inc., Teng LongApple, Atif MemonApple Inc.
Demonstrations14:20 - 14:30
Bruno LimaFaculty of Engineering, University of Porto and INESC TEC, João Pascoal FariaFaculty of Engineering, University of Porto and INESC TEC
icse-2020-New-Ideas-and-Emerging-Results14:30 - 14:40
Valerio TerragniUniversità della Svizzera Italiana, Pasquale SalzaUniversity of Zurich, Filomena FerrucciUniversity of Salerno
icse-2020-Software-Engineering-in-Practice14:40 - 15:00
Mingyu ParkLG Electronics, Hoon JangHyundai Motor Company, Taejoon ByunUniversity of Minnesota, Yunja ChoiKyungpook National University
icse-2020-papers15:00 - 15:20
Wing LamUniversity of Illinois at Urbana-Champaign, Kıvanç MuşluMicrosoft, Hitesh SajnaniMicrosoft , Suresh ThummalapentaMicrosoft
icse-2020-papers15:20 - 15:40
Dongjie ChenNanjing University, Yanyan JiangNanjing University, Chang XuNanjing University, Xiaoxing Ma, Jian LuNanjing University