ICST 2023
Sun 16 - Thu 20 April 2023 Dublin, Ireland
Thu 20 Apr 2023 15:00 - 15:30 at Macken - Session 3

Google Compute Engine runs millions of VMs with highly varied configurations. The size of the configuration input space makes it difficult to identify and test all parameter interactions and make traditional test approaches insufficient. Random testing is unable to detect bugs in the long tail of the input space, exhaustively testing each release against all VM configurations in prod is not computationally feasible, a static input matrix would quickly become out of date as new features are released. This talk presents our system for detecting untested parameter interactions before releasing new software to production by validating test results against a domain model. We use boolean rules to define the high-level compatibility requirements of each feature and automatically map them to physical configuration values. These rules are also used to generate new tests that provide comprehensive t-way test coverage over our configuration input space. By automatically generating these tests, we can keep our coverage up-to-date as product features change and also remove the need for engineers to manually write interaction tests.