How Good is Good Enough? Non-Inferiority Trials for Requirements Trade-Offs in Self-Adaptive Systems
Self-adaptive systems (SAS) must make runtime decisions to balance trade-offs among competing quality-of-service (QoS) requirements — such as cost, performance, and reliability — under uncertain and dynamic conditions. Current approaches to support this, such as Pareto-based or utility-driven methods, often lack a quantifiable notion of what constitutes an acceptable loss in one requirement in favor of another. Inspired by practices in clinical trials, we propose a novel, requirements-centric application of Non-Inferiority (NI) Trials to decision-making in SAS. We reinterpret the NI margin — traditionally used to determine the acceptability of new treatments — as a stakeholder-specified tolerance threshold for QoS trade-offs. This offers a statistically grounded method to assess whether a new decision-making technique satisfies stakeholder-defined ”good enough” thresholds compared to established alternatives. We apply this approach to compare two reinforcement learning techniques in an SAS context and demonstrate how it captures nuanced trade-off decisions in QoS satisfaction. We argue that NI Trials can complement existing RE methods by introducing a principled mechanism for reasoning about acceptable degradation, supporting negotiation, monitoring, and prioritization of requirements in uncertain environments.