Machine Translation (MT) is the backbone of a plethora of systems and applications that are present in users’ everyday lives. Despite the research efforts and progress in the MT domain, translation remains a challenging task and MT systems struggle when translating rare words, named entities, domain-specific terminology, idiomatic expressions and culturally specific terms. Thus, to meet the translation performance expectations of the users, engineers are tasked with periodically updating (fine-tuning) MT models to guarantee high translation quality. However, with ever-growing machine learning models, fine-tuning operations become increasingly more expensive, raising serious concerns from a sustainability perspective. Furthermore, not all fine-tunings are guaranteed to lead to increased translation quality, thus corresponding to wasted compute resource.

To address this issue and enhance the sustainability of MT systems, we present FLEXICO, a new approach to engineer self-adaptive MT systems, which leverages (i) black-box predictors to estimate the expected benefits of fine-tuning MT models; and (ii) probabilistic model checking techniques to automate the reasoning about when the benefits of fine-tuning outweigh its costs. Our empirical evaluation on two MT models and language-pairs and across up to 9 domains demonstrates the predictive performance of the black-box models that estimate the expected benefits of fine-tuning, as well as their domain-generalizability. Finally, we show that FLEXICO optimizes system utility when compared to naive baselines, decreasing the number of fine-tunings required to achieve high translation quality.