There are several widely accepted metrics to measure code quality that are currently being used in both research and practice to detect code smells and to find opportunities for code improvement. Although these metrics have been proposed as a proxy of code quality, recent research suggests that more often than not, state-of-the-art code quality metrics do not successfully capture quality improvements in the source code as perceived by developers. More specifically, results show that there may be inconsistencies between, on the one hand, the results from metrics for cohesion, coupling, complexity, and readability, and, on the other hand, the interpretation of these metrics in practice. As code improvement tools rely on these metrics, there is a clear need to identify and resolve the aforementioned inconsistencies. This will allow for the creation of tools that are more aligned with developers’ perception of quality, and can more effectively help source code improvement efforts. In this study, we investigate 548 instances of source code readability improvements, as explicitly stated by internal developers in practice, from 63 engineered software projects. We show that current readability models fail to capture readability improvements. We also show that tools to calculate additional metrics, to detect refactorings, and to detect style problems are able to capture characteristics that are specific to readability changes and thus should be considered by future readability models.