JSON is a data format used pervasively in web APIs, cloud computing,
NoSQL databases, and increasingly also machine learning.
To ensure that JSON data is compatible with an application, one
can define a JSON schema and use a validator to check data against
the schema. However, because validation can happen only once
concrete data occurs during an execution, it may detect data compatibility
bugs too late or not at all. Examples include evolving
the schema for a web API, which may unexpectedly break client
applications, or accidentally running a machine learning pipeline
on incorrect data. This paper presents a novel way of detecting
a class of data compatibility bugs via JSON subschema checking.
Subschema checks find bugs before concrete JSON data is available
and across all possible data specified by a schema. For example,
one can check if evolving a schema would break API clients or if
two components of a machine learning pipeline have incompatible
expectations about data. Deciding whether one JSON schema is
a subschema of another is non-trivial because the JSON Schema
specification language is rich. Our key insight to address this challenge
is to first reduce the richness of schemas by canonicalizing
and simplifying them, and to then reason about the subschema
question on simpler schema fragments using type-specific checkers.
We apply our subschema checker to thousands of real-world
schemas from different domains. In all experiments, the approach
is correct whenever it gives an answer (100% precision and correctness),
which is the case for most schema pairs (93.5% recall), clearly
outperforming the state-of-the-art tool. Moreover, the approach
reveals 43 previously unknown bugs in popular software, most
of which have already been fixed, showing that JSON subschema
checking helps finding data compatibility bugs early.