In response to challenges in software supply chain security, several organisations have created infrastructures to independently build commodity open source projects and release the resulting binaries for Java/Maven and other software ecosystems. Build platform variability can strengthen security as it facilitates the detection of compromised build environments. Furthermore, by improving the security posture of the build platform and collecting provenance information during the build, the resulting artifacts can be used with greater trust. Such offerings are now available from Google, Oracle and RedHat. The availability of multiple binaries built from the same sources creates new challenges and opportunities, and raises questions such as: “Does build A confirm the integrity of build B?” or “Can build A reveal a compromised build B?”.
To answer such questions requires a notion of equivalence between binaries. We demonstrate that the obvious approach based on bitwise equality has significant shortcomings in practice, and propose an alternative approach based on levels of equivalence, inspired by clone detection types.

We demonstrate the value of these new levels through several experiments. For this purpose, we construct a dataset consisting of Java binaries (jar files) built from the same sources independently by different providers, resulting in 14,156 pairs of binaries in total.
We then compare the compiled class files in those jar files and find that for 3,750 pairs of jars (26.49%) there is at least one such file that is different, also forcing the jar files and their cryptographic hashes to be different. However, based on the new equivalence levels, we can still establish that many of them are practically equivalent; the number of pairs of jars with non-equivalent classes drops to 13.65 % in some cases. We evaluate several candidate equivalence relations on a semi-synthetic dataset that provides oracles consisting of pairs of binaries that either should be, or must not be equivalent.

This technique has been applied to evaluate artifacts built from source and used within Oracle’s Graal Development Kit for Micronaut (GDK) product.