This document presents the artifact for the paper “Hypertesting of Programs: Theoretical Foundation and Automated Test Generation’”, accepted for publication in the proceedings of the 46th International Conference on Software Engineering (ICSE ’24 - Research Track). The paper proposes a novel systematic approach for automated testing of hyperproperties, a recent formalism introduced to define correctness requirements that involve relations between multiple program executions. To empirically validate the proposed approach, the paper provides HyperFuzz and HyperEvo, two test generators targeting the Non-Interference security requirement, that rely respectively on fuzzing and search algorithms. The empirical evaluation has been carried out in the form of a correlation study and a performance comparison with the dynamic taint analysis tool Phosphor, on a set of benchmark Java programs.

The goal of the artifact is to facilitate the replication of the results presented in such empirical evaluation, as well as to explain how to use the tools HyperFuzz and HyperEvo. The artifact includes the usage documentation of the two tools, their executables (together with third party libraries and the executable of Phosphor), the dataset of Java programs used in the evaluation, and a detailed description of how to replicate the experiments.

Within the ICSE ’24 Artifact Evaluation Track, we claim the Available and Reusable badges, and we expect the reviewers to be familiar with Unix command-line shells (e.g., Bash). The artifact has been tested on a Linux machine (based on Ubuntu 20.04.3 LTS with Linux 5.11.0-27 kernel), and it is publicly available on Zenodo. We also provide a virtual machine image (in the Open Virtualization Format), to replicate experiments in a clean environment with all dependencies already installed.