Artifact EvaluationICST 2026

Call for Artifacts

The Artifact Evaluation track at ICST provides a platform to promote, share, and catalog research artifacts of accepted software engineering papers. Authors of papers accepted to the Research track are invited to submit an artifact for evaluation. Papers will be given the IEEE Artifact Available and Artifact Reviewed badges if their artifacts meet the criteria described below. The ICST artifact evaluation track will use a single-anonymous review process. During the review period, PC/author discussions will be enabled to facilitate artifact assessment.

A key goal of us is to help authors make their artifacts available, reusable and functional. To this end, we strongly encourage authors to provide a clean image (e.g. with Docker) as part of their artifacts for any software components.

Important Dates

All dates below are in the AoE (Anywhere on Earth) timezone.

• 27 February 2026: Artifact submission deadline. Update: Extended to 6 March 2026 AoE
• 7 March - 23 March 2026: Review period.
• 27 March 2026: Author notification.

Badge Criteria

Artifact Available

A paper will be awarded the IEEE “Artifact Available” badge if the following two criteria are fulfilled:

1. Its artifact is placed in a publicly accessible archival repository, and a DOI or link to this persistent repository is provided.
2. Its artifact is properly documented, at the minimum with a README file explaining the meaning of each file and its content.

Artifact Reviewed

A paper will be awarded the IEEE “Artifact Reviewed” badge if its artifact is documented, consistent, complete, exercisable, and includes appropriate evidence of verification and validation. Moreover, the documentation and structure of the artifact should be good enough so that reuse and repurposing are facilitated. The terms are clarified below:

• Documented: At a minimum, an inventory of artifacts is included, and sufficient description is provided to enable the artifacts to be exercised.
• Consistent: The artifacts are relevant to the associated paper, and contribute in some inherent way to the generation of its main results.
• Complete: To the extent possible, all components relevant to the paper in question are included. (Proprietary artifacts need not be included. If they are required to exercise the package, then this should be documented, along with instructions on how to obtain them. Proxies for proprietary data should be included so as to demonstrate the analysis.)
• Exercisable: Included scripts and/or software used to generate the results in the associated paper can be successfully executed, and included data can be accessed and appropriately manipulated.

Submission Instructions

Only authors of papers accepted to the 2026 Research track can submit candidate artifacts.

By the submission deadline, register your research artifact at the EasyChair site by submitting a 2 pages (max) abstract in PDF format describing your artifact.

For the artifact available badge, authors must offer download information showing how reviewers can access and execute (if appropriate) their artifact.

Authors must perform the following steps to submit an artifact:

1. Prepare the artifact
2. Make the artifact available
3. Document the artifact
4. Submit the artifact

1. Prepare the artifact

Both executable and non-executable artifacts may be submitted.

Executable artifacts consist of a tool or software system. For these artifacts, authors should prepare an installation package so that the tool can be installed and run in the evaluator’s environment. Following the instructions below, provide enough associated instruction, code, and data such that an average CS professional could build, install, and run the code within a reasonable time-frame. If installation and configuration requires more than 30 minutes, the artifact is unlikely to be accepted on practical grounds, simply because the PC will not have sufficient time to evaluate it.

When preparing executable packages for submission, we recommend vetting the artifact on a clean machine to confirm that it can be setup in a reasonable time frame. We strongly encourage authors to consider using a Docker (or VirtualBox VM) image for this process. Besides providing a clean environment to assess the installation instructions, the resulting image can be submitted as part of the artifact to allow quick replication. In particular, if the artifact contains or requires the use of a special tool or any other non-trivial piece of software, the authors must provide a VirtualBox VM image or a Docker container image with a working environment containing the artifact and all the necessary tools.

Non-executable artifacts only contain data and documents that can be used with a simple text editor, a PDF viewer, or some other common tool (e.g., a spreadsheet program in its basic configuration). These artifacts can be submitted as a single, optionally compressed package file (e.g., a tar, zip, or tar.gz file).

2. Make the artifact available

The authors need to make the packaged artifact available so that the PC can access it.

Artifacts must be made available via an archival repository, such as Software Heritage (see their submission guide), which provides long-term availability of software source code. Other often used solutions, more focused on long-term data archival, include Figshare and Zenodo. Please note that platforms that do not guarantee long-term archival, which presently includes GitHub, generally do not qualify. However open source software on Github that has a history of regular updates over at least 5 years, that have formal releases that generate a DOI on a tool like Zenodo will be considered.

3. Document the artifact

The authors need to write and submit documentation explaining how to obtain, unpack, and use their artifact in detail. The artifact submission must only describe the technicalities of the artifacts and uses of the artifact that are not already described in the paper. Please provide (in the archival repository created above):

• A copy of the accepted paper in pdf format including the link to the archival repository.
• A LICENSE file describing the distribution rights. For submissions aiming for the Available badge, the license needs to ensure public availability. We recommend adopting an open source license for executable artifacts and open data license for non-executable artifacts.
• A README file (in Markdown, plain text, or PDF format) that describes the artifact with all appropriate sections from the following:
• Purpose: a brief description of what the artifact does.
  • Include a list of badge(s) the authors are applying for as well as the reasons why the authors believe that the artifact deserves that badge(s).
• Provenance: where the artifact can be obtained, preferably with a link to the paper’s preprint if publicly available.
• Data (for artifacts which focus on data or include a nontrivial dataset): cover aspects related to understanding the context, data provenance, ethical and legal statements (as long as relevant), and storage requirements.
• Setup (for executable artifacts): provide clear instructions for how to prepare the artifact for execution. This includes:
  • Hardware: performance, storage, and device-type (e.g. GPUs) requirements.
  • Software: Docker or VM requirements, or operating system & package dependencies if not provided as a container or VM. Providing a Dockerfile or image, or at least confirming the tool’s installation in a container is strongly encouraged. Any deviation from standard environments needs to be reasonably justified.
• Usage (for executable artifacts): provide clear instructions for how to repeat/replicate/reproduce the main results presented in the paper. Include both:
  • A basic usage example or a method to test the installation. For instance, it may describe what command to run and what output to expect to confirm that the code is installed and operational.
  • Detailed commands to replicate the major results from the paper.

4. Submit the artifact

Overview

1. Access the ICST EasyChair submission URL: https://easychair.org/my2/conference?conf=icst2026
2. Click the “make a new submission” link, then “Artifact Evaluation” link.
3. Provide the Author / Paper title & abstract / Keywords / and Artifact abstract, as described below.

Submission Details

By the abstract submission deadline (see important dates), register your research artifact at the EasyChair site by submitting a 2-page abstract describing your artifact. The abstract should include the paper title, the purpose of the research artifact, the badge(s) you are claiming, the technology skills assumed by the reviewer evaluating the artifact, and and details on where to access the artifact. Please also mention if running your artifact requires any specific Operating Systems or other, unusual environments.

The PC may contact the authors, via the submission system, during the review period to request clarifications on the basic installation and start-up procedures or to resolve simple installation problems. Reviewers will be encouraged to attempt to execute submitted software artifacts early on, to minimize the time spent iterating on making the artifact functional and in turn provide enough time to ensure that all artifacts can be made reusable.

The most up-to-date information will be provided on this website: https://conf.researchr.org/track/icst-2026/icst-2026-artifact-evaluation.