Accepted Papers

Call for Artifacts

SEAMS 2023 continues to encourage its community members to build dedicated artifacts that support driving, communicating, comparing, and evaluating their research on software engineering for adaptive and self-managing systems. In this spirit, the SEAMS 2023 artifacts track exists to review, promote, share, and catalog research artifacts that bring value to the community.

NOTE: While this call is about dedicated artifacts, we also encourage authors to submit artifacts used for their long research papers to SEAMS 2023. Although the two submissions are independent (i.e., the artifact has its own artifact paper), if both are accepted, the badges awarded to the artifact paper will be awarded to the research paper too.

Types of Artifacts

According to ACM’s "Artifact Review and Badging (Version 1.1)” policy, an artifact is “a digital object that was either created by the authors to be used as part of the study or generated by the experiment itself. For example, artifacts can be software systems, scripts used to run experiments, input datasets, raw data collected in the experiment, or scripts used to analyze results.” A formal review of such artifacts promotes artifacts of high quality that promotes reproducibility and replicability of research results and that drive the research of the whole SEAMS community. Hence, artifacts of interest for SEAMS include but are not limited to:

• Testbeds / Exemplars, which are implementations or detailed specifications of systems that pose and highlight fundamental or characteristic challenges in this community, and that self-adaptive systems should address.
• Datasets, which are data (e.g., logging data, sensor data, system traces, survey raw data) that can be used to develop, evaluate, and compare self-adaptation approaches.
• Frameworks, which are tools and services illustrating and implementing self-adaptation techniques or algorithms that are potentially useful in different contexts and that other researchers could use and customize to specific contexts.

Thus, frameworks primarily support developing self-adaptation approaches whereas testbeds / exemplars and datasets particularly support evaluating and comparing different approaches. However, this list is not exhaustive. If your proposed artifact is not on this list, please email the Artifacts Chairs before submitting.

Quality of Artifacts

According to the ACM’s "Artifact Review and Badging (Version 1.1)” policy, SEAMS aims for artifacts that are available and reusable so that other researchers can access and built upon the artifacts to drive research in the overall SEAMS community. Thus, artifacts have to be made permanently available—latest after acceptance and before publication of the artifact—using publisher repositories (ACM or IEEE), institutional repositories, or open (commercial) repositories (e.g., figshare, Zenodo, or GitHub/Zenodo) that provide permanent and unique identifiers for the artifacts. Personal web pages are not acceptable for this purpose. Moreover, artifacts should be reusable by other researchers. To facilitate reuse, they should be carefully documented, well-structured, complete, exercisable, as well as appropriately verified and validated (e.g., use of the artifact in a study).

Important Dates

• Abstract deadline (firm): 25 January 2023
• Papers submission due (firm): 1 February 2023
• Notification of decision: 8 March 2023
• Camera-ready version due: 24 March 2023

For detailed information on how to submit artifacts to SEAMS 2023, please see the Submission tab.

Review Process and Selection Criteria

All submitted artifacts will be reviewed by at least three members of the program committee. Each artifact will be evaluated in relation to the expectations set by the artifact paper. In addition to just running the artifact, the evaluators will read the paper and may try to tweak provided inputs and create new ones, to test the limits of the system.

Artifacts will be evaluated using the following criteria:

• Community value: Does the artifact bring value to the SEAMS community? Is this value clearly explained in the paper? Can the artifact be readily used by other researchers?
• Insightfulness: Does the artifact address or identify a gap in previous work?
• Timeliness: Does the artifact address a problem that is timely?
• Usability: Is it complete and easy to understand? Is it carefully documented and well-structured? Is it accompanied by tutorial notes? and other documentation? Is it exercisable? If the artifact is executable, is it easy to download, install, and execute? Has it been validated? Is it functional? Is it reusable?

SUBMISSION LINK: Abstracts and papers must be submitted via EasyChair: https://easychair.org/conferences/?conf=seams2023

Submission includes an artifact paper and the artifact itself. The paper should be submitted via EasyChair (link to appear). The paper should provide a link from where the reviewers can access and download the artifact (it does not need to be a link to the permanent repository at this stage). Artifacts must not have been previously published or concurrently submitted elsewhere.

Artifact Paper

An artifact paper is of max 6 pages + 1 page references. It should include a synopsis or description of the problem that is being addressed, a description of the context(s) in which the artifact would be useful, a list of the challenges that it poses for self-adaptation, the link to the artifact, and examples of its use in at least one area of self-adaptive systems. Accepted artifact papers will be included in the proceedings, and authors will be given an opportunity to present their artifact at SEAMS 2023.

Artifact papers must conform to the IEEE formatting instructions IEEE Conference Proceedings Formatting Guidelines (title in 24pt font and full text in 10pt type, LaTeX users must use \documentclass[10pt,conference]{IEEEtran} without including the compsoc or compsocconf options).

There will be a best artifact award recognizing the work of authors who contribute the most useful artifact to the community. Moreover, we will do our best to work with the IEEE Xplore and ACM Portal administrator to add badges to the electronic versions of the authors’ paper(s).

Artifact

Authors must perform the following steps to submit an artifact:

Packaging the Artifact

When packaging your artifact, it is important to keep in mind: a) how accessible the artifact is to other researchers, and b) the fact that the artifact evaluators have very limited time to assess each artifact. The setup for your artifact should take less than 30 minutes or it is unlikely to be endorsed simply because the committee will not have sufficient time to evaluate it. If you envision difficulties, please provide your artifact with a working environment in a VirtualBox VM image or a Docker container image so that the artifact can be run and exercised. Otherwise, the artifact can be packaged in a single archive file (zip or tar.gz) or its code base can be provided by a public repository. In either case, the artifact should be exercisable and appropriately validated.

Documenting the Artifact

To facilitate reuse, an artifact should be complete and carefully documented. Therefore, it must:

• be self-contained, that is, it contains the artifact itself, which may include source code, executables, data, a virtual machine image, documents and other content deemed relevant by the authors. Please use open formats for documents (e.g., csv for data). Publicly available external tools or libraries used to exercise and use the artifact do not have to be included in the artifact;
• include documentation that describes the artifact. It is acceptable to refer to the artifact paper for a more detailed description of the artifact. The entry point to the documentation should be easy to identify (e.g., README.md or index.html file in the top-level directory of the artifact). The documentation should contain:
• a Getting Started section that should stress the key elements of your artifact and that should enable the reviewers to run, execute or analyze your artifact without any technical difficulty. In this context, requirements and side effects of running the artifact should be discussed.
• step-by-step instructions on how to download, install, run, and “play” the artifact, and how to check the results of the execution. These instructions should both show how you propose to evaluate your artifact, and be useful for a new user of your artifact to get started.
• where appropriate, descriptions of and links to files (included in the archive or generated by executing the artifact) that represent expected outputs (e.g., the log files expected to be generated by the artifact on the given inputs).
• if applicable, descriptions of how the artifact can be customized and extended to be reused in a different research context of self-adaptive systems.
• include a license file or statement describing the distribution rights. Note that a reusable artifact requires some kind of open source license.

Optionally, the authors are encouraged to include in the documentation a link to a short video (YouTube, max. 5 minutes) demonstrating the artifact.

Making the Artifact Available

Regardless of the packaging, the artifact should be made available to the artifact reviewers through a link to a public repository (e.g., GitHub) or to a single archive file.

Artifacts accepted to the SEAMS 2023 program have to be made permanently available to the public by the time the camera ready version of the paper is due. This must be done using an archival repository such as publisher repositories at ACM or IEEE, institutional repositories, or open (commercial) repositories such as figshare and Zenodo. For instance, Zenodo supports archiving snapshots of GitHub repositories and provides DOIs for such snapshots.