Openness in science is key to fostering progress via transparency, reproducibility and replicability. While open access and open data are two fundamental pillars in open science, it is open data that builds the core for excellence in evidence-based research. As ESEM is the leading conference for evidence-based research methodologies in software engineering, we decided to actively support setting standards for how we conduct this kind of research.
To this end, we have explicitly committed ourselves to take the first step in fostering openness of our research outcomes. The steering principle is that all research output should be accessible to the public and that empirical studies should be reproducible. In particular, we actively support to adopt open data and open source principles and encourage all contributing authors to disclose the (anonymized and curated) empirical data to increase reproducibility and replicability.
Open Data and Open Source
Fostering open data should be done
- By archiving data on preserved archives such as zenodo.org and figshare.com, so that the data will receive a DOI and become citable. Figshare has options to mark the uploads as privately shared, for single and double blind peer review.
- Using the CC0 dedication (or the CC-BY 4.0 license) when publishing the data (automatic when using zenodo.org and figshare.com), as explained here.
- Please, follow these guidelines by Daniel Graziotin on how to submit anonymized resources to Zenodo and Figshare, and then turn them into open data upon acceptance of your paper.
Please note that the success of the open science initiative depends on the willingness (and possibilities) of authors to disclose their data and that all submissions will undergo the same review process independent of whether or not they disclose their analysis code or data. We encourage authors to make data available upon submission (either privately or publicly) and especially upon acceptance (publicly).
In any case, we ask authors who cannot disclose industrial or otherwise non-public data, for instance due to non-disclosure agreements, to please provide an explicit (short) statement in the paper.
Similarly, we encourage authors to make their research software accessible as open source and citable, using the same process.
We ask the authors to avoid putting the data/software on their own websites or systems like Dropbox, version control systems (SVN, Git), or service like Academia.edu and ResearchGate. Personal websites are prone to changes and errors, and more than 30% of them will not work in a 4 years period. Moreover, authors (or anybody else) should not have the ability to delete data once it is public. Using an archived repository only is simply a better approach.
Open Access
We encourage ESEM authors to self-archive their pre- and post-prints in open, preserved repositories. This is legal and allowed by all major publishers including ACM (granted in the copyright transfer agreement), and it lets anybody in the world reach your paper.
If the authors of your paper wish to do this, we recommend: Upon acceptance to ESEM, revise your article according to the peers comments, generate a PDF version of it (postprint), and submit it to arXiv.org, which supports article versioning.
Note that you are not allowed to self-archive the PDF of the published article (that is, the publisher proof or the Digital Library version) per policy of the ACM.
In case of questions, please approach the Open Science chair Fabio Calefato and Alexander Serebrenik.