Write a Blog >>
ICSE 2023
Sun 14 - Sat 20 May 2023 Melbourne, Australia

Call for Papers

ICSE is the premier forum for presenting and discussing the most recent and significant technical research contributions in the field of Software Engineering. In the technical track, we invite high quality submissions of technical research papers describing original and unpublished results of software engineering research.

Please note the following important changes for 2023:

  1. As previous ICSE editions, we apply an open science policy. This year, we will ask a reviewer to perform a lightweight check on the shared artifacts (see below).
  2. There will be a second response period for a subset of the submitted papers. Please check our calendar of important dates.

Also, we will continue the initiative started in 2022, giving emphasis to the significance of the research contributions (see review criteria).

Research of Interest

ICSE welcomes submissions addressing topics across the full spectrum of Software Engineering, being inclusive of quantitative, qualitative, and mixed-methods research.

Topics of interest include:

● API design and evolution

● Apps and app store analysis

● Autonomic systems and self adaptation

● Configuration management

● Crowd-based software engineering

● Debugging and fault localization

● Design for quality, incl. privacy and security by design

● Distributed and collaborative software engineering

● Diversity, inclusion, fairness of software

● Embedded and cyber-physical systems

● Ethics in software engineering

● Evolution and maintenance

● Feedback, user, and requirements management

● Formal methods

● Green and sustainable technologies

● Human aspects of software engineering

● Human-computer interaction

● Legal aspects of software engineering

● Machine learning with and for SE

● Mining software repositories

● Model checking

● Modeling and model-driven engineering

● Parallel and distributed systems

● Performance analysis and testing

● Privacy and security

● Program analysis

● Program comprehension

● Program repair

● Program synthesis

● Programming languages

● Recommender systems

● Refactoring

● Release engineering and DevOps

● Reliability and safety

● Requirements engineering

● Reverse engineering

● SE for machine learning

● Search-based software engineering

● Software architecture and product design

● Software economics

● Software ecosystems

● Software metrics and prediction models

● Software processes

● Software reuse

● Software services and cloud-based systems

● Software testing

● Software traceability

● Software visualization

● Variability and product lines

Review Criteria

Each paper submitted to the Technical Track will be evaluated based on the following criteria:

Soundness: The extent to which the paper’s contributions and/or innovations address its research questions and are supported by rigorous application of appropriate research methods

Significance: The extent to which the paper’s contributions can impact the field of software engineering, and under which assumptions (if any)

Novelty: The extent to which the contributions are sufficiently original with respect to the state-of-the-art

Verifiability and Transparency: The extent to which the paper includes sufficient information to understand how an innovation works; to understand how data was obtained, analyzed, and interpreted; and how the paper supports independent verification or replication of the paper’s claimed contributions. New this year: the artifacts attached to the paper (or hyperlinked to it) will be checked by at least one reviewer.

Presentation: The extent to which the paper’s quality of writing meets the high standards of ICSE, including clear descriptions, as well as adequate use of the English language, absence of major ambiguity, clearly readable figures and tables, and adherence to the formatting instructions provided below.

Reviewers will carefully consider all of these criteria during the review process, and authors should take great care in clearly addressing them all. The paper should clearly explain the claimed contributions, and how they are sound, significant, novel, and verifiable, as described above.

Each paper will be handled by an area chair. The role of the area chairs is to ensure a reviewing consistency among papers submitted within the same area of research. For this reason, we will ask the authors, upon submission, to identify up to two main areas to which the paper belongs, among the following ones:

● Artificial Intelligence and Software Engineering

● Analysis and Testing

● Dependability

● Requirements, modeling, and design

● Social Aspects

● Software Evolution

● Software Analytics

Program chairs will ultimately assign a paper to an area chair, considering the authors’ selection, the paper’s content, and (if applicable) keeping into account possible conflicts of interest.

For more information on how the ICSE PC will interpret and use these criteria in the paper evaluation process, please see the ICSE 2023 Review Process and Guidelines.

How to Submit

Submissions must conform to the IEEE conference proceedings template, specified in the IEEE Conference Proceedings Formatting Guidelines (title in 24pt font and full text in 10pt type, LaTeX users must use \documentclass[10pt,conference]{IEEEtran} without including the compsoc or compsocconf options).

● All submissions must not exceed 10 pages for the main text, inclusive of all figures, tables, appendices, etc. Two more pages containing only references are permitted. All submissions must be in PDF. Accepted papers will be allowed one extra page for the main text of the camera-ready version.

● Submissions must strictly conform to the IEEE conference proceedings formatting instructions specified above . Alterations of spacing, font size, and other changes that deviate from the instructions may result in desk rejection without further review.

● By submitting to the ICSE Technical Track, authors acknowledge that they are aware of and agree to be bound by the ACM Policy and Procedures on Plagiarism and the IEEE Plagiarism FAQ . In particular, papers submitted to ICSE 2023 must not have been published elsewhere and must not be under review or submitted for review elsewhere whilst under consideration for ICSE 2023 . Contravention of this concurrent submission policy will be deemed a serious breach of scientific ethics, and appropriate action will be taken in all such cases. To check for double submission and plagiarism issues, the chairs reserve the right to (1) share the list of submissions with the PC Chairs of other conferences with overlapping review periods and (2) use external plagiarism detection software, under contract to the ACM or IEEE, to detect violations of these policies.

● The ICSE 2023 Technical Track will employ a double-anonymous review process. Thus, no submission may reveal its authors’ identities. The authors must make every effort to honor the double-anonymous review process. In particular:

Authors’ names must be omitted from the submission.

○ All references to the author’s prior work should be in the third person.

○ While authors have the right to upload preprints on ArXiV or similar sites, they should avoid specifying that the manuscript was submitted to ICSE 2023.

○ During review, authors should not publicly use the submission title.

● Further advice, guidance, and explanation about the double-anonymous review process can be found in the Q&A page.

● By submitting to the ICSE Technical Track, authors acknowledge that they conform to the authorship policy of the ACM , and the authorship policy of the IEEE .

Submissions to the Technical Track that meet the above requirements can be made via the Technical Track submission site ( https://icse2023.hotcrp.com) by the submission deadline. Any submission that does not comply with these requirements may be desk rejected without further review.

We encourage the authors to upload their paper info early (and can submit the PDF later) to properly enter conflicts for double-anonymous reviewing. Authors are encouraged to try out the experimental SIGSOFT Submission Checker to detect violations to the formatting and double anonymous guidelines. (Mind that the tool is based on heuristics. Therefore it may miss violations, and it can raise false alarms. The requirements listed in this call for papers take precedence over the results of the tool when deciding whether a paper meets the submission guidelines.)

Open Science Policy

The research track of ICSE 2023 is governed by the ICSE 2023 Open Science policies. In summary, the steering principle is that all research results should be accessible to the public and, if possible, empirical studies should be reproducible. In particular, we actively support the adoption of open data and open source principles and encourage all contributing authors to disclose (anonymized and curated) data to increase reproducibility and replicability. Note that sharing research data is not mandatory for submission or acceptance. However, sharing is expected to be the default, and non-sharing needs to be justified. We recognize that reproducibility or replicability is not a goal in qualitative research and that, similar to industrial studies, qualitative studies often face challenges in sharing research data. For guidelines on how to report qualitative research to ensure the assessment of the reliability and credibility of research results, see the Q&A page .

Upon submission to the research track, authors are asked

● to make their data available to the program committee (via upload of supplemental material or a link to an anonymous repository) – and provide instructions on how to access this data in the paper; or

● to include in the paper an explanation as to why this is not possible or desirable; and

● to indicate if they intend to make their data publicly available upon acceptance.

NEW! This year, at least one reviewer will check whether the enclosed package contains what is declared in the paper. This quality check process will be very lightweight, and the main aim is to ensure that authors do not submit (partially) empty packages.

Supplementary material can be uploaded via the HotCrp site or anonymously linked from the paper submission. Although PC members are not required to look at this material, we strongly encourage authors to use supplementary material to provide access to anonymized data, whenever possible. Authors are asked to carefully review any supplementary material to ensure it conforms to the double-anonymous policy (described above). For example, code and data repositories may be exported to remove version control history, scrubbed of names in comments and metadata, and anonymously uploaded to a sharing site to support review. One resource that may be helpful in accomplishing this task is this blog post .

Upon acceptance, authors have the possibility to separately submit their supplementary material to the ICSE 2023 Artifact Evaluation track, for recognition of artifacts that are reusable, available, replicated or reproduced.

Mentoring for Prospective Authors

We are organizing an “Ask me Anything” (AMA) Session on Best practices for a successful ICSE paper in June 2022 for prospective authors to learn from the 2022 ICSE PC co-chairs, Andreas Zeller and Daniela Damian.

Details about this event will be announced a couple of months before the event will happen.

Author Response Periods

ICSE 2023 will offer response periods for authors. In such periods, the authors will have the opportunity to inspect the reviews, and to answer specific questions raised by the program committee.

ICSE 2023 will foresee two response periods. The first one is scheduled after all reviews have been completed, and serves to inform the subsequent decision making process. Authors will be able to see the full reviews, including the reviewer scores as part of the author response process. The second one follows a period of review discussion, and will involve only some papers, for which additional reviews were needed, or where, during the discussion, further questions for the authors emerged.

Note that:

● Participating in the response period is optional for the authors, i.e., omitting a rebuttal does not necessarily prejudice the paper’s outcome.

● Not being invited to participate in the second response period does not mean the paper has been rejected (nor accepted), but just that reviewers felt that a second response period was unnecessary.

Withdrawing a Paper

Authors can withdraw their paper at any moment until the final decision has been made, through the paper submission system. Resubmitting the paper to another venue before the final decision has been made without withdrawing from ICSE 2023 first is considered a violation of the concurrent submission policy, and will lead to automatic rejection from ICSE 2023 as well as any other venue adhering to this policy.

Important Dates

● Technical Track Submissions Deadline: September 1, 2022

● Technical Track Author First Response Period (all papers): November 14–16, 2022

● Technical Track Author Second Response Period (some papers): November 28–29, 2022

● Technical Track Acceptance Notification: December 9, 2022

● Technical Track Camera Ready: TBA

Conference Attendance Expectation

If a submission is accepted, at least one author of the paper is required to register for ICSE 2023 and present the paper. [We will add more info on this as soon as the ICSE 2023 format is finalized.]

ICSE 2023 open science policy

As a conference sponsored by ACM SIGSOFT, ICSE 2023 adheres to open science policies as adopted by SIGSOFT. The text below is based on v0.9.9 of these policies.

Open science policies

Openness in science is key to fostering progress via transparency and availability of all outputs produced at each investigative steps. Transparency and availability of research outputs allow better reproducibility, replicability of quantitative studies and recoverability of qualitative studies. Open science builds the core for excellence in evidence-based research.

As an internationally renowned forum for researchers, practitioners, and educators to present and discuss the most recent innovations, trends, experiences, and challenges in the field of software engineering, ICSE 2023 (continuing the tradition of previous editions) actively supports setting standards for how we conduct this kind of research.

To this end, we have explicitly committed ourselves to foster openness to our research outcomes. In particular, we support the adoption of open data and open source principles. We encourage all contributing authors to disclose the (anonymized and curated) data to increase reproducibility, replicability, and/or recoverability of the studies.

Principles

Research output should be publicly and freely accessible by anyone, permanently.

Artifacts related to a study (which include, but are not limited to, raw and transformed data, extended proofs, appendices, analysis scripts, software, virtual machines and containers, and qualitative codebooks) and the paper itself should, in principle, be made available on the Internet:

  1. without any barrier (e.g., paywalls, registration forms, request mechanisms),
  2. under a proper open license that specifies purposes for re-use and repurposing, properly archived and preserved ,
  3. provided that there are no ethical, legal, technical, economic, or sensible barriers preventing the disclosure.

Open artifacts

Fostering artifacts as open data and open source should be done as:

● Archived on preserved digital repositories such aszenodo.org,figshare.com,www.softwareheritage.org, osf.io, or institutional repositories. GitHub, GitLab, and similar services for version control systems do not offer properly archived and preserved data. Personal or institutional websites, consumer cloud storage such as Dropbox, or services such as Academia.edu and Researchgate.net do not provide properly archived and preserved data.

● Released under a proper open data license such as the CC0 dedication or the CC-BY 4.0 license when publishing the data.

● Software can be released under an open source license.

● Different open licenses, if mandated by institutions or regulations, are also permitted.

● We encourage authors to make artifacts available upon submission (either privately or publicly) and upon acceptance (publicly).

Supporting statement

We ask authors to provide a supporting statement on the data availability (or lack thereof) in their submitted papers in a section named Data Availability after the Conclusion section.

Authors who cannot disclose data for the reasons stated in the principles of the policies should provide a short statement in their submitted papers in a section named Data Availability after the Conclusion section.

Please note that the success of the open science initiative depends on the willingness (and possibilities) of authors to disclose their data and that all submissions will undergo the same review process independent of whether they disclose their analysis code or data.

HOWTOs

A step-by-step approach to disclosing artifacts for (doubly-anonymous) peer review and make it open data upon acceptance is available online .

A step-by-step approach to automatically archive a GitHub repository to Zenodo.org is available at https://guides.github.com/activities/citable-code/ .

A step-by-step approach to automatically archive a GitHub repository to figshare.com is available at https://knowledge.figshare.com/articles/item/how-to-connect-figshare-with-your-github-account .

A proposal for artifact evaluation by SIGSOFT is available at https://github.com/acmsigsoft/artifact-evaluation .

A proposal for open science in software engineering, including explanations for structuring an open artifact, is available at https://arxiv.org/abs/1904.06499 .

Open Access

We encourage ICSE 2023 authors to self-archive their pre- and post-prints in open and preserved repositories. Self-archiving is legal and allowed by most publishers (granted in the copyright transfer agreement), and it will enable anybody in the world to reach papers barrier-free.

Upon acceptance to ICSE 2023, we encourage authors to revise their article according to the peers’ comments, generate a PDF version of it (post-print), and submit it to arXiv.org or their institutional repository.

Unless authors are willing to pay the publisher for open access of their published papers (gold open access), they should pick the “arXiv.org - Non-exclusive license to distribute” license https://arxiv.org/licenses/nonexclusive-distrib/1.0/license.html when submitting to arXiv.

Authors should avoid a Creative Commons license for their preprints, in any repository, if the published papers are not open access. More infos available here: https://avandeursen.com/2016/11/06/green-open-access-faq/#creative-commons .

Note: Authors are not allowed to self-archive the PDF of the published article as typeset by the publisher (a.k.a. “publisher proof,” “published paper,” “the digital library version”).

HOWTOs

A comprehensive FAQ for open access and self-archiving is available at https://avandeursen.com/2016/11/06/green-open-access-faq/ .

Instructions for reviewers

ICSE 2023 has adopted an open science stance and introduced guidelines for authors (available at https://conf.researchr.org/track/icse-2023/icse-2023-open-science-policies ). The policies invite authors to provide all research artifacts for peer review, self-archive their pre- and post-prints, and archive artifacts as open data upon acceptance. We kindly ask you to pay attention to the following, while reviewing:

  1. All open science steps are optional for authors and reviewers. You are invited, but not required, to inspect the provided artifacts as part of your review efforts.
  2. All reasons for partial disclosure of data (or lack thereof) should be trusted.
  3. Submissions have to undergo the same review process independent of whether they disclose their analysis code or data. You are invited to complain in your review of any absence of data, but please do not let it influence your review of submissions. You are free to welcome further disclosure of data and help authors in doing so, with your review.
  4. Open science is challenging for qualitative studies. Please be welcoming of qualitative studies which open their artifacts even in a limited way. Furthermore, when evaluating artifacts from qualitative studies that was made available for peer review, please keep in mind that authors of qualitative studies might have underlying ontological and epistemological stances that differ from those of authors of quantitative studies. Concepts such as replicability and reproducibility might apply partially or not apply at all with qualitative studies.
  5. Providing research artifacts might introduce issues with doubly-anonymous reviews. We ask you not to actively hunt the identity of authors, especially in case they self-archived a preprint of their submission. License

These open science policies are based on open science policies by Daniel Graziotin, licensed under CC0 1.0.

Please note that the processes described in this document are not used by all tracks. Check in the call for papers whether

  • sharing data is expected, and how.
  • double-anonymous review is used or not.

Empirical Studies and Sharing of Data

I am doing research with industry. What if I cannot share data from my research?

We absolutely welcome research with industry, as it often conveys important lessons about software engineering in practice – and we perfectly understand that industry data may be subject to confidentiality issues or legal requirements. If you cannot share data, please state the reason in the submission form and the paper; a typical wording would be "The raw data obtained in this study cannot be shared because of confidentiality agreements". Having said that, even sharing a subset of your data (for instance, the data used for figures and tables in the paper, an anonymized subset, or one that aggregates over the entire dataset), analysis procedures or scripts, would be useful.

I am doing user studies. What if I cannot share data from my empirical study?

We absolutely welcome user studies! However, we also perfectly understand that sharing raw data can be subject to constraints such as privacy issues. If you cannot share data, please state the reason in the submission form and the paper; a typical wording would be "The raw data obtained in this study cannot be shared because of privacy issues". Having said that, even sharing a subset of your data (for instance, the data used for figures and tables in the paper, an anonymized subset, or one that aggregates over the entire dataset), analysis procedures or scripts, would be useful.

I am doing qualitative research. What information should I include to help reviewers assess my research results and the readers use my results?

Best practices for addressing the reliability and credibility of qualitative research suggest providing detailed arguments and rationale for qualitative approaches, procedures and analyses. Therefore, authors are advised to provide as much transparency as possible into these details of their study. For example, clearly explain details and decisions such as 1) context of study, 2) the participant-selection process and the theoretical basis for selecting those participants, 3) collection of data or evidence from participants, and 4) data analysis methods, e.g. justify their choice theoretically and how they relate to the original research questions, and make explicit how the themes and concepts were identified from the data. Further, provide sufficient detail to bridge the gap between the interpretation of findings presented and the collected evidence by, for example, numbering quotations and labeling sources. Similar to replicability in quantitative research, the transparency aims to ensure a study’s methods are available for inspection and interpretation. However, replicability or repeatability is not the goal, as qualitative methods are inherently interpretive and emphasize context. As a consequence, reporting qualitative research might require more space in the paper; authors should consider providing enough evidence for their claims while being mindful with the use of space.

Finally, when qualitative data is counted and used for quantitative methods, authors should report the technique and results in assessing rigour in data analysis procedures, such as inter-reliability tests or triangulation over different data sources or methods—, and justify how they achieved rigour if no such methods were used.

I can make my data set / my tool available, but it may reveal my identity. What should I do?

See this question under "double-anonymous submissions", below.

Double-Anonymous Submissions

Why double-anonymous?

There are many reasons for a submission track to employ a double-anonymous review process – not the least being the considerable number of requests to do so from the community. For more information on motivations for double-anonymous reviewing, see Claire Le Goues’s very well-argued, referenced and evidenced blog posting in favor of double-anonymous review processes for Software Engineering conferences . See also a list of double-anonymous resources from Robert Feldt, as well as a more formal study of the subject by Moritz Beller​​ and Alberto Bacchelli​​.

How can I prepare my paper for double-anonymous reviewing?

You must make every reasonable effort to honor the double-anonymous review process, but you do not need to guarantee that your identity is undiscoverable. The double-anonymous aspect of the review process is not to set up an adversarial identity-discovery process. Essentially, the guiding principle should be to maximize the number of people who could plausibly be authors, subject to the constraint that no change is made to any technical details of the work. Therefore, you should ensure that the reviewers are able to read and review your paper without needing to know who any of the authors are. Specifically, this involves at least adhering to the following three points:

  1. Omit all authors’ names from the title page.
  2. Refer to your own work in the third person. You should not change the names of your own tools, approaches or systems, since this would clearly compromise the review process; it would also violate the constraint that “no change is made to any technical details of the work”. Instead, refer to the authorship or provenance of tools, approaches or systems in the third person, so that it is credible that another author could have written your paper.
  3. Do not rely on non-anonymous supplementary material (your web site, your github repository, a youTube channel, a companion technical report or thesis) in the paper or in the rebuttal submitted during the clarification period. Supplementary information might result in revealing author identities.

Here is some excellent advice on anonymization from ACM .

I previously published an earlier version of this work in a venue that doesn’t have double-anonymous. What should I do about acknowledging that previous work?

If the work you are submitting for review has previously been published in a non-peer-reviewed venue (e.g., arXiv.org, or a departmental tech report), there is no need to cite it, because work that has not been refereed is not truly part of the scientific literature.

If the previous work is published in a peer-reviewed venue, then it should be cited, but in the third person so that it is not revealed that the cited work and the submitted paper share one or more authors.

Our submission makes use of work from a PhD or master’s thesis, dissertation, or report which has been published. Citing the dissertation might compromise anonymity. What should we do?

It’s perfectly OK to publish work arising from a PhD or master’s degree, and there’s no need to cite it in an ICSE submission that is undergoing double-anonymous review because prior dissertation publication does not compromise novelty. In the final post-review, camera-ready version of the paper, please do cite the dissertation to acknowledge its contribution, but in any submission to an ICSE track employing a double-anonymous review process, please refrain from citing the dissertation, to increase anonymity.

You need not worry whether or not the dissertation has appeared. Your job is to ensure that your submission is readable and reviewable, without the reviewers needing to know the identities of the submission’s authors. You do not need to make it impossible for the reviewers to discover the authors’ identities. The referees will be trying hard not to discover the authors’ identity, so they will likely not be searching the web to check whether there is a dissertation related to this work.

What if we want to cite some unpublished work of our own (as motivation for example)?

If the unpublished paper is an earlier version of the paper you want to submit to ICSE and is currently under review, then you have to wait until your earlier version is through its review process before you can build on it with further submissions (this would be considered double-submission and violates ACM plagiarism policy and procedures ). Otherwise, if the unpublished work is not an earlier version of the proposed ICSE submission, then you should simply make it available on a website, for example, and cite it in the third person to preserve anonymity, as you are doing with others of your works. If your work is a tool, a data set, or some other resource, see the question on ‘resources already made available’, above.

Can I disseminate a non-anonymized version of my submitted work by discussing it with colleagues, giving talks, publishing it at ArXiV, etc.?

You can discuss and present your work that is under submission at small meetings (e.g., job talks, visits to research labs, a Dagstuhl or Shonan meeting), but you should avoid broadly advertising it in a way that reaches the reviewers even if they are not searching for it. Whenever possible, please avoid posting your manuscript on public archives (e.g, ArXiV) before or during the submission period. Would you still prefer to do so, carefully avoid adding to the manuscript any reference to ICSE 2023 (E.g., using footnotes saying “Submitted to ICSE 2023”).

I can make my data set / my tool available, but it may reveal my identity. What should I do?

Please make an effort to anonymize your data set / your tool such that it does not reveal your identity. If that is impossible, place a warning next to the link that this may reveal your identity.