Research TrackFORGE 2024

Welcome to the website of the special event of AI Foundation Models and Software Engineering (FORGE 2024) in ICSE 2024!

The AI Foundation Models and Software Engineering event aims to bring researchers, practitioners, and educators from the AI and Software Engineering community to solve the new challenges that we meet in the era of foundation models.

Foundation models (e.g., ChatGPT and Llama) have attracted great attention from both academia and industry. In Software Engineering, several studies showed that Large-Language Models (LLMs) achieved remarkable performance in various tasks, including code generation, testing, code review, and program repair. Recently, many LLM-based development tools have been released to improve software development and show great potential, for example, GitHub Copilot and Amazon CodeWhisperer.

FORGE aims to bring together researchers and practitioners to explore new frontiers in using foundation models on SE tasks, boost software development productivity, and improve code quality.

Forge 2024 will be held on Sunday, April 14, 2024, in Lisbon, Portugal.

Accepted Papers

Call for Papers

The special event of AI Foundation Models and Software Engineering (FORGE 2024) in ICSE 2024 aims to bring researchers, practitioners, and educators from the AI and Software Engineering community to solve the new challenges that we meet in the era of foundation models.

Foundation models (e.g., ChatGPT and Llama) have attracted great attention from both academia and industry. In Software Engineering, several studies showed that Large-Language Models (LLMs) achieved remarkable performance in various tasks, including code generation, testing, code review, and program repair. Recently, many LLM-based development tools have been released to improve software development and show great potential, for example, GitHub Copilot and Amazon CodeWhisperer.

FORGE aims to bring together researchers and practitioners to explore new frontiers in using foundation models on SE tasks, boost software development productivity, and improve code quality.

FORGE 2024 will be held on Sunday, April 14, 2024, in Lisbon, Portugal.

Topic of Interests

We solicit submissions describing original and unpublished results of theoretical, empirical, conceptual, and experimental software engineering research related to Software Engineering with Foundation Models. Topics of interest include but are not limited to:

1. FM for Requirement Engineering and Software Design
2. FM for Code Generation/Reuse
3. FM for Software Quality Assurance (e.g., including code review, analysis, testing, and debugging)
4. FM to support software evolution (e.g., refactoring, technical debt management)
5. FM for Software Security and Privacy
6. FM for AIOps
7. FM for software supply chain management, e.g., FM-based vulnerability identification, software composition analysis
8. LLM Agents for SE tasks, e.g., how to use various FMs (e.g., LangChain) to complete a SE task.
9. Prompt Engineering for Software Development
10. Legal Aspects of using FM

Awards

The best papers will be awarded with an ACM SIGSOFT Distinguished Paper Award at FORGE. A selection of the best papers will be invited to a Special Issue of Empirical Software Engineering (EMSE).

How to Submit

We accept both full and new idea papers:

1. Full Papers are expected to present new techniques, and/or provide research results, and/or report industry/open-source practice when applying foundation models for SE, and should be evaluated in a scientific way. Full Paper must not exceed 10 pages for the main text, inclusive of all figures, tables, appendices, etc. Two more pages containing only references are permitted.

2. New Idea Papers should present new ideas in the field, e.g., new directions or techniques that are not yet fully developed and/or evaluated, or visions that show the future of AI foundation models and SE. Accepted new idea papers will present their ideas in a short lightning talk. New Idea Paper must not exceed 4 pages for the main text, inclusive of all figures, tables, appendices, etc. Two more pages containing only references are permitted.

All submissions must be in PDF. The page limit is strict, and it will not be possible to purchase additional pages at any point in the process (including after acceptance).

Formatting instructions are available at https://www.acm.org/publications/proceedings-template for both LaTeX and Word users. LaTeX users must use the provided acmart.cls and ACM-Reference-Format.bst without modification, enable the conference format in the preamble of the document (i.e., \documentclass[sigconf,review]{acmart}), and use the ACM reference format for the bibliography (i.e., \bibliographystyle{ACM-Reference-Format}). The review option adds line numbers, thereby allowing referees to refer to specific lines in their comments.

Note, we use double-anonymous reviewing. Be sure to remove the list of authors from the submitted paper. If citing your own prior work, please do so in the third person to obscure the relationship you have with it. For advice, guidance, and explanation about the double-anonymous review process, see ICSE Research Track’s Q&A page.

By submitting your article to an ACM Publication, you are hereby acknowledging that you and your co-authors are subject to all ACM Publications Policies, including ACM’s new Publications Policy on Research Involving Human Participants and Subjects. Alleged violations of this policy or any ACM Publications Policy will be investigated by ACM and may result in a full retraction of your paper, in addition to other potential penalties, as per ACM Publications Policy.

Please ensure that you and your co-authors obtain an ORCID ID, so you can complete the publishing process for your accepted paper. ACM has been involved in ORCID from the start and we have recently made a commitment to collect ORCID IDs from all of our published authors. The collection process has started and will roll out as a requirement throughout 2022. We are committed to improving author discoverability, ensuring proper attribution, and contributing to ongoing community efforts around name normalization; your ORCID ID will help in these efforts.

All papers must be written in English.

All papers should be made accessible to people with disabilities. Some guidelines from the SIGACCESS community are available here: https://assets21.sigaccess.org/creating_accessible_pdfs.html.

Please submit your paper on HotCRP: https://forge-2024.hotcrp.com/

Review Criteria

Following the review criteria of ICSE 2024, each paper submitted to the FORGE 2024 will be evaluated based on the following criteria:

1. Novelty: The novelty and innovativeness of contributed solutions, problem formulations, methodologies, theories, and/or evaluations, i.e., the extent to which the paper is sufficiently original with respect to the state-of-the-art.

2. Rigor: The soundness, clarity, and depth of a technical or theoretical contribution, and the level of thoroughness and completeness of an evaluation.

3. Relevance: The significance and/or potential impact of the research to the field of software engineering.

4. Verifiability and Transparency: The extent to which the paper includes sufficient information to understand how an innovation works; to understand how data was obtained, analyzed, and interpreted; and how the paper supports independent verification or replication of the paper’s claimed contributions. Any artifacts attached to or linked from the paper may be checked by one reviewer.

5. Presentation: The clarity of the exposition in the paper.

Reviewers will carefully consider all of the above criteria during the review process, and authors should take great care in clearly addressing them all. The paper should clearly explain and justify the claimed contributions.