The SCORE Contest is aimed at promoting and fostering software engineering in universities worldwide. Student teams from all over the world participate in a competition for students from undergraduate to master’s level. Each team will develop a software project chosen from a list of projects proposed and sponsored by program committee members. The final deliverable is a report and an accompanying executable system. The evaluation will be based on the quality of all aspects of the software engineering process followed, as well as the resulting system.
In order to accommodate a wide range of academic calendars, the SCORE 2021 Contest will run from March 2020 to March 2021. Finalist teams will be invited to ICSE 2021 in Madrid, Spain.
Team Registration: Please, fill out this form by November 30, 2020.
Summary report submission: deadline for summary report submission via Easychair
Download the flyer:
Call for Participation
The sixth edition of the Student Contest on Software Engineering (SCORE) is part of the 43rd International Conference on Software Engineering (ICSE 2021).
SCORE is a worldwide competition for undergraduate and master’s level students. It emphasizes the engineering aspects of software development, not limited to programming. Student teams participating in the contest will complete a full-lifecycle software project on a selected project topic.
To take part in the competition, teams must register and follow the contest rules as outlined below. Upon completion of their work, teams will submit an initial project report (via Easychair; then, qualifying teams will submit full software products covering the whole software development process. After a careful evaluation carried out by the SCORE Program Committee, a limited number (about 3 to 5 – the exact number will be advertised later) of finalist teams will be invited to the final project presentation during ICSE 2021 (which will be held virtually) and will receive their awards in digital format at the end of the conference.
Please consider promoting the SCORE contest to your students and colleagues. A flyer is available for download. Also, be sure to pay attention to the SCORE timeline and important dates.
Between March and November 2020, teams of 3-8 students can enter SCORE 2021: they select a project topic among those offered, and register using this form. Their goal is to undertake a full-fledged software engineering project that adheres to the chosen topic description and encompasses all aspects of the engineering process, including planning, requirements, design, implementation, and testing.
Following the best software engineering practices, teams document the process and its outcomes using formal and informal notations, configuration management tools, and process-specific techniques. They collect all artifacts and documentation and produce a detailed project report. By the 10 January 2021 deadline, teams submit a summary report not exceeding 20 pages via Easychair, which gives a self-contained summary of their project development.
Based on the summary reports, the SCORE Program Committee selects a number of semi-finalist teams to submit their complete project reports and artifacts by the 1 March 2021 deadline. After evaluating the complete reports and artifacts of semi-finalist teams, the SCORE Program Committee selects a limited number of finalist teams by the 22 March 2021 deadline and invites them to present their projects during ICSE 2021, which will be held virtually.
An overall SCORE 2021 winner will be selected by the Program Committee after presentations at the ICSE 2021 conference. All finalists will be recognized at the conference. The winning team will receive an award certificate in digital format.
After the SCORE 2021 competition ends, the summary reports of the projects that are selected as finalists will be published on the SCORE contest website.
(see also: FAQ)
Project topics must be selected from the official SCORE projects list. Self-defined or any other project topic will not be accepted for SCORE submission. Always remember that the main focus of the participants should be to produce state-of-the-art software by following a sound software engineering process.
Each project topic description lists one or more reference persons in the SCORE Program Committee who may be contacted with questions regarding the project topic that are not covered in its description. Interactions with a project’s reference persons are limited to teams that have officially registered for that project topic and are subject to restrictions and rules specified in the project description.
Teams that wish to participate in SCORE must register using this form. Registration must be done no later than 30 November 2020.
Participating teams must be composed exclusively of students, at the undergraduate and at the graduate (Master’s) level. Every team must have no less than 3 members. Teams are strongly advised to have no more than 5 members, although we will allow teams of up to 8 participants. Teams may be formed and projects may be developed as part of a software engineering course. Also, teams can be composed of students from different institutions and, in general, teams can work and collaborate from a distance [see For Software Engineering Instructors].
Every team must designate 2 contact persons, to whom communications and inquiries will be addressed. One contact person must be a member of the team; the other may be another team member or a faculty person who is supervising the students (for example if the project is carried out in the context of a software engineering course). However, the faculty person cannot actively participate in the development of the project with the supervised team.
Since SCORE is a Software Engineering contest, participating teams are required to undertake, at least partially, all aspects of the engineering process, including planning, requirements, design, implementation, and testing. Requirements should be described adequately (by traditional means or perhaps by means of user stories, use cases, and scenarios). The outcome of the design phase should be a document that at least describes the architecture model. Implementation must follow the principles of modern software engineering (this includes proper source code documentation). Testing must include unit testing, and possibly integration and acceptance testing.
Projects may focus on some aspects of the process or system and devote more time and space to them in their reports, provided that no aspect is completely neglected, and all are documented. Project documentation must be entirely in English. Reports written in any other language will not be considered or evaluated.
Teams are free to choose their own development approach and to organize their process accordingly. However, they should provide adequate process documentation to show they defined and followed the chosen process.
The use of basic software engineering tools such as source code repositories with version control management, build systems, and testing frameworks is expected; the documented, proper use of more advanced tools will be favorably valued in project reviews. Continuous integration systems, static analysis tools, advanced testing and coverage tools, formal methods, and other types of methods and tools can strengthen teams’ software engineering methodology and ultimately aid them in producing a better product.
The appropriate use of diagrams in project documentation is expected. Different projects will have different needs, so there are no fixed requirements, but both formal and informal diagrams can aid in understanding and documenting both requirements and design.
Program Committee members will not evaluate projects developed by teams including members who are or have been affiliated with the PC members’ institution at any time (regardless of which project is developed by the teams). As for conflict timespan, ACM conflict of interest rules generally apply to the SCORE contest.
Unless exceptions are explicitly stated in advance, all artifacts produced by the teams will be treated confidentially by the PC during the evaluation phase. All projects will be released under a FLOSS license, which is indicated by the project sponsor in their detailed proposal. If the suggested license cannot be accepted (e.g., due to institution requirements), teams are requested to reach out to the sponsor before submitting the initial report, to motivate and negotiate an alternative license.
SCORE 2021 projects cannot be performed as part of paid industrial work. Team members may be supported in their studies by assistantships, scholarships, fellowships, or the like, but duties associated with the support must not be tied to participation in the SCORE project.
To take part in the SCORE contest, teams must register before the 30 November 2020 deadline. Then, before the 10 January 2021 deadline, they must submit a report of no more than 20 pages via EasyChair. Such a report must be formatted single-column with no less than an 11pt body font. The report should describe the various artifacts produced during the development.
The report must include: documentation of the software engineering process; overview of the requirements; overview of the design, including trade-offs and choices; description of the implementation, including platform choices, team assignments, design changes; description of verification and validation activities and outcomes; description and evidence of implementation operation; and a post-mortem analysis with project reflection, lessons learned, and other important non-product outcomes of the project.
It is expected that full project documentation will not fit into the 20-page report, thus you will need to choose your material and craft your report carefully. It would be good to include evidence of project progress such as graphs or analyses of repository activity, growth of user story base, project velocity graphs over iterations, and other such data analysis that shows project progress throughout its timeline and equitable team participation.
Each submitted report will be evaluated by at least 2 members of the SCORE Program Committee. The evaluation will be based on standard quality criteria for software development.
From the report evaluations, a select number of teams will be asked to submit final, full deliverables before the 1 March 2021 deadline. There is no fixed cutoff on the number of teams selected; rather, it will depend on the number of high-quality submissions that are received.
The final deliverable requires you to update your final report and upload it again to EasyChair – updating the previous submission. The final report must include a new section titled "Demo” with a link to access the source code and a link to the Demo to show a fully functioning project. For web applications, it is sufficient to provide an URL pointing to a publicly accessible server where the project has been fully deployed. For other types of projects, we recommend providing a fully functioning project installation in a self-contained Docker container or VirtualBox virtual machine (VM) image. The project documentation should include instructions on how to install and run the application. For projects that involve mobile or other special platforms, for example, the virtual machine can be based on a simulator of the platform running the developed application. Other alternatives are welcome as long as they make the life of Program Committee members easy when it comes to installing and running the projects.
Regardless of the choice, teams are encouraged, wherever possible, to use non-commercial platforms for their installation. If a project depends on commercial software, it must be properly licensed.
The final deliverable (regardless of the chosen format) must include or make available to the Program Committee the implementation code and other development outcomes (work products such as specifications, tests, verification experiments, and so on). The teams are responsible to deliver all the material that is necessary to run and fully evaluate their product (this will include any non-standard, non-free, or non-publicly available development tools, libraries, run-time environments, and so on).
Failing to comply with all these requirements is expected to have a large, negative impact on the evaluation of the project. The evaluation will be based on the quality of all aspects of the project (process followed, development outcomes, and so on).
The Program Committee will then select a small number of overall SCORE finalists based on the final deliverables. Depending on the final budget made available, one or more representatives from these teams will be invited to present their projects at the ICSE 2021 conference (see below for more).
After the decision of ICSE 2021 to go virtual due to the COVID-19 pandemic, finalist teams will present their work remotely. Therefore, travel compensations are not needed anymore. We reserve to post full and final details about any form of prizes or awards for the finalists and/or winning team.
We anticipate that the ICSE 2021 conference will provide a financial award to help offset travel expenses to one member per team for finalist teams. Free registration to the main conference will be offered to a limited number of members of the finalist teams. Anyway, full and final details about financial support to finalist teams will be posted when the overall conference budget is finalized.
- SCORE Registration of Project Teams Open: From 30 March 2020
- SCORE Submissions of Summary Reports Open: From 1 June 2020
- SCORE Teams Registration Deadline: 30 November 2020
- SCORE Summary Report Submission Deadline: 10 January 2021
- SCORE Notification of Semi-Finalists: 8 February 2021
- SCORE Submission of Full Deliverables Deadline [Semi-Finalists Only]: 1 March 2021
- SCORE Notification of Finalists Invited to ICSE 2021: 22 March 2021
- SCORE Final Presentation [Finalists Only]: 22-30 May 2021
Should you need further clarification, do not hesitate to contact the SCORE 2021 co-chairs:
For SE Instructors
We encourage software engineering instructors to consider advertising SCORE to their students and to use one of SCORE’s project topics for their own software engineering teaching assignments. This can be of great benefit for both students and instructors. Here are a few hints to best exploit the opportunities offered by SCORE.
SCORE projects are challenging and should solve real-life problems, so your students can have a unique learning experience. Typically, SCORE projects can be, but do not have to be, developed during a university software engineering course. In some cases it may happen that only certain parts of a SCORE project can fit with the specific focus of your course; in such cases, we suggest that you invite the best students to form teams and officially enter the SCORE contest as a follow up of their work accomplished during the course.
Administering SCORE projects and supervising SCORE teams is an excellent way to motivate and challenge your best students, as well as to make your own school and courses internationally visible by the quality of their achievements. Similarly, one of SCORE’s main goals is to reach a large base of potential (and talented!) participants.
For Project Sponsors
NOTE: Only SCORE PC members may sponsor a project. This page is for them
We need a good number of projects for teams to choose from! A project sponsor (or proposer) does not need to expect to take on a tremendous burden of work. Yes, you will need to define the project, but you can determine your own level of interaction with teams (even down to none), and we can adjust your reviewing activity to compensate. So, propose a project!
Project Proposal Structure
A project definition needs to have:
- A project short name: try to come up with a catchy, self-explanatory name for your project.
- A short description (abstract) of the idea.
- A longer narrative of the problem that gives enough detail for a team to actually go out and work on the project.
- A scoping statement for the project that helps teams understand what must be, what may be, and what does not need to be part of the project, in order to help them understand what is asked for.
- A statement of any process requirements that teams must follow.
- A statement of any environmental constraints (equipment, external interfaces, or other external constraints) that must be satisfied.
- Any restrictions or requirements on what external software cannot or can be used, including commercial tools, open-source libraries or frameworks, etc.
- The selected license under which the developed project must be released – by default GPL (check the FAQ for more).
- A statement of the level of involvement that you as the project sponsor will undertake; this can be something like “will answer selected questions once per week”; also indicate how to reach out (email, Slack, etc.)
You are encouraged to use this project template. You may also want to look up project descriptions from previous editions of SCORE to get an idea of the variety of aspects that you can cover.
Conflicts and Restrictions
A project sponsor has a conflict with a team if one or more of the team members are students of the same institution as the sponsor during project development or if the sponsor has collaborated with one or more team members in research or other work-related activities during the years 2020 and/or 2021.
Accordingly, a team cannot develop any projects proposed by a sponsor who is in conflict with any members of such team. The team can still enter SCORE by picking another project topic. In this case, the sponsor will not review the team’s submissions.
- ChatManager: An Enterprise Instant Messaging abstraction API
- SEESRF: Software Engineering Extensible Study Replication Framework
- PlaDat: Placement Dating for Students
- Howzit: Secure Peer-to-Peer Messaging
- Pycee 2.0: Enhanced Python Compiler Error Messages via Stack Overflow
- ContextSwitcher: Focus on the Thing to be Done
- ADR Manager: The Software Architect’s Favorite Tool
- DevDIDs: Developers’ Decentralized Identifiers
- TrainSE: Training resources for software engineering projects
- AI4Agile: Developing AI-enabled tool support for user story refinement in JIRA
- SmartHealth: Predicting medical events from health data
- ILC: Inconsistency License Checker
- VBP: Support for Voice-Based Programming
- CLup: Customers Line-up
- SLDM: Saving Lives through Data Merging
- Lepic: Supporting the assessment and management of reading fluency
Enterprise Instant Messaging (EIM) platforms are collaboration tools adopted by many companies that allow them to exchange messages and to share files, as well as aggregating notifications from third-party tools. Interaction models offered by these platforms are somewhat similar. Quite the opposite, APIs offered by these tools are often very different from each other, thus making it difficult to integrate third-party applications with different EIM platforms. In this project, you will develop a library that abstracts the interaction layer of EIM platforms. This library will allow configuration and interaction with at least the two most common platforms on the market: Slack by Slack Technologies and Teams by Microsoft.
Sponsors: Giuseppe Santoro and Nicola Sanitate, ApuliaSoft, Italy
Making an experiment fully replicable can be very challenging. Researchers are often discouraged from building a complete replication package (which includes both data and scripts) because it takes a large amount of time. This happens because (i) they need to select the scripts they used among all the ones they produced for making tests, and (ii) some operations may have been performed manually (e.g., the selection of the GitHub repositories to study). In this project, you will implement SEESRF, a framework that will allow Software Engineering researchers to easily (i) automate most of their experiments, and (ii) automatically generate replication packages for their experiments.
Sponsor: Simone Scalabrino, University of Molise, Italy
This project will create a matching service to connect placement students with potential employers. Using a “dating” service metaphor, the site will allow employers to post details of placements available and students to apply for placements. This project will create a database-driven mobile-friendly website to encourage employers and students to engage in year-long paid industrial placements as part of an honours degree programme.
Sponsor: Julian M. Bass, University of Salford, Manchester, UK
Smartphones provide a variety of person-to-person messaging options, generally including vendor-provided SMS clients, as well as many third-party messaging apps such as WhatsApp, Signal, Telegram, and many more. Many of these provide message encryption services. However, these apps all require registration with some central service or authority and generally require providing some personally identifiable information (PII). While this simplifies a variety of administrative details and can greatly ease finding other contacts using the same service, this central authority does represent a point of vulnerability and potential surveillance. Howzit provides completely server-free directory-free secure messaging. Users establish one-on-one connections with an initial in-person setup step, after which they are able to exchange encrypted messages with no need to reveal PII to a central intermediary.
Sponsor: Paul T. Robinson, Sony Interactive Entertainment, USA
Compilers tend to produce cryptic and uninformative error messages, leaving programmers confused and requiring them to spend precious time to resolve the underlying error. The goal of this project is to build the next version of Pycee, a plugin integrated with the popular Sublime Text IDE to provide enhanced compiler error messages for the Python programming language. Pycee automatically queries Stack Overflow to provide customised and summarised information within the IDE. See https://cs.adelaide.edu.au/~christoph/esem19.pdf.
Sponsor: Christoph Treude, University of Adelaide, Australia
ContextSwitcher is Mylyn for the Windows desktop. Suppose you are working on a task. You have opened IntelliJ, some browser tabs, and a Word document. Now, an urgent email comes in. This requires you to switch the context to another task. When you left that task, you had different browser windows and different Word documents opened. ContextSwitcher restores these at a finger type. In that way, it helps you to dive into “the zone” to keep focused on the current task to be done.
Sponsor: Oliver Kopp, JabRef, Germany
An Architectural Decision is a software design choice that addresses a functional or non-functional requirement that is architecturally significant. The Markdown Architectural Decision Records allow for quickly capturing Architectural Decisions. These decisions typically cover the programming language used and the cloud provider taken. They can also cover the technology used for two objects interacting. Currently, the decisions are captured in structured plain-text markdown files. In this project, a web-based user interface for creating, listing, and modifying for Architectural Decision Records should be implemented. GitHub should be used as a file “storage” backend to put the project-focus on the UI and the interaction with the backend.
Sponsors: Oliver Kopp, JabRef, Germany and Christoph Fehling
For today’s digital lives, digital identities are used across every app, service, and device. Decentralized Identifiers (DIDs) are expected to empower users to have greater control over their data. For DIDs, blockchain is considered to be a platform to register identifiers that users can use to identify themselves. This project aims to develop a prototype of software developers’ DIDs on blockchain to enable them to present verifiable claims of the past activities in the belonging software development projects.
Sponsor: Hideaki Hata, Nara Institute of Science and Technology, Japan
With the rapid evolution of software technology, including frameworks, testing and deployment tools, there is a need for a repository of high-quality software engineering resources. Students need access to tutorials and videos that they can use to self-learn. The resources should be better organised than just performing a search on Google. The resources should be lightly controlled to ensure inappropriate material does not appear, be easy to search, and allow for comments on the effectiveness of the learning materials.
Sponsor: Leon Sterling, University of Melbourne, Australia
AI will transform software development practice in many aspects, from automating basic administration tasks to delivering analytics-driven risk predictions and estimation, facilitating project planning, and making actionable recommendations. Agile methods (e.g. Scrum) have been widely used in industry to develop software systems. This project aims to develop an AI-powered app that is integrated into the JIRA platform and supports agile teams in refining user stories, including decomposing an epic into a number of user stories; splitting user stories into small stories; and breaking a user story into a number of specific project tasks.
Sponsor: Hoa Khanh Dam, University of Wollongong, Australia
Over the past 10 years, healthcare data has moved from being largely on paper to being almost completely digitized in electronic medical records (EMRs). EMR systems are a rich source of information that can provide life-saving insights into patients’ health. Despite clear evidence of the major benefits of ensuring safety and quality in the provision of health care services, EMR information is underutilised. In this study, we will harness the power of EMR data and Artificial Intelligence to predict various medical events, such as infections and high cholesterol. We will apply machine learning to health data from EMRs to learn what causes various health outcomes, such as infections and high blood pressure. This app will be used as an augmented intelligence tool for doctors to help them provide more personalized treatment to their patients.
Sponsor: Aldeida Aleti, Monash University, Australia
Sponsors: Igor Scaliante Wiese and Gustavo Pinto, Federal University of Technology - Paraná (UTFPR), Brazil
Thousands of software developers suffer from repetitive strain injuries such as carpal tunnel syndrome and tendonitis. Support for programming by voice has the potential to increase the productivity of developers afflicted by these problems. In addition, it can enable individuals with upper-body motor impairments, e.g, due to spinal cord injuries or strokes, to write code. In this project, we explore the development of a prototype system to support programming by voice.
Sponsor: Fernando Castor, Federal University of Pernambuco, Brazil
The coronavirus emergency has put a strain on society on many levels, due to many countries imposing lockdowns that allow people to exit their homes only for essential needs, and enforcing strict rules even when people are justified in going out (such as limiting the number of accesses to buildings and keeping a distance of at least one meter between people). In particular, grocery shopping—a most essential need—can become a challenge in the presence of such strict rules. Indeed, supermarkets need to restrict access to their stores to avoid having crowds inside, which typically results in long lines forming outside, which are themselves a source of hazards. In these trying times, people turn to technology, and in particular to software applications, to help navigate the challenges created by the imposed restrictions. The goal of this project is to develop an easy-to-use application that, on the one side, allows store managers to regulate the influx of people in the building and, on the other side, saves people from having to line up and stand outside of stores for hours on end. Sponsor: Matteo Rossi, Politecnico di Milano, Italy
In the aftermath of disasters, mobile teams of medics travel to remote areas to diagnose illnesses, treat injuries, and administer drugs to support the sick and injured. However, due to lack of internet connectivity, medics must often rely on stored information that may not have been updated for days. Furthermore, details about new patients will remain on local databases until the medical teams return to areas with some form of connectivity. Why is this a problem? Refugees on the move may be treated at different locations. When their records are finally uploaded, it can be difficult to match patients to their previous treatments in other locations, which can have life-threatening implications. The Fast Electronic Medical Record (fEMR) system is an open-source Electronic Medical Records system tailored for “pop-up” clinics. This project involves enhancing fEMR to merge or link patient records, created in different locations that appear to document the same person. This might be done, for example, using natural language processing, facial recognition, machine learning, etc.
Sponsors: Sarah Draugelis (President) and Andy Mastie (CTO), Team fEMR, UK
Reading skills development opens up the world to the students, and, beyond that, it expands and enhances learning development. If someone has good reading skills, he/she can also improve the learning skills because he/she will be able to interpret texts and questions better. This project’s main goal is to develop a tool to support reading fluency assessment and management, as shown with manual analysis in previous work (Celeste et al., 2018). The software aims to help professionals that work with students’ literacy and kid’s relatives.
Sponsors: Fabiana Mendes, George Marsicano, and Letícia Celeste - University of Brasília (UnB), Brazil Luciana Mendonça Alves - Federal University of Minas Gerais (UFMG), Brazil
The submission of summary reports opens on Dec 1, 2020.
Projects will be submitted via Easychair.
- Remember the 20-page limit for the initial report submission. For other details, refer to the FAQ
- When filling out your submission form on EasyChair, please edit the Title field as follows: Name of group - Project Acronym
Q) Upon submission of the project who will have copyright/ownership over the project?
- Student teams own the work that they do unless the students’ institution(s) have other rules that supersede these rights.
Q) How large must a team participating to SCORE be?
- The minimum number of people in a team is 3, in order to guarantee that project development involves real teamwork. The maximum number of people in a team is 8, but we encourage teams to have no more than 5 members.
Q) Do I have to be a student to take part in SCORE? What kind of student?
- Yes, you must be a full-time student, and you must provide proof of this fact when submitting your project. We allow both undergraduate and Master’s students, but no Ph.D. students. Teams can mix undergraduate and graduate students.
Q) Why not Ph.D. students?
- It is a matter of fairness, as Ph.D. students are usually well versed in software engineering theory and practice, having studied the subject in great depth both in classes and during their research activity, which gives them an unfair advantage over much less experienced competitors.
Q) I am currently a student at the Master’s level, but I will receive my degree in the year 2020, then I will start a Ph.D. program. Can I still be part of a team participating in SCORE?
- It depends on when the summary report is submitted. We will check the status of the members of a participating team at the time of the submission of the summary report (which can be any time between July 2020 and January 2021). If you are still a student at the undergraduate/Master’s level at that time, then you (and your team) will be allowed to participate in SCORE 2021.
Q) I am currently a student at the undergraduate/Master’s level, but I will graduate in the year 2020, and then I will start working for a company. Can I still be part of a team participating in SCORE?
- It depends on when the summary report is submitted. We will check the status of the members of a participating team at the time of the submission of the summary report (which can be any time between July 2020 and January 2021). If you are still a student at the undergraduate/Master’s level at that time, then you (and your team) will be allowed to participate in SCORE 2021.
Q) Can members of the same team come from different institutions or countries?
- Yes, they can. Diverse teams are very welcome, although SCORE does not provide any mechanism to form or facilitate the interaction in such teams, so the team formation and management is up to the team members themselves.
Q) Is it required for each team to have a reference faculty member? Is it advised? What is a faculty advisor’s role?
- There is no mandatory requirement on having a reference faculty member, although it is fine (and probably useful) if teams have one. It has to be understood that the faculty referent must be at most a general advisor, and must not participate actively in the development of the project by the team.
Q) What happens if a team composition changes after the team has registered?
- A limited amount of changes in the team composition are allowed even after registration. For instance, the team may grow in size, or one person may be replaced. The contact person, however, cannot be changed. We also require that changes in team composition be promptly reported by updating the registration data in EasyChair.
See also the page dedicated to this topic.
Q) I teach a Software Engineering course at my University. Can my students develop SCORE projects within the course framework and timeline?
- Absolutely! This is allowed and encouraged, is an excellent way to reach a large base of potential (and talented!) participants, and it is a great way for instructors to provide challenging projects to their students.
Q) How can I accommodate SCORE within my course schedule?
- The SCORE timeline is designed to allow a project to be undertaken during 2020, to match well with most university course schedules. The target SCORE project size is approximately a standard one-term team project.
Q) How is the evaluation process managed?
- The evaluation is a two-phase process. First, every team delivers a summary report (the deadline is 10 January 2020, but reports can be delivered earlier; see the CfP for suggestions on the production of the report). The report will be evaluated by members of SCORE’s Program Committee. Based on this first evaluation, a subset of the teams will be invited to submit full project documentation (see here), which will be used by the PC to select the finalists (to be invited to ICSE) and the winners (from among the invited finalists). The final evaluation may also depend on the teams’ oral presentations given at ICSE: suggestions will be given to the finalists on how to organize their presentations.
Q) How are project submissions evaluated?
- Since the level of complexity varies across proposals, these are the broad areas that will be considered in the assessment. For finalist projects-only, the working software that can be demoed will be the key deliverable and will be judged by the PC, client, and chairs in relation to how well the software meets the requirements.
- Requirements: The quality of the software requirements and the relevant documentation in terms of clarity, consistency, comprehensiveness.
- Design: The quality of design of the system including an appropriate level of detail.
- Implementation (for finalist projects): Evidence showing that the system is functional and about decisions related to implementation, decisions, and technologies used.
- Collaboration: Clear evidence of how the team collaborates internally and with the client to develop the software system.
- Verification/Validation: The effectiveness and rigor of the software quality assurance process and outcomes.
Q ) Under what license will the project have to be released?
- SCORE 2020 will not force a specific license, which instead will be indicated by each sponsor in their project proposal. Still, (a) by default, we recommend the GPL of the LGPL licenses, which will grant reviewers access to the code for evaluation (see below) while preventing at the same time anyone to close-source and become proprietary of the project code afterward; (b) should their institution have different requirements, student teams and instructors are invited to reach out to the sponsors to identify an alternative license – in this case, we request teams to add the written agreement (even an email will suffice) to the project report when submitting it for evaluation. If you need more information about different FLOSS licenses, see here and here.
Q ) Should the project source code be accessible?
- Yes, regardless of the final license of the project, teams passing the first evaluation will be required to make the source code available to PC members as part of the full documentation. For this reason, we require teams to share their project on a code hosting platform such as GitHub or GitLab. All the material will be treated confidentially by the SCORE organizers and PC members.
Q) Can my team send the application code/test results/verification results, instead of (or in addition to) a 20-page document, as the summary report?
- No, you will be allowed to submit only the 20-page report. You will be asked for additional, more in-depth information and deliverables for your project only if you are selected as a semi-finalist for the second phase of the evaluation.
Q) Does this mean that we can delay doing the implementation until we are selected as semi-finalists for the second phase of the evaluation process?
- No. Given the short time (about a month and a half, from mid-January to March 2021) between the summary report and the final deliverable, it is expected that your project will be completed in all its aspects and fully documented by the time you submit the summary report. The report also documents your development process, which must have actually occurred.
Q) We are registered for the SCORE contest; do we have to wait until January 2021 to submit our summary report, even if we have finished it well in advance?
- No, you don’t have to wait until January 2021 to submit the project’s summary report. If your project is completed earlier than the submission deadline, you can submit it when you are finished. When you are ready, please submit to EasyChair.
Q) We have already submitted the summary report for our project, but submissions are still open: can we submit a new version of the report?
- Yes, you can modify your submission and send us a new, revised summary report as long as submissions are still open. Only your latest submission will be considered for evaluation (and will have to respect all criteria for eligibility for evaluation, including those on team composition).
Q) We have submitted the summary report well in advance of the closing date for submission: will you evaluate our project and notify us if we have been selected to send the final deliverable before the notification date?
- No. To coordinate the program committee (PC) work, to be considerate to the PC members’ own need to schedule their reviewing, and to be sure that all reviews are equitable, all reviewing will take place between the report submission deadline and the notification deadline (8 February 2021).
Q) If our team is selected as a finalist, we’ll have a relatively short time to arrange our trip to Madrid. In particular, we may have trouble getting visas to travel. What can we do about that? If your team is selected for submitting the final deliverable, you have a good (approximately 50%) chance of being selected as a finalist. Hence, we suggest that you tentatively start making your travel arrangements as soon as you are selected as a semi-finalist in February 2021. In addition, if you think the process for getting your visas may be particularly long, notify us as soon as possible, possibly when submitting your first summary report. ACM issues visa support letters after conference registration. We will do our best to notify the finalists as soon as possible, and to help to make their travel arrangements in time. More information about this issue will be made available when the submission deadlines are approaching.
Q) ICSE 2021 is going virtual, what about SCORE?
- As part of ICSE, SCORE 2021 is going virtual too. This means that finalists teams will be invited to present their work remotely. Final details about dates, infrastructures, and presentation options (e.g., live or recorded demo) will be posted on due time.