Carolyn Seaman, Jeffrey C. Carver

Abstract:

In 2020, a group of software engineering researchers led by Dr. Paul Ralph at Dalhousie University, at the invitation of ACM SIGSOFT, released a set of standards for the conduct and review of empirical studies in software engineering. The standards have been proposed, but not adopted, as a guide for future ESEM reviewers. Like most guidelines and standards, this one has not been empirically validated. During ISERN 2022, ISERN members participated in a session that constituted an empirical study in which they were guided through an evaluation of the 2020 standard, using submitted manuscripts that they had previously reviewed. This session, in 2023, will report the results of that study from ISERN 2022.

Session Goals:

• Report the results of the evaluation of the empirical software engineering standards
• Provide feedback to the maintainers of the standard
• Provide input into decisions about the appropriate use of the standards

Development of the Session: (How will the session be conducted? How much interaction?)

There will be a presentation reporting the results of the evaluation in 2022. There will be Q&A. Participants will be asked for their opinion about what should be done with the results (shared with the standards maintainers, published, etc.).

What means for interaction will be used or required?

There will simply be a slide presentation and then open discussion.

Background and recommended reading:

None

Expected Outcomes and Plan for Continuing the Work beyond ISERN:

A report will be written to document the results of the 2022 evaluation, as well as any input from this session in 2023. This report will be submitted to the team responsible for maintaining the 2020 standards. Depending on the feedback from this session, the organizers might consider submitting the report for publication.