MSR 2023
Dates to be announced Melbourne, Australia
co-located with ICSE 2023
Tue 16 May 2023 11:12 - 11:24 at Meeting Room 109 - Documentation + Q&A II Chair(s): Maram Assi

In Stack Overflow (SO), the quality of posts (i.e., questions and answers) is subjectively evaluated by users through a voting mechanism. The net votes (upvotes - downvotes) obtained by a post are often considered an approximation of its quality. However, about half of the questions that received working solutions got more downvotes than upvotes. Furthermore, about 18% of the accepted answers (i.e., verified solutions) also do not score the maximum votes. All these counter-intuitive findings cast doubts on the reliability of the evaluation mechanism employed at SO. Moreover, many users raise concerns against the evaluation, especially downvotes to their posts. Therefore, rigorous verification of the subjective evaluation is highly warranted to ensure a non-biased and reliable quality assessment mechanism. In this article, we compare the subjective assessment of questions with their objective assessment using 2.5 million questions and ten text analysis metrics. According to our investigation, (1) four objective metrics agree, (2) two metrics do not agree, (3) one metric either agrees or disagrees, and (4) the remaining three metrics neither agree nor disagree with the subjective evaluation. We then develop machine learning models to classify the promoted and discouraged questions. Our models outperform the state-of-the-art models with a maximum of about 76%–87% accuracy.

Tue 16 May

Displayed time zone: Hobart change

11:00 - 11:45
Documentation + Q&A IITechnical Papers / Data and Tool Showcase Track at Meeting Room 109
Chair(s): Maram Assi Queen's University
11:00
12m
Talk
Understanding the Role of Images on Stack Overflow
Technical Papers
Dong Wang Kyushu University, Japan, Tao Xiao Nara Institute of Science and Technology, Christoph Treude University of Melbourne, Raula Gaikovina Kula Nara Institute of Science and Technology, Hideaki Hata Shinshu University, Yasutaka Kamei Kyushu University
Pre-print
11:12
12m
Talk
Do Subjectivity and Objectivity Always Agree? A Case Study with Stack Overflow Questions
Technical Papers
Saikat Mondal University of Saskatchewan, Masud Rahman Dalhousie University, Chanchal K. Roy University of Saskatchewan
Pre-print
11:24
6m
Talk
GiveMeLabeledIssues: An Open Source Issue Recommendation System
Data and Tool Showcase Track
Joseph Vargovich Northern Arizona University, Fabio Marcos De Abreu Santos Northern Arizona University, USA, Jacob Penney Northern Arizona University, Marco Gerosa Northern Arizona University, Igor Steinmacher Northern Arizona University
Pre-print Media Attached
11:30
6m
Talk
DocMine: A Software Documentation-Related Dataset of 950 GitHub Repositories
Data and Tool Showcase Track
11:36
6m
Talk
PENTACET data - 23 Million Code Comments and 500,000 SATD comments
Data and Tool Showcase Track
Murali Sridharan University of Oulu, Leevi Rantala University of Oulu, Mika Mäntylä University of Oulu