Software product forums is a platform filled with user feedback that utilises the sub-forum feature to categorise user discussion into themes. These sub-forums are very similar to classification labels that have been used to automatically classify user feedback on other platforms such as Troubleshooting and Feature Request. It would be very beneficial to the CrowdRE community if these sub-forum categories can be utilised in a research setting as it would reduce the effort required to label content for classification manually. However, no research has been done on the accuracy of these sub-forum categorisations in software product forums. In this exploratory study, we examined the accuracy of user categorised posts in two software product forums and discovered that users incorrectly categorise more than 20% of the posts during submission. Our discovery suggests that at the current stage, sub-forum categories should not be trusted as a label to classify feedback automatically.
Oliver Karras TIB - Leibniz Information Centre for Science and Technology, Eklekta Kristo Leibniz University Hannover, Jil Klünder Leibniz Universität Hannover
Meira Levy Shenkar College of Engineering, Design and Art, Irit Hadar University of Haifa, Assaf Krebs Shenkar College of Engineering, Design and Art, Idit Barak Shenkar College of Engineering, Design and Art
Oliver Karras TIB - Leibniz Information Centre for Science and Technology, Eduard C. Groen Fraunhofer IESE, Javed Ali Khan University of Science and Technology Bannu, Sören Auer TIB - Leibniz Information Centre for Science and Technology