Mon 26 Oct 2015 15:30 - 15:50 at Grand Station 5 - Group Discussion Chair(s): Craig Anslow, Thomas LaToza, Joshua Sunshine

While it looks like controlled trials with human envolvement are more and more often applied in software science, there is the problem that there are hardly any explicitly documented community standards that need to be addressed by such trials. This leads to a number of problems: experimenters cannot be sure whether an experiment they perform does represent the current state-of-the-art, reviewers have no guidelines to check whether a critique they have in mind is valid or not, and readers from experiments have hardly any chance to check, whether the results of an experiment, they are currently reading, should be taken serious and should be permitted to chance the reader’s perspective on a certain topic. This paper discusses the problem of missing community agreements and makes some first proposals with respect to subjects, training, measurements, experimental designs, and documentation.

On the Need to Define Community Agreements for Controlled Experiments with Human Subjects (plateau2015-hanenberg.pdf)91KiB

Mon 26 Oct

Displayed time zone: Eastern Time (US & Canada) change

15:30 - 17:00
Group DiscussionPLATEAU at Grand Station 5
Chair(s): Craig Anslow Middlesex University, London, Thomas LaToza George Mason University, Joshua Sunshine Carnegie Mellon University
15:30
20m
Talk
On the Need to Define Community Agreements for Controlled Experiments with Human Subjects -- A Discussion Paper
PLATEAU
Stefan Hanenberg University of Duisburg-Essen, Andreas Stefik University of Nevada, Las Vegas
File Attached
15:50
70m
Talk
Group discussion
PLATEAU