Testing is an important aspect of software development, but unfortunately, it is often neglected. While test quality analyses such as code coverage or mutation analysis inform developers about the quality of their tests, such reports are viewed only sporadically during continuous integration or code review, if they are considered at all, and their impact on the developers’ testing behavior therefore tends to be negligible. To actually influence developer behavior, it may rather be necessary to motivate developers directly within their programming environment, while they are coding. We introduce Blinded, a gamified plugin for the popular IntelliJ Java Integrated Development Environment, which rewards developers for positive testing behavior using a multi-level achievement system: A total of 27 different achievements, each with incremental levels, provide affirming feedback when developers exhibit commendable testing behavior, and provide an incentive to further continue and improve this behavior. A controlled experiment with 49 participants given a Java programming task reveals substantial differences in the testing behavior triggered by Blinded: Incentivized developers write more tests, achieve higher coverage and mutation scores, run their tests more often, and achieve functionality earlier.
Thu 18 AprDisplayed time zone: Lisbon change
14:00 - 15:30 | Testing 4Research Track / Journal-first Papers at Grande Auditório Chair(s): Shiva Nejati University of Ottawa | ||
14:00 15mTalk | Concrete Constraint Guided Symbolic Execution Research Track Yue Sun Institute of Information Engineering, CAS, China, Guowei Yang University of Queensland, Shichao Lv College of Cyberspace Security, Chinese Academy of Sciences, Zhi Li Institute of Information Engineering, Chinese Academy of Sciences, China, Limin Sun Institute of Information Engineering, Chinese Academy of Sciences, School of Cyber Security, University of Chinese Academy of Sciences, Pre-print | ||
14:15 15mTalk | Improving Testing Behavior by Gamifying IntelliJ Research Track DOI Pre-print | ||
14:30 15mTalk | SCTrans: Constructing a Large Public Scenario Dataset for Simulation Testing of Autonomous Driving Systems Research Track Jiarun Dai Fudan University, Bufan Gao Fudan University, Mingyuan Luo Fudan University, Zongan Huang Fudan University, Zhongrui Li Fudan University, Yuan Zhang Fudan University, Min Yang Fudan University | ||
14:45 15mTalk | Curiosity-Driven Testing for Sequential Decision-Making Process Research Track Junda He Singapore Management University, Zhou Yang Singapore Management University, Jieke Shi Singapore Management University, Chengran Yang Singapore Management University, Singapore, Kisub Kim Singapore Management University, Singapore, Bowen Xu North Carolina State University, Xin Zhou Singapore Management University, Singapore, David Lo Singapore Management University | ||
15:00 15mTalk | Detecting Logic Bugs in Graph Database Management Systems via Injective and Surjective Graph Pattern Transformation Research Track Yuancheng Jiang National University of Singapore, Jiahao Liu National University of Singapore, Jinsheng Ba National University of Singapore, Roland H. C. Yap National University of Singapore, Singapore, Zhenkai Liang National University of Singapore, Manuel Rigger National University of Singapore DOI Pre-print | ||
15:15 7mTalk | Testing Causality in Scientific Modelling Software Journal-first Papers Andrew Graham Clark The University of Sheffield, Michael Foster The University of Sheffield, Neil Walkinshaw The University of Sheffield, Robert Hierons The University of Sheffield, Benedikt Prifling Ulm University, Volker Schmidt Ulm University, Robert D. Turner The University of Sheffield | ||
15:22 7mTalk | HybridCISave: A Combined Build and Test Selection Approach in Continuous Integration Journal-first Papers | ||