An Explainable Automated Model for Measuring Software Engineer Contribution
Software engineers play an important role throughout the software development life-cycle, particularly in industry emphasizing quality assurance and timely delivery. Contribution measurement provides proper incentives to software engineers that motivate them to continuously improve the quality and efficiency of their work. However, existing research tends to ignore contribution measurement for software engineers in practice, relying heavily on peer review and lacking objectivity and transparency. Specifically, these studies still have two weaknesses. First, a few studies explore which metrics can be useful for contribution measurement in practice. Second, managers measure the contribution of software engineers based on their experience and lack of explainable automated tools to assist them.
To this end, we conduct mixed-method studies to investigate contribution measurement in the enterprise, and then propose an explainable model for measuring the contribution of software engineers. First, we collect and synthesize 16 metrics for contribution measurement by interviewing 18 industrial practitioners. Second, we propose an explainable model, called \textbf{MEMENTO}, integrating \textbf{M}ulti-dim\textbf{E}nsional \textbf{ME}trics for measuri\textbf{N}g con\textbf{T}ributi\textbf{O}n. We conduct an industrial case study with a global enterprise to evaluate and refine \ourapproach. Finally, we administer a survey to industrial practitioners to verify whether the explainable model is useful for contribution analysis, which resulted in 67 valid responses. \ourapproach is used by the enterprise to measure the contribution of 5,174 employees. A total of 100 employees are identified as low contributors by \ourapproach, and the five reasons for their low contribution are determined based on the results of our explainable model. The results indicate that \ourapproach can effectively measure the contribution of software engineers, which provides a practical reference for researchers interested in contribution measurement.
Wed 30 OctDisplayed time zone: Pacific Time (US & Canada) change
15:30 - 16:30 | Software processResearch Papers / Industry Showcase / Journal-first Papers at Gardenia Chair(s): Tim Menzies North Carolina State University | ||
15:30 15mTalk | An Explainable Automated Model for Measuring Software Engineer Contribution Research Papers Yue Li Nanjing University, He Zhang Nanjing University, Yuzhe Jin Nanjing University, Zhong Ren State Key Laboratory of Novel Software Technology, Software Institute, Nanjing University Nanjing, Jiangsu, China, Liming Dong Nanjing University, Jun Lyu Nanjing University, Lanxin Yang Nanjing University, David Lo Singapore Management University, Dong Shao Nanjing University | ||
15:45 15mTalk | Context-Aware Automated Sprint Plan Generation for Agile Software Development Research Papers Elvan Kula Delft University of Technology, Arie van Deursen Delft University of Technology, Georgios Gousios Endor Labs & Delft University of Technology | ||
16:00 15mTalk | Agile meets quantum: a novel genetic algorithm model for predicting the success of quantum software development project Journal-first Papers Arif Ali Khan University of Oulu, Muhammad Azeem Akbar LUT University, Valtteri Lahtinen QUANSCIENT, Marko Paavola VTT Technical Research Ctr of Finland Ltd, Oulu, Finland, Mahmood Niazi King Fahd University of Petroleum and Minerals, Mohammed Naif Alatawi Information Technology Department, Faculty of Computers and Information Technology, University of Tabuk, Tabuk, Saudi Arabia, Shoayee Dlaim Alotaibi Department of Artificial Intelligence and Data Science, College of Computer Science and Engineering, University of Hail, Hail, Saudi Arabia | ||
16:15 15mTalk | How many pomodoros do professional engineers need to complete a microtask of programming? Industry Showcase Shinobu Saito NTT, IIMURA Yukako NTT Corporation, Emad Aghayi George Mason University, Thomas LaToza George Mason University |