Dealing with Data Challenges when Delivering Data-Intensive Software Solutions
The predicted increase in demand for data-intensive solution development is driving the need for software, data, and domain experts to effectively collaborate in multi-disciplinary data-intensive software teams (MDSTs). We conducted a socio-technical grounded theory study through interviews with 24 practitioners in MDSTs to better understand the challenges these teams face when delivering data-intensive software solutions. The interviews provided perspectives across different types of roles including domain, data and software experts, and covered different organisational levels from team members, team managers to executive leaders. We found that the key concern for these teams is dealing with data-related challenges. In this paper, we present a theory of dealing with data challenges that explains the challenges faced by MDSTs including gaining access to data, aligning data, understanding data, and resolving data quality issues; the context in and condition under which these challenges occur, the causes that lead to the challenges, and the related consequences such as having to conduct remediation activities, inability to achieve expected outcomes and lack of trust in the delivered solutions. We also identified contingencies or strategies applied to address the challenges including high-level strategic approaches such as implementing data governance, implementing new tools and techniques such as data quality visualisation and monitoring tools, as well as building stronger teams by focusing on people dynamics, communication skill development and cross-skilling. Our findings have direct implications for practitioners and researchers to better understand the landscape of data challenges and how to deal with them.
Fri 19 AprDisplayed time zone: Lisbon change
14:00 - 15:30 | Human and Social 7Journal-first Papers / Software Engineering in Practice / Research Track at Luis de Freitas Branco Chair(s): Igor Steinmacher Northern Arizona University | ||
14:00 15mTalk | “I tend to view ads almost like a pestilence”: On the Accessibility Implications of Mobile Ads for Blind Users Research Track Ziyao He University of California, Irvine, Syed Fatiul Huq University of California, Irvine, Sam Malek University of California at Irvine | ||
14:15 15mTalk | Constructive Code Review: Managing the Impact of Interpersonal Conflicts in Practice Software Engineering in Practice Pavlina Wurzel Goncalves University of Zurich, Joao S. V. Goncalves University of Zurich, Alberto Bacchelli University of Zurich | ||
14:30 15mTalk | Motivating Open Source Collaborations Through Social Network Evaluation: A Gamification Practice from Alibaba Software Engineering in Practice Shengyu Zhao Tongji University, Xiaoya Xia East China Normal University, Brian Fitzgerald Lero - The Irish Software Research Centre and University of Limerick, Xiaozhou Li University of Oulu, Valentina Lenarduzzi University of Oulu, Davide Taibi University of Oulu and Tampere University , Rong Wang Alibaba Group, will wang , Chunqi Tian Yongji University | ||
14:45 15mTalk | Objectives and Key Results in Software Teams: Challenges, Opportunities and Impact on Development Software Engineering in Practice Jenna L. Butler Microsoft Research, Thomas Zimmermann Microsoft Research, Christian Bird Microsoft Research | ||
15:00 7mTalk | Dealing with Data Challenges when Delivering Data-Intensive Software Solutions Journal-first Papers Ulrike Maria Graetsch , Hourieh Khalajzadeh Deakin University, Australia, Rashina Hoda Monash University, Mojtaba Shahin RMIT University, John Grundy Monash University Link to publication DOI Pre-print | ||
15:07 7mTalk | What’s (Not) Working in Programmer User Studies? Journal-first Papers Matthew C. Davis Carnegie Mellon University, Emad Aghayi , Thomas LaToza George Mason University, Xiaoyin Wang University of Texas at San Antonio, Brad A. Myers Carnegie Mellon University, Joshua Sunshine Carnegie Mellon University Link to publication DOI | ||
15:14 7mTalk | Confirmation Bias and Time Pressure: A Family of Experiments in Software Testing Journal-first Papers Iflaah Salman Lappeenranta-Lahti University of Technology (LUT), Burak Turhan University of Oulu, Robert Ramač Faculty of Technical Sciences, University of Novi Sad, Vladimir Mandić Faculty of Technical Sciences, University of Novi Sad |