Test environments for large-scale software systems – an industrial study of intrinsic and extrinsic success factors
The characteristics of the test environment are of vital importance to its ability to support the organizations testing objectives. This paper seeks to address the need for a structured and reliable approach, which can be used by companies and other organizations to optimize their test environments in each individual case. The reported study included a series of interviews with 30 individuals, a series of focus groups with in total 31 individuals, and a cross-company workshop with 30 participants from five large-scale companies, operating in different industry segments. The study resulted in a list of success factors, including characteristics and capabilities existing within a test environment (intrinsic success factors) but also properties not inherent to the test environment, but still vital for a successfully implemented test environment (extrinsic success factors). This distinction is important, as the root causes differ and as addressing them requires distinct approaches – not only of technology, but of organization, communication and collaboration. We find that successful implementations of test environments for large-scale software systems depend primarily on how they support the company’s business strategy, test organization and product testability (extrinsic success factors). Based on this, test environments can then be optimized to improve test environment capabilities, usability and stability (intrinsic success factors). The list of intrinsic and extrinsic success factors was well received by all five companies included in the study, supporting that the intrinsic and extrinsic success factors for test environments can be applied to a large segment of the software industry.
Mon 17 AprDisplayed time zone: Dublin change
16:00 - 18:00 | Session 7: Testing for Safery & Security Industry / Research Papers / Journal-First Papers / Previous Editions at Hanover Chair(s): Eric Bodden Heinz Nixdorf Institut, Paderborn University and Fraunhofer IEM | ||
16:00 20mTalk | Learning Non-robustness using Simulation-based Testing: a Network Traffic-shaping Case Study Industry Baharin Aliashrafi Jodat University of Ottawa, Shiva Nejati University of Ottawa, Mehrdad Sabetzadeh University of Ottawa, Patricio Saavedra RabbitRun Technologies Inc Pre-print | ||
16:20 20mTalk | Test environments for large-scale software systems – an industrial study of intrinsic and extrinsic success factors Journal-First Papers | ||
16:40 20mTalk | Assessing the Effectiveness of Input and Output Coverage Criteria for Testing Quantum Programs Previous Editions Shaukat Ali Simula Research Laboratory, Paolo Arcaini National Institute of Informatics
, Xinyi Wang , Tao Yue Simula Research Laboratory DOI | ||
17:00 20mTalk | Heap Fuzzing: Automatic Garbage Collection Testing with Directed Random Events Research Papers Guillermo Polito Inria, Cristal, UMR 9189, Université de Lille, Pablo Tesone Univ. Lille, Inria, CNRS, Centrale Lille, UMR 9189 CRIStAL, Pharo Consortium, Jean Privat Université du Québec à Montréal (UQAM), Nahuel Palumbo Université Lille, CNRS, Centrale Lille, Inria, UMR 9189 - CRIStAL, Stéphane Ducasse Inria; University of Lille; CNRS; Centrale Lille; CRIStAL | ||
17:20 20mTalk | MagicMirror: Towards High-Coverage Fuzzing of Smart Contracts Research Papers Huadong Feng University of Texas at Arlington, Xiaolei Ren University of Texas at Arlington, Qiping Wei University of Texas at Arlington, Jeff Yu Lei University of Texas at Arlington, Raghu Kacker National Institute of Standards and Technology, Richard Kuhn National Institute of Standards and Technology, Dimitris Simos SBA Research |