How Do Different Types of Testing Goals Affect Test Case Design?
Test cases are designed in service of goals, e.g., functional correctness or performance. Unfortunately, we lack a clear understanding of how specific goal types influence test design. In this study, we explore this relationship through interviews and a survey with software developers, with a focus on identification and importance of goal types, quantitative relations between goals and tests, and personal, organizational, methodological, and technological factors.
We identify nine goal types and their importance, and perform further analysis of three—correctness, reliability, and quality. We observe that test design for correctness forms a “default” design process that is modified when pursuing other goals. For the examined goal types, test cases tend to be simple, with many tests targeting a single goal and each test focusing on 1–2 goals at a time. We observe differences in testing practices, tools, and targeted system types between goal types. In addition, we observe that test design can be influenced by organization, process, and team makeup. This study provides a foundation for future research on test design and testing goals.
Tue 19 SepDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
09:10 - 11:00 | Session on Test automation and designIFIP-ICTSS 35th International Conference on Testing Software and Systems at Room 5 - Conference room Chair(s): Mercedes G. Merayo Universidad Complutense de Madrid | ||
09:10 20mTalk | Multi-device, robust, and integrated Android GUI testing: A conceptual framework IFIP-ICTSS 35th International Conference on Testing Software and Systems Riccardo Coppola Politecnico di Torino, Luca Ardito Politecnico di Torino, Marco Torchiano Politecnico di Torino | ||
09:30 30mTalk | How Do Different Types of Testing Goals Affect Test Case Design? IFIP-ICTSS 35th International Conference on Testing Software and Systems Pre-print | ||
10:00 30mTalk | Empirical Verification of TQED - a New Test Design Heuristic Technique IFIP-ICTSS 35th International Conference on Testing Software and Systems | ||
10:30 30mTalk | Who is Afraid of Test Smells? Assessing Technical Debt from Developer Actions IFIP-ICTSS 35th International Conference on Testing Software and Systems |