Barriers to Shift-Left Security: The Unique Pain Points of Writing Automated Tests Involving Security Controls
Background: Automated unit and integration tests allow software development teams to continuously evaluate their application’s behavior and ensure requirements are satisfied. Interest in explicitly testing security at the unit and integration levels has risen as more teams begin to shift security left in their workflows, but there is little insight into any potential pain points developers may experience as they learn to adapt their existing skills to write these tests.
Aims: Identify security unit and integration testing pain points that could negatively impact efforts to shift security (testing) left to this level.
Method: An mixed-method empirical study was conducted on 525 Stack Overflow and Security Stack Exchange posts related to security unit and integration testing. Latent Dirichlet Allocation (LDA) was applied to identify commonly discussed topics, pain points were learned through qualitative analysis, and links were analyzed to study commonly-shared resources.
Results: Nine topics representing security controls, components, and scenarios were identified; Authentication was the most commonly tested control. Developers experienced seven pain points unique to security unit and integration testing, which were all influenced by the complexity of security control designs and implementations. Most linked resources were other Q&A posts, but repositories and documentation for security tools and libraries were also common.
Conclusions: Developers may experience several unique pain points when writing tests at this level involving security controls. Additional resources are needed to guide developers through these challenges, which should also influence the creation of strategies and tools to help shift security testing to this level. To accelerate this, actionable recommendations for practitioners and future research directions based on these findings are highlighted.
Tue 12 OctDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
15:30 - 16:35
Testing & Security 2Technical Papers / Emerging Results and Vision papers at ESEM ROOM
Chair(s): Davide Fucci Blekinge Institute of Technology
|Barriers to Shift-Left Security: The Unique Pain Points of Writing Automated Tests Involving Security Controls|
Danielle Gonzalez Rochester Institute of Technology and Microsoft, Paola Peralta Perez Rochester Institute of Technology, Mehdi Mirakhorli Rochester Institute of TechnologyDOI
|Security Smells Pervade Mobile App Servers|
Pascal Gadient University of Bern, Marc-Andrea Tarnutzer University of Bern, Oscar Nierstrasz University of Bern, Switzerland, Mohammad Ghafari University of AucklandPre-print
|Who are Vulnerability Reporters? A Large-scale Empirical Study on FLOSS|
Nikolaos Alexopoulos Technical University of Darmstadt, Andy Meneely Rochester Institute of Technology, Dorian Arnouts Technical University of Darmstadt, Max Mühlhäuser Technical University of DarmstadtPre-print
|Python Crypto Misuses in the Wild|
Emerging Results and Vision papers
Anna-Katharina Wickert TU Darmstadt, Germany, Lars Baumgärtner TU Darmstadt, Florian Breitfelder TU Darmstadt, Mira Mezini TU Darmstadt, GermanyPre-print Media Attached
|Web Application Testing: Using Tree Kernels to Detect Near-duplicate States in Automated Model Inference|
Emerging Results and Vision papers
Anna Corazza Università degli Studi di Napoli Federico II, Sergio Di Martino Università degli Studi di Napoli Federico II, Adriano Peron Università degli Studi di Napoli Federico II, Luigi Libero Lucio Starace Università degli Studi di Napoli Federico IIPre-print Media Attached