ISSTA 2022
Mon 18 - Fri 22 July 2022 Online

Static analysis tools support developers in detecting potential coding issues, such as bugs or vulnerabilities. Research on static analysis emphasizes its technical challenges but also mentions severe usability shortcomings. These shortcomings hinder the adoption of static analysis tools, and in some cases, user dissatisfaction even leads to tool abandonment.

To comprehensively assess the current state of the art, this paper presents the first systematic usability evaluation in a wide range of static analysis tools. We derived a set of 36 relevant criteria from the scientific literature and gathered a collection of 46 static analysis tools complying with our inclusion and exclusion criteria—a representative set of mainly non-proprietary tools. Then, we evaluated how well these tools fulfill the aforementioned criteria.

The evaluation shows that more than half of the considered tools offer poor warning messages, while about three-quarters of the tools provide hardly any fix support. Furthermore, the integration of user knowledge is strongly neglected, which could be used for improved handling of false positives and tuning the results for the corresponding developer. Finally, issues regarding workflow integration and specialized user interfaces are proved further.

These findings should prove useful in guiding and focusing further research and development in the area of user experience for static code analyses.

Wed 20 Jul

Displayed time zone: Seoul change

16:20 - 17:40
Session 3-1: Static Analysis and Specifications Testing CTechnical Papers at ISSTA 1
Chair(s): Ding Li Peking University
16:20
20m
Talk
A Large-scale Study of Usability Criteria addressed by Static Analysis Tools
Technical Papers
Marcus Nachtigall Heinz Nixdorf Institute, Paderborn University, Michael Schlichtig Heinz Nixdorf Institute, Paderborn University, Eric Bodden University of Paderborn; Fraunhofer IEM
DOI
16:40
20m
Talk
An Empirical Study on the Effectiveness of Static C/C++ Analyzers for Vulnerability Detection
Technical Papers
Stephan Lipp Technical University of Munich, Sebastian Banescu Technical University of Munich, Alexander Pretschner TU Munich
DOI Pre-print
17:00
20m
Talk
Combining Static Analysis Error Traces with Dynamic Symbolic Execution (Experience Paper)
Technical Papers
Frank Busse Imperial College London, Pritam Gharat Imperial College London, Cristian Cadar Imperial College London, UK, Alastair F. Donaldson Imperial College London
DOI Pre-print
17:20
20m
Talk
Path-Sensitive Code Embedding via Contrastive Learning for Software Vulnerability Detection
Technical Papers
Xiao Cheng University of Technology Sydney, Guanqin Zhang University of Technology Sydney, Haoyu Wang Huazhong University of Science and Technology, China, Yulei Sui University of New South Wales
DOI

Thu 21 Jul

Displayed time zone: Seoul change

03:00 - 04:00
Session 1-7: Static Analysis and Specifications Testing ATechnical Papers at ISSTA 1
Chair(s): Raghavan Komondoor IISc Bengaluru
03:00
20m
Talk
A Large-scale Study of Usability Criteria addressed by Static Analysis Tools
Technical Papers
Marcus Nachtigall Heinz Nixdorf Institute, Paderborn University, Michael Schlichtig Heinz Nixdorf Institute, Paderborn University, Eric Bodden University of Paderborn; Fraunhofer IEM
DOI
03:20
20m
Talk
An Empirical Study on the Effectiveness of Static C/C++ Analyzers for Vulnerability Detection
Technical Papers
Stephan Lipp Technical University of Munich, Sebastian Banescu Technical University of Munich, Alexander Pretschner TU Munich
DOI Pre-print
03:40
20m
Talk
Combining Static Analysis Error Traces with Dynamic Symbolic Execution (Experience Paper)
Technical Papers
Frank Busse Imperial College London, Pritam Gharat Imperial College London, Cristian Cadar Imperial College London, UK, Alastair F. Donaldson Imperial College London
DOI Pre-print