Write a Blog >>
ISSTA 2021
Sun 11 - Sat 17 July 2021 Online
co-located with ECOOP and ISSTA 2021

The most popular static taint analysis tools for Android allow users to change the underlying analysis algorithms through configuration options. However, the large configuration spaces make it difficult for developers and users alike to understand the full capabilities of these tools, and studies to-date have only focused on individual configurations. In this work, we present the first study that evaluates the configurations in Android taint analysis tools, focusing on the two most popular tools, FlowDroid and DroidSafe. First, we perform a manual code investigation to better understand how configurations are implemented in both tools. We formalize the expected effects of configuration option settings in terms of precision and soundness partial orders which we use to systematically test the configuration space. Second, we create a new dataset of 756 manually classified flows across 18 open-source real-world apps and conduct large-scale experiments on this dataset and micro-benchmarks. We observe that configurations make significant tradeoffs on the performance, precision, and soundness of both tools. The studies to-date would reach different conclusions on the tools' capabilities were they to consider configurations or use real-world datasets. In addition, we study the individual options through a statistical analysis and make actionable recommendations for users to tune the tools to their own ends. Finally, we use the partial orders to test the tool configuration spaces and detect 21 instances where options behaved in unexpected and incorrect ways, demonstrating the need for rigorous testing of configuration spaces.

Thu 15 Jul

Displayed time zone: Brussels, Copenhagen, Madrid, Paris change

19:00 - 20:20
Session 12 (time band 1) Configuration and Dependency Technical Papers at ISSTA 2
Chair(s): Jonathan Bell Northeastern University
19:00
20m
Talk
Fixing Dependency Errors for Python Build Reproducibility
Technical Papers
Suchita Mukherjee University of California at Davis, Abigail Almanza University of California at Davis, Cindy Rubio-González University of California at Davis
DOI
19:20
20m
Talk
Test-Case Prioritization for Configuration TestingACM SIGSOFT Distinguished Paper
Technical Papers
Runxiang Cheng University of Illinois at Urbana-Champaign, Lingming Zhang University of Illinois at Urbana-Champaign, Darko Marinov University of Illinois at Urbana-Champaign, Tianyin Xu University of Illinois at Urbana-Champaign
DOI
19:40
20m
Talk
The Impact of Tool Configuration Spaces on the Evaluation of Configurable Taint Analysis for Android
Technical Papers
Austin Mordahl University of Texas at Dallas, Shiyi Wei University of Texas at Dallas
DOI Pre-print
20:00
20m
Talk
Challenges and Opportunities: An In-Depth Empirical Study on Configuration Error Injection Testing
Technical Papers
Wang Li National University of Defense Technology, Zhouyang Jia National University of Defense Technology, Shanshan Li National University of Defense Technology, Yuanliang Zhang National University of Defense Technology, Teng Wang National University of Defense Technology, Erci Xu National University of Defense Technology, Ji Wang National University of Defense Technology, Xiangke Liao National University of Defense Technology
DOI File Attached

Sat 17 Jul

Displayed time zone: Brussels, Copenhagen, Madrid, Paris change

02:40 - 03:40
Session 24 (time band 2) Android and Web Application TestingTechnical Papers at ISSTA 2
Chair(s): Omer Tripp IBM Thomas J. Watson Research Center
02:40
20m
Talk
WebEvo: Taming Web Application Evolution via Detecting Semantic Structure Changes
Technical Papers
Fei Shao Case Western Reserve University, Rui Xu Case Western Reserve University, Wasif Haque University of Texas at Dallas, Jingwei Xu Peking University, Ying Zhang Peking University, Wei Yang University of Texas at Dallas, Yanfang Ye Case Western Reserve University, Xusheng Xiao Case Western Reserve University
DOI File Attached
03:00
20m
Talk
The Impact of Tool Configuration Spaces on the Evaluation of Configurable Taint Analysis for Android
Technical Papers
Austin Mordahl University of Texas at Dallas, Shiyi Wei University of Texas at Dallas
DOI Pre-print
03:20
20m
Talk
An Infrastructure Approach to Improving Effectiveness of Android UI Testing Tools
Technical Papers
Wenyu Wang University of Illinois at Urbana-Champaign, Wing Lam University of Illinois at Urbana-Champaign, Tao Xie Peking University
Link to publication DOI Media Attached