Understanding and Finding System Setting-Related Defects in Android Apps
Fri 16 Jul 2021 10:20 - 10:40 at ISSTA 2 - Session 18 (time band 3) Android Application Testing 2 Chair(s): Shin Hwei Tan
Android, the most popular mobile system, offers a number of user-configurable system settings (e.g., network, location, and permission) for controlling devices and apps. Even popular, well-tested apps may fail to properly adapt their behaviors to diverse setting changes, thus frustrating their users. However, there exists no effort to systematically investigate such defects. To this end, we conduct the first empirical study to understand the characteristics of these setting-related defects (in short as “setting defects”), which reside in apps and are triggered by system setting changes.We devote substantial manual effort (over three person-months) to analyze 1,074 setting defects from 180 popular apps on GitHub. We investigate their impact, root causes and consequences. We find that setting defects have wide, diverse impact on apps’ correctness, and the majority of them (=70.7%) are non-crash (logic) bugs, thus could not be automatically detected by existing app testing techniques due to the lack of test oracles.
Motivated and guided by our study, we introduce setting-wise metamorphic fuzzing, the first automated testing approach to effectively detecting setting defects without explicit oracles. Our key insight is that an app’s behavior should, in most cases, remain consistent if a given setting is changed and later properly restored, or exhibit expected differences if not restored. We realize our approach in SetDroid, an automated, end-to-end GUI testing tool, for detecting both crash and non-crash setting defects. SetDroid has been evaluated on 26 popular, open-source apps and detected 42 unique, previously unknown setting defects in 24 apps. To date, 33 have been confirmed and 21 fixed. We also apply SetDroid on five highly popular industrial apps, namely WeChat, QQMail, TikTok, CapCut and AlipayHK, all of which have billions of monthly active users. SetDroid successfully detects 17 previously unknown setting defects in these apps’ latest releases — all defects have been confirmed and fixed by the app vendors. The majority of SetDroid-detected defects (49 out of 59) are non-crash bugs, which could not be detected by existing testing tools (as our evaluation confirms). These results clearly demonstrate SetDroid’s strong effectiveness and practicality.
Thu 15 JulDisplayed time zone: Brussels, Copenhagen, Madrid, Paris change
00:20 - 01:20 | Session 5 (time band 2) Android Application Analysis and TestingTechnical Papers at ISSTA 1 Chair(s): William G.J. Halfond University of Southern California | ||
00:20 20mTalk | GUIDER: GUI Structure and Vision Co-Guided Test Script Repair for Android Apps Technical Papers Tongtong Xu Nanjing University, Minxue Pan Nanjing University, Yu Pei Hong Kong Polytechnic University, Guiyin Li Nanjing University, Xia Zeng Tencent, Tian Zhang Nanjing University, Yuetang Deng Tencent, Xuandong Li Nanjing University DOI | ||
00:40 20mTalk | Understanding and Finding System Setting-Related Defects in Android Apps Technical Papers Jingling Sun East China Normal University, Ting Su East China Normal University, Junxin Li East China Normal University, Zhen Dong National University of Singapore, Geguang Pu East China Normal University, Tao Xie Peking University, Zhendong Su ETH Zurich DOI Media Attached | ||
01:00 20mTalk | HomDroid: Detecting Android Covert Malware by Social-Network Homophily Analysis Technical Papers Yueming Wu Huazhong University of Science and Technology, Deqing Zou Huazhong University of Science and Technology, Wei Yang University of Texas at Dallas, Xiang Li Huazhong University of Science and Technology, Hai Jin Huazhong University of Science and Technology DOI | ||
Fri 16 JulDisplayed time zone: Brussels, Copenhagen, Madrid, Paris change
10:00 - 11:20 | Session 18 (time band 3) Android Application Testing 2Technical Papers at ISSTA 2 Chair(s): Shin Hwei Tan Southern University of Science and Technology | ||
10:00 20mTalk | GUIDER: GUI Structure and Vision Co-Guided Test Script Repair for Android Apps Technical Papers Tongtong Xu Nanjing University, Minxue Pan Nanjing University, Yu Pei Hong Kong Polytechnic University, Guiyin Li Nanjing University, Xia Zeng Tencent, Tian Zhang Nanjing University, Yuetang Deng Tencent, Xuandong Li Nanjing University DOI | ||
10:20 20mTalk | Understanding and Finding System Setting-Related Defects in Android Apps Technical Papers Jingling Sun East China Normal University, Ting Su East China Normal University, Junxin Li East China Normal University, Zhen Dong National University of Singapore, Geguang Pu East China Normal University, Tao Xie Peking University, Zhendong Su ETH Zurich DOI Media Attached | ||
10:40 20mTalk | Parema: An Unpacking Framework for Demystifying VM-Based Android Packers Technical Papers Lei Xue Hong Kong Polytechnic University, Yuxiao Yan Xi'an Jiaotong University; Hong Kong Polytechnic University, Luyi Yan Hong Kong Polytechnic University, Muhui Jiang Hong Kong Polytechnic University, Xiapu Luo Hong Kong Polytechnic University, Dinghao Wu Pennsylvania State University, Yajin Zhou Zhejiang University DOI | ||
11:00 20mTalk | HomDroid: Detecting Android Covert Malware by Social-Network Homophily Analysis Technical Papers Yueming Wu Huazhong University of Science and Technology, Deqing Zou Huazhong University of Science and Technology, Wei Yang University of Texas at Dallas, Xiang Li Huazhong University of Science and Technology, Hai Jin Huazhong University of Science and Technology DOI | ||