Write a Blog >>
LCTES 2019
Sat 22 - Fri 28 June 2019 Phoenix, Arizona, United States
co-located with PLDI 2019
Sun 23 Jun 2019 16:35 - 16:40 at 105A - Session 4: Benchmarking and In-progress Works Chair(s): Hyunok Oh

Android app testing is challenging and time-consuming because fully testing all feasible execution paths is always difficult. Apps are usually tested in two alternative ways: human testing vs. automated testing. Prior studies focused on the comparison between different automated tools. However, some fundamental questions are still unexplored, including (1) how automated testing behaves differently from human testing, and (2) whether automated testing can fully or partially substitute human testing.

This paper presents our empirical study to investigate the open questions. As Monkey has been considered as one of the best automated testing tools due to its usability and relatively good coverage metrics, we applied Monkey to five Android apps of different domains and collected the dynamic event traces. Meanwhile, we also recruited eight users to manually test the same apps and gathered the traces. By comparing the collected run-time information of both testing methods, we revealed that i.) on average, the two methods triggered similar numbers of unique events; ii.) Monkey triggered system events more effectively while humans triggered more UI events; iii.) Monkey could mimic human behaviors when apps have UIs full of clickable widgets to trigger logically independent events; and iv.) Monkey was insufficient to test apps that require for information comprehension and problem-solving skills. Our research will shed light on future research that combines human expertise with the agility of Monkey testing.

Sun 23 Jun

LCTES-2019-papers
16:00 - 16:45: LCTES 2019 - Session 4: Benchmarking and In-progress Works at 105A
Chair(s): Hyunok OhHanyang Univ
LCTES-2019-papers16:00 - 16:15
Full-paper
Kyle DaruwallaUniversity of Wisconsin – Madison, Heng ZhuoUniversity of Wisconsin - Madison, Carly SchulzUniversity of Wisconsin - Madison, Mikko H. Lipasti
LCTES-2019-papers16:15 - 16:20
Short-paper
Greg StittUniversity of Florida, David CampbellUniversity of Florida
LCTES-2019-papers16:20 - 16:25
Short-paper
Andrea MaioliPolitecnico di Milano, Italy, Luca MottolaPolitecnico di Milano, Italy and RI.Se SICS, Sweden, Muhammad Hamad AlizaiLUMS, Pakistan, Junaid Haroon SiddiquiLahore University of Management Sciences
LCTES-2019-papers16:25 - 16:30
Short-paper
LCTES-2019-papers16:30 - 16:35
Short-paper
Yongseung YuHanyang University, Seokwon KangHanyang University, Yongjun ParkHanyang University
LCTES-2019-papers16:35 - 16:40
Short-paper
Mostafa MohammedVirginia Tech, Haipeng CaiWashington State University Pullman, Na MengVirginia Tech