Read It, Don't Watch It: Captioning Bug Recordings Automatically
Screen recordings of mobile applications are easy to capture and include a wealth of information, making them a popular mechanism for users to inform developers of the problems encountered in the bug reports. However, watching the bug recordings and efficiently understanding the semantics of user actions can be time-consuming and tedious for developers. Inspired by the conception of the video subtitle in movie industry, we present a lightweight approach CAPdroid to caption bug recordings automatically. CAPdroid is a purely image-based and non-intrusive approach by using image processing and convolutional deep learning models to segment bug recordings, infer user action attributes, and create descriptions as subtitles. The automated experiments demonstrate the good performance of CAPdroid to infer user actions from the recordings, and a user study confirms the usefulness of our generated step descriptions in assisting developers with bug replay.
Fri 19 MayDisplayed time zone: Hobart change
13:45 - 15:15 | Issue reporting and reproductionTechnical Track / DEMO - Demonstrations at Meeting Room 110 Chair(s): Daniel Russo Department of Computer Science, Aalborg University | ||
13:45 15mTalk | Incident-aware Duplicate Ticket Aggregation for Cloud Systems Technical Track Jinyang Liu The Chinese University of Hong Kong, Shilin He Microsoft Research, Zhuangbin Chen Chinese University of Hong Kong, China, Liqun Li Microsoft Research, Yu Kang Microsoft Research, Xu Zhang Microsoft Research, Pinjia He Chinese University of Hong Kong at Shenzhen, Hongyu Zhang The University of Newcastle, Qingwei Lin Microsoft Research, Zhangwei Xu Microsoft Azure, Saravan Rajmohan Microsoft 365, Dongmei Zhang Microsoft Research, Michael Lyu The Chinese University of Hong Kong | ||
14:00 15mTalk | Large Language Models are Few-shot Testers: Exploring LLM-based General Bug Reproduction Technical Track Pre-print | ||
14:15 15mTalk | On the Reproducibility of Software Defect Datasets Technical Track | ||
14:30 15mTalk | Context-aware Bug Reproduction for Mobile Apps Technical Track Yuchao Huang , Junjie Wang Institute of Software at Chinese Academy of Sciences; University of Chinese Academy of Sciences, Zhe Liu Institute of Software, Chinese Academy of Sciences, Song Wang York University, Chunyang Chen Monash University, Mingyang Li Institute of Software at Chinese Academy of Sciences; University of Chinese Academy of Sciences, Qing Wang Institute of Software at Chinese Academy of Sciences; University of Chinese Academy of Sciences | ||
14:45 15mTalk | Read It, Don't Watch It: Captioning Bug Recordings Automatically Technical Track Sidong Feng Monash University, Mulong Xie Australian National University, Yinxing Xue University of Science and Technology of China, Chunyang Chen Monash University Pre-print | ||
15:00 7mTalk | BURT: A Chatbot for Interactive Bug Reporting DEMO - Demonstrations Yang Song College of William and Mary, Junayed Mahmud George Mason University, Nadeeshan De Silva William & Mary, Ying Zhou University of Texas at Dallas, Oscar Chaparro College of William and Mary, Kevin Moran George Mason University, Andrian Marcus University of Texas at Dallas, Denys Poshyvanyk College of William and Mary |