Testing False Recalls in E-commerce Apps: a User-perspective Blackbox Approach
This program is tentative and subject to change.
Search components are essential in e-commerce apps, allowing users to find products and services. However, they often suffer from bugs, leading to false recalls, \textit{i.e.}, irrelevant search results. Detecting false recalls automatically is challenging. As users and shop owners adopt ambiguous natural language to describe their purchasing intentions and products, precise relevance determination becomes difficult. We propose \textbf{f}alse \textbf{r}ecall \textbf{H}ound (frHound), a black box testing approach targeting false recalls. The core idea of \MIG is to mimic users’ online purchasing behavior. Specifically, frHound first designs 37 features to align with how users process information during online shopping, explored by a comprehensive user study. Then, frHound uses an outlier detection technique to identify the most divergent search results, similar to how general users make purchasing decisions during online shopping. Those divergent search results are likely false recalls, as most search results are relevant during e-commerce searches. Experiments with real industry data show frHound reduces human labor, time, and financial costs associated with discovering false recalls by 36.74 times. In a seven-month trial with \textit{M-app}, a popular Chinese e-commerce platform, frHound identified 1282 false recalls, improving user satisfaction and reducing false recall discovery costs.