Write a Blog >>
Wed 12 Oct 2022 15:10 - 15:30 at Ballroom C East - Technical Session 13 - Application Domains Chair(s): Andrea Stocco

Optical character recognition (OCR) algorithms often run slow. They may take several seconds to recognize the texts on a GUI screen, which makes OCR-based widget localization in test automation unfriendly for use, especially on GPU-free computers. This paper first concludes a common type of widget text to be located in GUI testing: label text, which are short texts in widgets like buttons, menu items, and window titles. We then investigate the characteristics of texts on a GUI screen and introduce a fast GPU-independent Label Text Screening (LTS) technique to accelerate the OCR process for label text localization. The technique opens the black box of OCR engines and uses a combination of simple methods to avoid excessive text analysis on a screen as much as possible. Experiments show that, on the subject datasets, LTS reduces the average OCR-based label text localization time to a large extent. On 4k resolution GUI screens, it keeps the localization time below 0.5 seconds for over about 60% of cases without GPU support on a normal laptop computer. In contrast, the existing CPU-based approaches built on popular OCR engines Tesseract, PaddleOCR, and EasyOCR usually need over 2 seconds to achieve the same goal on the same platform. Even with GPU acceleration, they can hardly keep the analysis time in 1 second. We believe the proposed approach would be helpful for implementing OCR-based test automation tools.

Wed 12 Oct

Displayed time zone: Eastern Time (US & Canada) change

13:30 - 15:30
Technical Session 13 - Application DomainsNIER Track / Research Papers / Journal-first Papers at Ballroom C East
Chair(s): Andrea Stocco Università della Svizzera italiana (USI)
13:30
20m
Research paper
A Hybrid Approach for Inference between Behavioral Exception API Documentation and Implementations, and Its Applications
Research Papers
Hoan Anh Nguyen Amazon, Hung Phan , Samantha Syeda Khairunnesa Bradley University, Son Nguyen The University of Texas at Dallas, Aashish Yadavally University of Texas at Dallas, Shaohua Wang New Jersey Institute of Technology, Hridesh Rajan Iowa State University, Tien N. Nguyen University of Texas at Dallas
13:50
10m
Vision and Emerging Results
Property-Based Automated Repair of DeFi Protocols
NIER Track
Palina Tolmach Nanyang Technological University, Singapore, Institute of High Performance Computing, Agency for Science, Technology and Research (A*STAR), Singapore, Yi Li Nanyang Technological University, Shang-Wei Lin Nanyang Technological University
Pre-print
14:00
20m
Research paper
Automatically Detecting Visual Bugs in HTML5 <canvas> Games
Research Papers
Finlay Macklon University of Alberta, Mohammad Reza Taesiri University of Alberta, Markos Viggiato University of Alberta, Stefan Antoszko University of Alberta, Natalia Romanova Prodigy Education, Dale Paas Prodigy Education, Cor-Paul Bezemer University of Alberta
14:20
10m
Vision and Emerging Results
Reflecting on Recurring Failures in IoT Development
NIER Track
Dharun Anandayuvaraj Purdue University, James C. Davis Purdue University, USA
14:30
20m
Research paper
Empirical Study of System Resources Abused by IoT AttackersVirtual
Research Papers
Zijing Yin Tsinghua University, Yiwen Xu Tsinghua University, Chijin Zhou Tsinghua University, Yu Jiang Tsinghua University
14:50
20m
Paper
Large-Scale Empirical Study of Inline Assembly on 7.6 Million Ethereum Smart ContractsVirtual
Journal-first Papers
Xiao Peng China EverBright Bank, Shuwei Song University of Electronic Science and Technology of China, Xiao Peng China EverBright Bank, Xiapu Luo Hong Kong Polytechnic University, Xiao Peng China EverBright Bank, Xiao Peng China EverBright Bank, Ting Chen University of Electronic Science and Technology of China, Jiachi Chen Sun Yat-Sen University, Tao Zhang Macau University of Science and Technology (MUST), Xiaosong Zhang University of Electronic Science and Technology of China
Link to publication DOI
15:10
20m
Research paper
Accelerating OCR-Based Widget Localization for Test Automation of GUI ApplicationsVirtual
Research Papers
Ju Qian Nanjing University of Aeronautics and Astronautics, Yingwei Ma Nanjing University of Aeronautics and Astronautics, Chenghao Lin Nanjing University of Aeronautics and Astronautics, Lin Chen Nanjing University