Write a Blog >>
Wed 12 Oct 2022 16:10 - 16:30 at Banquet A - Technical Session 18 - Testing II Chair(s): Darko Marinov

Software systems powering OS kernels, basebands, bootloaders, firmware, IoT or automotive build the foundation of infrastructure that billions of people rely on every day. Testing these systems is crucial, especially as their complexity grows and they are often written in unsafe languages such as C/C++.

However, testing such complex systems poses significant challenges, e.g., custom hardware for which there is no emulator, or a non-trivial setup of testing and debugging on the target device. As a result, the commonly used testing techniques and tools are not always easily applicable.

An off-target (OT) testing is a promising technique which addresses these challenges: part of the code is extracted and adapted to run on a different hardware platform with better tool support, easier debugging and higher test throughput. Unfortunately, since the process of creating an OT program has been manual, the technique did not scale well and was mostly used in an ad hoc manner.

In this paper we present a novel complex systems testing approach called Auto Off-target (AoT). Based on the information extracted from the source code and from the build process, AoT can automatically generate OT programs in C. AoT goes beyond the code generation and provides mechanisms that help to recreate and discover the program state in the OT code. The generated OTs are self-contained and independent of the original build environment. As a result, pieces of complex or embedded software can be easily run, analyzed, debugged and tested on a standard x86_64 machine.

We evaluate AoT on tens of thousands of functions selected from OS kernels, a bootloader and a network stack. We demonstrate that majority of the generated OTs can be automatically tested with fuzzing and symbolic execution. We further used AoT in a bug finding campaign and discovered seven bugs in the Android redfin and oriole kernels powering Google Pixel 5 and 6 phones.

Wed 12 Oct

Displayed time zone: Eastern Time (US & Canada) change

16:00 - 18:00
Technical Session 18 - Testing IIResearch Papers / Tool Demonstrations / Journal-first Papers at Banquet A
Chair(s): Darko Marinov University of Illinois at Urbana-Champaign
16:00
10m
Demonstration
Shibboleth: Hybrid Patch Correctness Assessment in Automated Program Repair
Tool Demonstrations
Ali Ghanbari Iowa State University, Andrian Marcus University of Texas at Dallas
16:10
20m
Research paper
Auto Off-Target: Enabling Thorough and Scalable Testing for Complex Software Systems
Research Papers
Tomasz Kuchta Samsung Electronics, Bartosz Zator Samsung Electronics
DOI Pre-print
16:30
10m
Demonstration
Maktub: Lightweight Robot System Test Creation and Automation
Tool Demonstrations
Amr Moussa North Carolina State University, John-Paul Ore North Carolina State University
16:40
20m
Paper
Cerebro: Static Subsuming Mutant Selection
Journal-first Papers
Aayush Garg University of Luxembourg, Milos Ojdanic University of Luxembourg, Renzo Degiovanni SnT, University of Luxembourg, Thierry Titcheu Chekam SES S.A. & University of Luxembourg (SnT), Mike Papadakis University of Luxembourg, Luxembourg, Yves Le Traon University of Luxembourg, Luxembourg
Link to publication DOI
17:00
20m
Research paper
Natural Test Generation for Precise Testing of Question Answering SoftwareVirtual
Research Papers
Qingchao Shen Tianjin University, Junjie Chen Tianjin University, Jie M. Zhang King's College London, Haoyu Wang College of Intelligence and Computing, Tianjin University, Shuang Liu Tianjin University, Menghan Tian College of Intelligence and Computing, Tianjin University
Pre-print
17:20
20m
Paper
GloBug: Using global data in Fault LocalizationVirtual
Journal-first Papers
Nima Miryeganeh University of Calgary, Sepehr Hashtroudi University of Calgary, Hadi Hemmati University of Calgary
Link to publication DOI
17:40
20m
Research paper
Selectively Combining Multiple Coverage Goals in Search-Based Unit Test GenerationVirtual
Research Papers
Zhichao Zhou School of Information Science and Technology, ShanghaiTech University, Yuming Zhou Nanjing University, Chunrong Fang Nanjing University, Zhenyu Chen Nanjing University, Yutian Tang ShanghaiTech University
DOI Pre-print