ISSTA 2022
Mon 18 - Fri 22 July 2022 Online
Thu 21 Jul 2022 07:20 - 07:40 at ISSTA 2 - Session 2-8: Concurrency, IoT, Embedded B
Thu 21 Jul 2022 16:40 - 17:00 at ISSTA 1 - Session 3-5: Concurrency, IoT, Embedded C Chair(s): Stefan Winter

Image captioning (IC) systems, which automatically generate a text description of the salient objects in an image (real or synthetic), have seen great progress over the past few years due to the development of deep neural networks. IC plays an indispensable role in human society, for example, labeling massive photos for scientific studies and assisting visually-impaired people in perceiving the world. However, even the top-notch IC systems, such as Microsoft Azure Cognitive Services and IBM Image Caption Generator, may return incorrect results, leading to the omission of important objects, deep misunderstanding, and threats to personal safety.

To address this problem, we propose MetaIC, the \textit{first} metamorphic testing approach to validate IC systems. Our core idea is that the key caption constituents should exhibit directional changes after object insertion. Specifically, MetaIC (1) extracts objects from existing images to construct an object corpus; (2) inserts an object into an image via novel object resizing and location tuning algorithms; and (3) reports image pairs whose captions do not exhibit differences in an expected way. In our evaluation, we use MetaIC to test one widely-adopted image captioning API and five state-of-the-art (SOTA) image captioning models. Using 1,000 seeds, MetaIC successfully reports 16,825 erroneous issues with high precision (84.9%-98.4%). There are three kinds of errors: misclassification, omission, and incorrect quantity. We visualize the errors reported by MetaIC, which shows that flexible overlapping setting facilitates IC testing by increasing and diversifying the reported errors. In addition, MetaIC can be further generalized to detect label errors in the training dataset, which has successfully detected 151 incorrect labels in MS COCO Caption, a standard dataset in image captioning.

Thu 21 Jul

Displayed time zone: Seoul change

07:00 - 08:00
Session 2-8: Concurrency, IoT, Embedded BTechnical Papers at ISSTA 2
07:00
20m
Talk
A Large-Scale Empirical Analysis of the Vulnerabilities Introduced by Third-party Components in IoT Firmware
Technical Papers
Binbin Zhao Georgia Institute of Technology, Shouling Ji Zhejiang University, Jiacheng Xu Zhejiang University, Yuan Tian University of Virginia, Qiuyang Wei Zhejiang University, Qinying Wang Zhejiang University, Chenyang Lyu Zhejiang University, Xuhong Zhang Zhejiang University, Changting Lin Binjiang Institute of Zhejiang University, Jingzheng Wu Institute of Software, The Chinese Academy of Sciences, Raheem Beyah Georgia Institute of Technology
DOI
07:20
20m
Talk
Automated Testing of Image Captioning Systems
Technical Papers
BoXi Yu The Chinese University of Hong Kong, Shenzhen, Zhiqing Zhong South China University of Technology, Xinran Qin South China University of Technology, Jiayi Yao The Chinese University of Hong Kong, Shenzhen, Yuancheng Wang The Chinese University of Hong Kong, Shenzhen, Pinjia He The Chinese University of Hong Kong, Shenzhen
DOI
07:40
20m
Talk
LiRTest: Augmenting LiDAR Point Clouds for Automated Testing of Autonomous Driving Systems
Technical Papers
Guo An Nanjing University, Yang Feng Nanjing University, Zhenyu Chen Nanjing University
DOI
16:20 - 17:40
Session 3-5: Concurrency, IoT, Embedded CTechnical Papers at ISSTA 1
Chair(s): Stefan Winter LMU Munich
16:20
20m
Talk
Understanding Device Integration Bugs in Smart Home System
Technical Papers
Tao Wang , Kangkang Zhang Institute of Software Chinese Academy of Sciences, Wei Chen Institute of Software at Chinese Academy of Sciences, China, Wensheng Dou Institute of Software at Chinese Academy of Sciences; University of Chinese Academy of Sciences, Jiaxin Zhu Institute of Software at Chinese Academy of Sciences, China, Jun Wei Institute of Software at Chinese Academy of Sciences; University of Chinese Academy of Sciences, Tao Huang Institute of Software Chinese Academy of Sciences
DOI
16:40
20m
Talk
Automated Testing of Image Captioning Systems
Technical Papers
BoXi Yu The Chinese University of Hong Kong, Shenzhen, Zhiqing Zhong South China University of Technology, Xinran Qin South China University of Technology, Jiayi Yao The Chinese University of Hong Kong, Shenzhen, Yuancheng Wang The Chinese University of Hong Kong, Shenzhen, Pinjia He The Chinese University of Hong Kong, Shenzhen
DOI
17:00
20m
Talk
LiRTest: Augmenting LiDAR Point Clouds for Automated Testing of Autonomous Driving Systems
Technical Papers
Guo An Nanjing University, Yang Feng Nanjing University, Zhenyu Chen Nanjing University
DOI
17:20
20m
Talk
Precise and Efficient Atomicity Violation Detection for Interrupt-driven Programs via Staged Path Pruning
Technical Papers
Chao Li Beijing Institute of Control Engineering and Beijing Sunwise Information Technology Ltd, Rui Chen Beijing Institute of Control Engineering, Boxiang Wang Xidian University and Beijing Sunwise Information Technology Ltd, Tingting Yu Beijing Institute of Control Engineering and Beijing Sunwise Information Technology Ltd, Dongdong Gao Beijing Institute of Control Engineering and Beijing Sunwise Information Technology Ltd, Mengfei Yang China Academy of Space Technology, China
DOI