Write a Blog >>
ISSTA 2021
Sun 11 - Sat 17 July 2021 Online
co-located with ECOOP and ISSTA 2021
Fri 16 Jul 2021 10:00 - 10:20 at ISSTA 1 - Session 17 (time band 3) Testing Miscellaneous Systems Chair(s): Thuan Pham
Sat 17 Jul 2021 03:00 - 03:20 at ISSTA 1 - Session 23 (time band 2) Testing 4 Chair(s): Wensheng Dou

Security of smart contracts has attracted increasing attention in recent years. Many researchers have devoted themselves to devising testing tools for vulnerability detection. Each published tool has demonstrated its effectiveness through a series of evaluations on their own experimental scenarios. However, the inconsistency of evaluation settings such as different data sets or performance metrics, may result in biased conclusion.

In this paper, based on an empirical evaluation of widely used smart contract testing tools, we propose a unified standard to eliminate the bias in the assessment process. First, we collect 46,186 source-available smart contracts from four influential organizations. This comprehensive dataset is open to the public and involves different code characteristics, vulnerability patterns and application scenarios. Then we propose a 4-step evaluation process and summarize the difference among relevant work in these steps. We use nine representative tools to carry out extensive experiments. The results demonstrate that different choices of experimental settings could significantly affect tool performance and lead to misleading or even opposite conclusions. Finally, we generalize some problems of existing testing tools, and propose some possible directions for further improvement.

slides (issta21-p173-slides.pptx)1.67MiB

Fri 16 Jul

Displayed time zone: Brussels, Copenhagen, Madrid, Paris change

10:00 - 11:20
Session 17 (time band 3) Testing Miscellaneous Systems Technical Papers at ISSTA 1
Chair(s): Thuan Pham The University of Melbourne
10:00
20m
Talk
Empirical Evaluation of Smart Contract Testing: What Is the Best Choice?
Technical Papers
Meng Ren Tsinghua University, Zijing Yin Tsinghua University, Fuchen Ma Tsinghua University, Zhenyang Xu University of Waterloo, Yu Jiang Tsinghua University, Chengnian Sun University of Waterloo, Huizhong Li WeBank, Yan Cai Institute of Software at Chinese Academy of Sciences
DOI File Attached
10:20
20m
Talk
Interval Constraint-Based Mutation Testing of Numerical Specifications
Technical Papers
Clothilde Jeangoudoux MPI-SWS, Eva Darulova MPI-SWS, Christoph Lauter University of Alaska at Anchorage
DOI
10:40
20m
Talk
TERA: Optimizing Stochastic Regression Tests in Machine Learning Projects
Technical Papers
Saikat Dutta University of Illinois at Urbana-Champaign, Jeeva Selvam University of Illinois at Urbana-Champaign, Aryaman Jain University of Illinois at Urbana-Champaign, Sasa Misailovic University of Illinois at Urbana-Champaign
DOI
11:00
20m
Talk
Predoo: Precision Testing of Deep Learning Operators
Technical Papers
Xufan Zhang Nanjing University, Ning Sun Nanjing University, Chunrong Fang Nanjing University, Jiawei Liu Nanjing University, Jia Liu Nanjing University, Dong Chai Huawei, Jiang Wang Huawei, Zhenyu Chen Nanjing University
DOI

Sat 17 Jul

Displayed time zone: Brussels, Copenhagen, Madrid, Paris change

02:40 - 03:40
Session 23 (time band 2) Testing 4Technical Papers at ISSTA 1
Chair(s): Wensheng Dou Institute of Software at Chinese Academy of Sciences; University of Chinese Academy of Sciences
02:40
20m
Talk
Validating Static Warnings via Testing Code Fragments
Technical Papers
Ashwin Kallingal Joshy Iowa State University, Xueyuan Chen Iowa State University, Benjamin Steenhoek Iowa State University, Wei Le Iowa State University
DOI
03:00
20m
Talk
Empirical Evaluation of Smart Contract Testing: What Is the Best Choice?
Technical Papers
Meng Ren Tsinghua University, Zijing Yin Tsinghua University, Fuchen Ma Tsinghua University, Zhenyang Xu University of Waterloo, Yu Jiang Tsinghua University, Chengnian Sun University of Waterloo, Huizhong Li WeBank, Yan Cai Institute of Software at Chinese Academy of Sciences
DOI File Attached
03:20
20m
Talk
Model-Based Testing of Networked Applications
Technical Papers
Yishuai Li University of Pennsylvania, Benjamin C. Pierce University of Pennsylvania, Steve Zdancewic University of Pennsylvania
DOI Pre-print