Write a Blog >>
ESEM 2021
Mon 11 - Fri 15 October 2021
Wed 13 Oct 2021 14:35 - 14:50 at ESEM ROOM - Testing & Security 3 Chair(s): Robert Feldt

Background: Executing, verifying and enforcing credible transactions on permissionless blockchains is done using smart contracts. A key challenge with smart contracts is ensuring their correctness and security. Several test input generation techniques for detecting vulnerabilities in smart contracts have been proposed in the last few years. However, a comparison of proposed techniques to gauge their effectiveness is missing.

Aim: This paper conducts an empirical evaluation of testing techniques for smart contracts. The testing techniques we evaluated are: (1) Blackbox fuzzing, (2) Adaptive fuzzing, (3) Coverage-guided fuzzing with an SMT solver and (4) Genetic algorithm. We do not consider static analysis tools, as several recent studies have assessed and compared effectiveness of these tools.

Method: We evaluate effectiveness of the test generation techniques using (1) Coverage achieved - we use four code coverage metrics targeting smart contracts, (2) Fault finding ability - using artificially seeded and real security vulnerabilities of different types. We used two datasets in our evaluation - one with 1665 real smart contracts from Etherscan, and another with 90 real contracts with known vulnerabilities to assess fault finding ability.

Result: We find Adaptive fuzzing performs best in terms of coverage and fault finding over contracts in both datasets.

Conclusion: However, we believe considering dependencies between functions and handling Solidity specific features will help improve the performance of all techniques considerably.

Wed 13 Oct

Displayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

14:20 - 15:15
Testing & Security 3Emerging Results and Vision papers / Journal-first Papers / Technical Papers at ESEM ROOM
Chair(s): Robert Feldt Chalmers University of Technology, Sweden
On (Mis)Perceptions of Testing Effectiveness: An Empirical Study
Journal-first Papers
Sira Vegas Universidad Politecnica de Madrid, Patricia Riofrio , Esperanza Marcos Universidad Rey Juan Carlos, Natalia Juristo Universidad Politecnica de Madrid
Testing Smart Contracts: Which Technique Performs Best?
Technical Papers
Sefa Akca Uniersity of Edinburgh, Chao Peng University of Edinburgh, UK, Ajitha Rajan University of Edinburgh
Automated isolation for white-box test generation
Journal-first Papers
Dávid Honfi , Zoltán Micskei Budapest University of Technology and Economics
Link to publication DOI
Contextual Understanding and Improvement of Metamorphic Testing in Scientific Software Development
Emerging Results and Vision papers
Zedong Peng University of Cincinnati, Upulee Kanewala University of North Florida, Nan Niu University of Cincinnati

Information for Participants
Wed 13 Oct 2021 14:20 - 15:15 at ESEM ROOM - Testing & Security 3 Chair(s): Robert Feldt
Info for room ESEM ROOM: