Write a Blog >>
ICSE 2023
Sun 14 - Sat 20 May 2023 Melbourne, Australia
Fri 19 May 2023 16:15 - 16:30 at Meeting Room 102 - Metamorphic testing Chair(s): Shiva Nejati

This paper presents a data-driven debugging frame work to improve the trustworthiness of US tax preparation software systems. Given the legal implications of bugs in such software on its users, ensuring compliance and trustworthiness of tax preparation software is of paramount importance. The key barriers in developing debugging aids for tax preparation systems are the unavailability of explicit specifications and the difficulty of obtaining oracles. We posit that, since the US tax law adheres to the legal doctrine of precedent, the specifications about the outcome of tax preparation software for an individual taxpayer must be viewed in comparison with individuals that are deemed similar. Consequently, these specifications are naturally available as properties on the software requiring similar inputs provide similar outputs. Inspired by the metamorphic testing paradigm, we dub these relations metamorphic relations as they relate to structurally modified inputs.

In collaboration with legal and tax experts, we explicated metamorphic relations for a set of challenging properties from various US Internal Revenue Services (IRS) publications including Form 1040 (U.S. Individual Income Tax Return), Publication 596 (Earned Income Credit), Schedule 8812 (Qualifying Children and Other Dependents), and Form 8863 (Education Credits). While we focus on an open-source tax preparation software for our case study, the proposed framework can be readily extended to other commercial software. We develop a randomized search strategy to systematically explore the input space of software guided by metamorphic relations. We further aid this test-case generation by visually explaining the behavior of software on suspicious instances using easy-to-interpret decision-tree models. Our tool uncovered several accountability bugs with varying severity ranging from non-robust behavior in corner-cases (unreliable behavior when tax returns are close to zero) to missing eligibility conditions in the updated versions of software.

Fri 19 May

Displayed time zone: Hobart change

15:45 - 17:15
15:45
15m
Talk
MTTM: Metamorphic Testing for Textual Content Moderation Software
Technical Track
Wenxuan Wang The Chinese University of Hong Kong, Jen-tse Huang The Chinese University of Hong Kong, Weibin Wu Sun Yat-sen University, Jianping Zhang The Chinese University of Hong Kong, Yizhan Huang The Chinese University of Hong Kong, Shuqing Li The Chinese University of Hong Kong, Pinjia He Chinese University of Hong Kong at Shenzhen, Michael Lyu The Chinese University of Hong Kong
16:00
15m
Talk
Metamorphic Shader Fusion for Testing Graphics Shader Compilers
Technical Track
Dongwei Xiao The Hong Kong University of Science and Technology, Zhibo Liu Hong Kong University of Science and Technology, Shuai Wang Hong Kong University of Science and Technology
16:15
15m
Paper
Metamorphic Testing and Debugging of Tax Preparation Software
SEIS - Software Engineering in Society
Saeid Tizpaz-Niari University of Texas at El Paso, Verya Monjezi University of Texas at El Paso, Morgan Wagner University of Texas at El Paso, Shiva Darian University of Colorado Boulder, Krystia Reed University of Texas at El Paso, Ashutosh Trivedi University of Colorado Boulder
Pre-print
16:30
7m
Talk
Biasfinder: Metamorphic test generation to uncover bias for sentiment analysis systems
Journal-First Papers
Muhammad Hilmi Asyrofi School of Computing and Information Systems, Singapore Management University, Zhou Yang Singapore Management University, Imam Nur Bani Yusuf Singapore Management University, Singapore, Hong Jin Kang UCLA, Ferdian Thung Singapore Management University, David Lo Singapore Management University
16:37
7m
Talk
Automated Metamorphic Testing using Transitive Relations for Specializing Stance Detection Models
SEIP - Software Engineering in Practice
Alisa Arno IBM Research - Tokyo, Futoshi Iwama IBM Research - Tokyo, Mikio Takeuchi IBM Research - Tokyo
16:45
15m
Talk
MorphQ: Metamorphic Testing of the Qiskit Quantum Computing Platform
Technical Track
Matteo Paltenghi University of Stuttgart, Germany, Michael Pradel University of Stuttgart
Pre-print