Search-Based Detection of Deviation Failures in the Migration of Legacy Spreadsheet Applications
Many legacy financial applications exist as a collection of formulas implemented in spreadsheets. Migration of such code to a full-fledged system, written in a language such as Java, is an error-prone process. While small differences in the outputs of numerical calculations produced by the two artifacts are tolerated and expected in practice, large discrepancies could have serious financial implications. Such discrepancies are likely due to faults in the migrated implementation, manifesting themselves in what we refer to as deviation failures. In this paper, we introduce a search-based technique that seeks to uncover deviation failures in migrated code automatically, guided by the numerical difference in the output of an original spreadsheet compared to its migration. We evaluate different variants of this approach on two financial applications, produced by COMPANY [name redacted for double-blind review], who migrated their system from a Microsoft Excel spreadsheet to a Java application. Our evaluation involves 40 formulas with known and previously unknown faults in the Java code, which were accidentally introduced by developers during the migration process. While traditional random and branch coverage-based test generation techniques were only respectively able to detect approximately 25% and 31.66% of the faults in the migrated code, the search-based approaches proposed in this paper detected up to 70% of faults with the same test generation budget. Without restriction of the search budget, up to 90% of the known deviation failures were detected. In addition, three previously unknown faults were detected by this approach which were confirmed by the COMPANY experts.
Wed 18 JulDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
11:00 - 12:30 | Porting and RepairISSTA Technical Papers at Zurich II Chair(s): Julian Dolby IBM Thomas J. Watson Research Center | ||
11:00 20mTalk | Search-Based Detection of Deviation Failures in the Migration of Legacy Spreadsheet Applications ISSTA Technical Papers Mohammad M. Almasi University of Manitoba, Hadi Hemmati University of Calgary, Gordon Fraser University of Passau, Phil McMinn University of Sheffield, Janis Benefelds SEB Life and Pensions Holding AB | ||
11:20 20mTalk | Making Data-Driven Porting Decisions with Tuscan ISSTA Technical Papers Kareem Khazem University College London, Earl T. Barr University College London, Petr Hosek Google, Inc. | ||
11:40 20mTalk | Comparing developer-provided to user-provided tests for fault localization and automated program repair ISSTA Technical Papers René Just University of Massachusetts, USA, Chris Parnin NCSU, Ian Drosos University of California, San Diego, Michael D. Ernst University of Washington, USA | ||
12:00 20mTalk | Shaping Program Repair Space with Existing Patches and Similar Code ISSTA Technical Papers Jiajun Jiang Peking University, Yingfei Xiong Peking University, Hongyu Zhang The University of Newcastle, Qing Gao Peking University, Xiangqun Chen Peking University Pre-print | ||
12:20 10m | Q&A in groups ISSTA Technical Papers |