Null-pointer exceptions are serious problem for Java, and researchers have developed type-based nullness checking tools to prevent them. These tools, however, have a downside: they require developers to write nullability annotations, which is time-consuming and hinders adoption. Researchers have therefore proposed nullability annotation inference tools, whose goal is to (partially) automate the task of annotating a program for nullability. However, prior works rely on differing theories of what makes a set of nullability annotations good, making comparing their effectiveness challenging. In this work, we identify a systematic bias in some prior experimental evaluation of these tools: the use of “type reconstruction” experiments to see if a tool can recover erased developer-written annotations. We show that developers make semantic code changes while adding annotations to facilitate typechecking, leading such experiments to overestimate the effectiveness of inference tools on never-annotated code. We propose a new definition of the “best” inferred annotations for a program that avoids this bias, based on a systematic exploration of the design space. With this new definition, we perform the first head-to-head comparison of three extant nullability inference tools. Our evaluation showed the complementary strengths of the tools, and remaining weaknesses that could be addressed in future work.
Mon 23 JunDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
14:00 - 15:20 | Program Analysis 1Industry Papers / Research Papers at Cosmos 3A Chair(s): Shiyi Wei University of Texas at Dallas | ||
14:00 20mTalk | Dynamic Taint Tracking for Modern Java Virtual Machines Research Papers DOI | ||
14:20 20mTalk | HapTest: The Dynamic Analysis Framework for OpenHarmony Industry Papers Farong Liu Beihang University, Mingyi Zhou Beihang University, Yakun Zhang Peking University, Ting Su East China Normal University, Bo Sun Huawei, Jacques Klein University of Luxembourg, Xiang Gao Beihang University, Li Li Beihang University | ||
14:40 20mTalk | An Empirical Study of Suppressed Static Analysis Warnings Research Papers Huimin Hu University of Stuttgart, Yingying Wang University of British Columbia, Julia Rubin The University of British Columbia, Michael Pradel University of Stuttgart DOI | ||
15:00 20mTalk | A New Approach to Evaluating Nullability Inference Tools Research Papers Nima Karimipour University of California, Riverside, Erfan Arvan New Jersey Institute of Technology, Martin Kellogg New Jersey Institute of Technology, Manu Sridharan University of California at Riverside DOI |
Cosmos 3A is the first room in the Cosmos 3 wing.
When facing the main Cosmos Hall, access to the Cosmos 3 wing is on the left, close to the stairs. The area is accessed through a large door with the number “3”, which will stay open during the event.