Incremental Program Analysis (IPA) has gained increasing attention as an effective technique for maintaining up-to-date code insights by leveraging previously computed analysis results in response to program modifications. Consequently, a variety of IPA algorithms and tools have been proposed. However, their empirical performance in practical, real-world scenarios remains insufficiently investigated. To address this gap, this study presents a comprehensive examination of the current state-of-the-art in IPA evaluation. Specifically, we identify two critical limitations: (1) the lack of standardized benchmarks reflecting real-world program changes, and (2) the inadequacy and uneven distribution of evaluation metrics.

To overcome these challenges, we propose an automated pipeline for constructing realistic program change benchmarks and develop a unified evaluation framework compatible with a range of IPA tools. Using this infrastructure, we curated a large-scale benchmark dataset comprising 4,084 real-world program changes drawn from 20 open-source Java projects, and systematically evaluated two representative IPA tools. The results demonstrate that, although incremental analysis substantially improves efficiency compared to exhaustive analysis, existing IPA tools exhibit inconsistencies and markedly higher peak memory consumption. Finally, we distill practical insights from our findings to inform future research and development in the field of incremental program analysis.