Agent-based program repair offers the promise of automatically resolving complex bugs end-to-end by combining the planning, tool usage, and code-generating abilities of modern LLMs. Recent work has explored the use of agent-based repair approaches on the popular open-source SWE-Bench, a collection of bugs (and patches) from popular GitHub-hosted Python projects. In addition, various agentic approaches such as SWE-Agent have been proposed to solve bugs in this benchmark. This paper explores the opportunity of using a similar agentic approach to address bugs in an enterprise-scale context. We perform a systematic comparison of bugs in SWE-Bench and those found in Google’s issue tracking system and show that they have different distributions in terms of language diversity, size and spread of changes, and ease of localization.

Next, we implement Passerine, an agent similar in spirit to SWE-Agent that can work within Google’s environment and produce patches for bugs in Google’s code repository. To evaluate Passerine, we curate an evaluation set of 182 bugs, spanning human-reported (82) and machine-reported bugs (100) from Google’s issue tracking system. We show that with 20 trajectory samples Passerine can produce a plausible patch for 70% of machine-reported and 14.6% of human-reported bugs in our evaluation set. After manual examination, we found that 42% of machine-reported bugs and 13.4% of human-reported bugs have at least one patch that is semantically equivalent to the ground-truth patch. This establishes a lower bound on performance that suggests agent-based APR holds promise for large-scale enterprise bug repair.