Recent years have experienced growing contributions of AI coding agents that assist human developers in various software engineering tasks. However, this growing AI-assisted autonomy raises questions about security and trust.
In this paper, we analyze more than 33,000 AI-generated pull requests (PRs) and identify 675 security-related submissions made by agentic AIs. Then we examine the security-related PRs with a focus on recurring security weaknesses, review outcomes and latency, commit message quality, and rejection reasons. The results show that security-related AI PRs introduce a small set of recurring weaknesses such as regex inefficiencies, injection flaws, and path traversal. Many flawed contributions are still merged, while rejections often arise from social or process factors such as inactivity or missing test coverage. The commit message quality of AI PRs has a limited effect on acceptance or latency, in contrast to human PRs reported in previous studies. We also extend existing rejection taxonomies by adding categories that are unique to AI-generated security contributions. These findings offer new insights into the strengths and shortcomings of autonomous coding systems in secure software development.
Wed 10 JunDisplayed time zone: London change
13:30 - 15:00 | Security 2Research Papers / AI Models / Data at JMS 745 Chair(s): Minhaz Zibran Idaho State University | ||
13:30 15mTalk | Reassessing Dataset Quality and Fine-Tuning Practices for Vulnerability Detection on PrimeVul dataset AI Models / Data Norbert Szolnoki Sándor Department of Software Engineering, University of Szeged, Gabor Antal Department of Software Engineering, University of Szeged | ||
13:45 15mTalk | A Neuro-Symbolic Risk Calculus for Quantifying Security Posture in Microservice Systems Research Papers | ||
14:00 15mTalk | Insights into Security-Related AI-Generated Pull Requests Research Papers Md Fazle Rabbi Idaho State University, Asif Kamal Turzo University of Massachusetts Dartmouth, Arifa Islam Champa Idaho State University, Minhaz Zibran Idaho State University Pre-print | ||
14:15 15mTalk | Do Privacy Policies Match with the Logs? An Empirical Study of Privacy Disclosure in Android Application Logs Research Papers Zhiyuan Chen Rochester Institute of Technology, Love Jayesh Ahir Rochester Institute of Technology, Ahmad Suleiman Rochester Institute of Technology (RIT), Kundi Yao Ontario Tech University, Yiming Tang Rochester Institute of Technology, Weiyi Shang University of Waterloo, Daqing Hou pc Pre-print | ||