Towards a Benchmark for Dependency Decision-Making
AI coding agents increasingly modify real software repositories and make dependency decisions, including adding, removing, or updating third-party packages. These choices can materially affect security posture and maintenance burden, yet repository-level evaluations largely emphasize test passing and executability without explicitly scoring whether systems (i) reuse existing dependencies, (ii) avoid unnecessary additions, or (iii) select versions that satisfy security and policy constraints.
We propose DepDec-Bench, a benchmark for evaluating dependency decision-making beyond functional correctness. To ground DepDec-Bench in real-world behavior, we conduct a preliminary study of 117,062 dependency changes from agent- and human-authored pull requests across seven ecosystems. We show that coding agents frequently make dependency decisions with security consequences that remain invisible to test-focused evaluation: agents select PR-time known-vulnerable versions (2.46%) and exhibit net-negative security impact overall (net impact −98 vs. +1,316 for humans). These observations inform DepDec-Bench task families and metrics that evaluate safe version selection, reuse discipline, and restraint against dependency bloat alongside test passing.
Mon 13 AprDisplayed time zone: Brasilia, Distrito Federal, Brazil change
16:00 - 17:30 | |||
16:00 5mTalk | A Survey on AI-enabled Software Logging From Generation to Intelligent Analysis of Software Logs Journal Ahead Workshop (JAWs) Cong Wang Zhejiang University, China, Yaowei Sun Tianjin University of Science and Technology, xinguohua Tianjin University, Jiongchi Yu Singapore Management University, Guangquan Xu School of Cybersecurity, Tianjin University | ||
16:05 5mTalk | Towards a Benchmark for Dependency Decision-Making Journal Ahead Workshop (JAWs) Tanmay Singla Purdue University, Berk Çakar Purdue University, Paschal Amusuo Purdue University, James C. Davis Purdue University Pre-print | ||
16:10 5mTalk | A UX Perspective on the Role of Code Search in Mitigating Divergences in Software Design Journal Ahead Workshop (JAWs) Sofia Orellano Abal University of Falmouth, Ketai Qiu USI Università della Svizzera Italiana, Chengcheng Qu University of Falmouth, Luca Di Grazia University of St. Gallen | ||
16:15 5mTalk | Post-Velocity Software Engineering: When Change Throughput Outruns Assurance Capacity Journal Ahead Workshop (JAWs) Alessio Bucaioni Mälardalen University | ||
16:20 5mTalk | Automated Refinement of Behaviour Abstractions with Inferred Assertions Journal Ahead Workshop (JAWs) Nazareno Aguirre University of Rio Cuarto/CONICET, Argentina, and Guangdong Technion-Israel Institute of Technology, China, Diego Garbervetsky Departamento de Computación, FCEyN, UBA, Sebastian Uchitel Universidad de Buenos Aires / Imperial College, Tao Yue Beihang University, Man Zhang Beihang University, China | ||
16:25 5mTalk | Advances in Software Metrics Thresholds Definition - a Rigorous Statistical Approach Journal Ahead Workshop (JAWs) Diego Gonçalves Federal Center for Technological Education of Minas Gerais, Kecia Ferreira Federal Center for Technological Education of Minas Gerais, Guilherme Oliveira Federal Center for Technological Education of Minas Gerais | ||
16:30 5mTalk | Characterizing Software Leadership: A Mixed-Methods Approach from Social Media to Industry Journal Ahead Workshop (JAWs) Murilo Coelho Universidade Estadual do Ceará, Denivan Campos University of Molise, Italy, Mariana Maia Bezerra Atlântico Institute, Matheus Paixao State University of Ceará, Allysson Allex Araújo Federal University of Cariri, Sávio Freire Federal Institute of Ceará | ||
16:35 5mTalk | Can You Help Me? An Uncertainty-Aware Self-Adaptive Delegation Approach Journal Ahead Workshop (JAWs) Matheus Chagas State University of Ceará, Lucas Alves State University of Ceara, Brazil, Paulo Maia State University of Ceará, Andrea Zisman The Open University | ||
16:40 5mTalk | QLM: CodeQL Query Synthesis with Compositional PoC Validation Journal Ahead Workshop (JAWs) weigang he University of Technology Sydney, Xiao Cheng Macquarie University, Shuangxiang Kan UNSW, Wei Li UNSW Sydney, Xingrui Yu CFAR, Agency for Science, Technology and Research (A*STAR), Ivor Tsang CFAR, Agency for Science, Technology and Research (A*STAR), Yulei Sui University of New South Wales | ||
16:45 5mTalk | Revisiting Code Clone Genealogies: A New Taxonomy Based on Developer Actions Journal Ahead Workshop (JAWs) Denis Sousa State University of Ceara, Brazil, Italo Uchoa State University of Ceará, Thiago Lima State University of Ceara, Brazil, Matheus Paixao State University of Ceará, Chaiyong Rakhitwetsagul Mahidol University, Thailand | ||
16:50 5mTalk | IDEX: LLM-Guided Debug for Semantic Mismatch in Text Corpora Journal Ahead Workshop (JAWs) Shihao Weng Nanjing University, Yang Feng Nanjing University, Yining Yin Nanjing University, Yuan Yao Nanjing University, Jia Liu Nanjing University, Zhihong Zhao Nanjing University | ||
16:55 25mAwards | Award paper presentation Journal Ahead Workshop (JAWs) | ||
17:20 10mDay closing | Closing Journal Ahead Workshop (JAWs) | ||