ScanCoder: Leveraging Human Attention Patterns to Enhance LLMs for Code
This program is tentative and subject to change.
Code comprehension is a fundamental challenge in software engineering that impacts developer productivity and software quality. While Large Language Models (LLMs) demonstrate strong capabilities in code generation and summarization, they process code differently from human developers, who employ strategic attention patterns focused on semantically critical elements. Recent research has successfully integrated human attention patterns captured through eye-tracking into AI models for software engineering tasks, however, existing human-AI approaches face critical limitations that prevent widespread practical deployment, particularly for LLM enhancement. Existing approaches to incorporate human cognitive insights face scalability limitations due to resource-intensive eye-tracking studies and lack empirical validation for cross-language generalizability.
We present \textit{ScanCoder}, a framework that integrates cognitive simulation with LLM enhancement through (1) generating human-like attention patterns at scale using minimal eye-tracking data via cognitive simulation with Adaptive Character of Thought-Rational (ACT-R) architecture, and (2) cognitively-guided fine-tuning that emphasizes tokens according to their cognitive salience and attention order. Our approach demonstrates cross-language transfer by applying C++-derived cognitive patterns to enhance Java programming tasks. Comprehensive evaluation on CodeXGLUE benchmarks shows consistent improvements across different LLM architectures, achieving gains of 39% for CrystalBLEU completion metrics and 22% for BERTScore summarization. Mechanistic analysis reveals that cognitive guidance reshapes model attention in task-dependent ways, increasing focus on semantically critical tokens by 2.5×. This work establishes the first scalable framework for integrating simulated human cognitive patterns into LLM training, enabling more interpretable and effective code understanding.
This program is tentative and subject to change.
Thu 9 JulDisplayed time zone: Eastern Time (US & Canada) change
14:00 - 15:30 | |||
14:00 20mTalk | Not All RAGs Are Created Equal: A Component-Wise Empirical Study for Software Engineering Tasks Research Papers Qiang Ke Huazhong University of Science and Technology, Yanjie Zhao Huazhong University of Science and Technology, Hongjin Leng Xiamen University Malaysia, Shengming Zhao Fudan University, Haoyu Wang Huazhong University of Science and Technology | ||
14:20 20mTalk | ScanCoder: Leveraging Human Attention Patterns to Enhance LLMs for Code Research Papers Yueke Zhang Vanderbilt University, Yifan Zhang Vanderbilt University, Zihan Fang Vanderbilt University, Greg Trafton Naval Research Laboratory, Daniel Levin Vanderbilt University, Kevin Leach Vanderbilt University, Yu Huang Vanderbilt University | ||
14:40 20mTalk | CodeUltraFeedback: An LLM-as-a-Judge Dataset for Aligning Large Language Models to Coding Preferences Journal-First Paper Martin Weyssow DIRO, Université de Montréal, Aton Kamanda DIRO, Université de Montréal, Xin Zhou Singapore Management University, Singapore, Houari Sahraoui DIRO, Université de Montréal | ||
15:00 20mTalk | Empirical Studies of Parameter Efficient Methods for Large Language Models of Code and Knowledge Transfer to R Journal-First Paper Amirreza Esmaeili University of British Columbia, Iman Saberi University of British Columbia Okanagan, Fatemeh Hendijani Fard University of British Columbia, Okanagan | ||
15:20 10mTalk | AutoChecklist: Automated Checklist Refinement for LLM Judges Industry Papers Mansi Uniyal Microsoft, Mukul Singh Microsoft, Gust Verbruggen Microsoft, Vu Le Microsoft, Sumit Gulwani Microsoft | ||