This program is tentative and subject to change.
As the adoption of LLMs becomes more widespread in software coding ecosystems, a pressing issue has emerged: Does the generated code contain social bias and unfairness, such as those related to age, gender, and race? This issue concerns the integrity, fairness, and ethical foundation of software applications that depend on the code generated by these models but are underexplored in the literature. This paper presents a novel bias testing framework that is specifically designed for code generation tasks. Based on this framework, we conduct an extensive empirical study on the biases in code generated by five widely studied LLMs (i.e., PALM-2-CodeChat-bison, Claude-instant-1, GPT-3.5-turbo, GPT-4-turbo, and GPT-4). Our findings reveal that biases are prevalent. For example, 13.47% to 49.10% of the codes generated by these LLMs have biased behaviors towards gender. Moreover, we study five bias mitigation prompt strategies that are commonly used in current code generation scenarios, i.e., zero-shot, one-shot, few-shot, and two Chain-of-Thought (CoT) prompts, with and without provided feedback-driven refinement. Our evaluation results illustrate that using direct prompt engineering strategies has limited effectiveness in mitigating bias, but our test execution feedback can help to reduce the ratio of code biases to a large extent (e.g., from 59.88% to 4.79% for GPT-4)\footnote{This paper potentially contains offensive information for some groups.}.
This program is tentative and subject to change.
Mon 17 NovDisplayed time zone: Seoul change
14:00 - 15:30 | |||
14:00 10mTalk | QuanBench: Benchmarking Quantum Code Generation with Large Language Models Research Papers | ||
14:10 10mTalk | Token Sugar: Making Source Code Sweeter for LLMs through Token-Efficient Shorthand Research Papers Zhensu Sun Singapore Management University, Chengran Yang Singapore Management University, Singapore, Xiaoning Du Monash University, Zhou Yang University of Alberta, Alberta Machine Intelligence Institute , Li Li Beihang University, David Lo Singapore Management University | ||
14:20 10mTalk | FGIT: Fault-Guided Fine-Tuning for Code Generation Research Papers Lishui Fan Zhejiang University, Zhongxin Liu Zhejiang University, Haoye Wang Hangzhou City University, Lingfeng Bao Zhejiang University, Xin Xia Zhejiang University, Shanping Li Zhejiang University | ||
14:30 10mTalk | Mixture-of-Experts Low-Rank Adaptation for Multilingual Code Summarization Research Papers Tianchen Yu School of Software Engineering, South China University of Technology, Li Yuan School of Software Engineering, South China University of Technology, Guangzhou, China, Hailin Huang South China University of Technology, Jiexin Wang South China University of Technology, Yi Cai School of Software Engineering, South China University of Technology, Guangzhou, China | ||
14:40 10mTalk | EfficientEdit: Accelerating Code Editing via Edit-Oriented Speculative Decoding Research Papers Peiding Wang Beihang university, Li Zhang Beihang University, Fang Liu Beihang University, Yinghao Zhu Beihang University, Wang Xu Tsinghua University, Lin Shi Beihang University, Xiaoli Lian Beihang University, China, Minxiao Li Beihang university, Bo Shen Huawei Cloud Computing Technologies Co., Ltd., Binzhang Fu Huawei Technologies, n.n. Pre-print | ||
14:50 10mTalk | Bias Testing and Mitigation in LLM-based Code Generation Journal-First Track Dong Huang The University of Hong Kong, Jie M. Zhang King's College London, Qingwen Bu Shanghai Jiao Tong University, Xiaofei Xie Singapore Management University, Junjie Chen Tianjin University, Heming Cui University of Hong Kong | ||
15:00 10mTalk | FastCoder: Accelerating Repository-level Code Generation via Efficient Retrieval and Verification Research Papers Qianhui Zhao Beihang University, Li Zhang Beihang University, Fang Liu Beihang University, Xiaoli Lian Beihang University, China, Meng Qiaoyuanhe Beihang University, Ziqian Jiao Beihang University, Zetong Zhou Beihang University, Jia Li , Lin Shi Beihang University Pre-print | ||
15:10 10mTalk | AlignCoder: Aligning Retrieval with Target Intent for Repository-Level Code Completion Research Papers Tianyue Jiang Sun Yat-sen University, Yanli Wang Sun Yat-sen University, Yanlin Wang Sun Yat-sen University, Daya Guo , Ensheng Shi Huawei, Yuchi Ma Huawei Cloud Computing Technologies, Jiachi Chen Sun Yat-sen University, Zibin Zheng Sun Yat-sen University | ||
15:20 10mTalk | Effectiveness of symmetric metamorphic relations on validating the stability of code generation LLM Journal-First Track Chan Pak Yuen Department of Computer Science, City University of Hong Kong, Kowloon, Hong Kong, China, Jacky Keung City University of Hong Kong, Zhen Yang Shandong University | ||