Large Language Models (LLMs) have shown their superior capability in natural language processing, promoting extensive LLM-powered applications to be the new portals for people to access various content on the Internet. However, LLM-powered applications do not have sufficient security considerations on untrusted content, leading to potential threats. In this paper, we reveal content poisoning, where attackers can tailor attack content that appears benign to humans but causes LLM-powered applications to generate malicious responses. To highlight the impact of content poisoning and inspire the development of effective defenses, we systematically analyze the attack, focusing on the attack modes in various content, exploitable design features of LLM application frameworks, and the generation of attack content. We carry out a comprehensive evaluation on five LLMs, where content poisoning achieves an average attack success rate of 89.60%. Additionally, we assess content poisoning on four popular LLM-powered applications, achieving the attack on 72.00% of the content. Our experimental results also show that existing defenses are ineffective against content poisoning. Finally, we discuss potential mitigations for LLM application frameworks to counter content poisoning.
Wed 30 OctDisplayed time zone: Pacific Time (US & Canada) change
10:30 - 12:00 | AIWareResearch Papers / Journal-first Papers at Camellia Chair(s): Vladimir Filkov University of California at Davis, USA | ||
10:30 15mTalk | Imperceptible Content Poisoning in LLM-Powered Applications Research Papers Quan Zhang Tsinghua University, Chijin Zhou Tsinghua University, Gwihwan Go Tsinghua University, Binqi Zeng Central South University, Heyuan Shi Central South University, Zichen Xu The Nanchang University, Yu Jiang Tsinghua University | ||
10:45 15mTalk | What Makes a High-Quality Training Dataset for Large Language Models: A Practitioners’ Perspective Research Papers Xiao Yu Huawei, Zexian Zhang Wuhan University of Technology, Feifei Niu University of Ottawa, Xing Hu Zhejiang University, Xin Xia Huawei, John Grundy Monash University Media Attached | ||
11:00 15mTalk | Prompt Sapper: A LLM-Empowered Production Tool for Building AI Chains Journal-first Papers Yu Cheng Jiangxi Normal University, Jieshan Chen CSIRO's Data61, Qing Huang School of Computer Information Engineering, Jiangxi Normal University, Zhenchang Xing CSIRO's Data61, Xiwei (Sherry) Xu Data61, CSIRO, Qinghua Lu Data61, CSIRO | ||
11:15 15mTalk | Efficient Detection of Toxic Prompts in Large Language Models Research Papers Yi Liu Nanyang Technological University, Huijia Sun ShanghaiTech University, Ling Shi Nanyang Technological University, Gelei Deng Nanyang Technological University, Yuqi Chen ShanghaiTech University, Junzhe Yu ShanghaiTech University, Yang Liu Nanyang Technological University | ||
11:30 15mTalk | Exploring ChatGPT App Ecosystem: Distribution, Deployment and Security Research Papers Chuan Yan University of Queensland, Mark Huasong Meng National University of Singapore, Liuhuo Wan University of Queensland, Tian Yang Ooi University of Queensland, Ruomai Ren University of Queensland, Guangdong Bai University of Queensland | ||
11:45 15mTalk | DataRecipe — How to Cook the Data for CodeLLM? Research Papers Kisub Kim Singapore Management University, Singapore, Jounghoon Kim Chinese University of Hong Kong, Hong Kong, Byeongjo Park Chungbuk National University, Korea, Dongsun Kim Korea University, Chun Yong Chong Monash University Malaysia, Yuan Wang Independent Researcher, Hong Kong, Tiezhu Sun University of Luxembourg, Xunzhu Tang University of Luxembourg, Jacques Klein University of Luxembourg, Tegawendé F. Bissyandé University of Luxembourg | ||