Exploring ChatGPT App Ecosystem: Distribution, Deployment and Security
As the flagship large language lodel (LLM) product of OpenAI, ChatGPT has gained global attention for its remarkable ability to handle complex natural language understanding and generation tasks. Inspired by the success of the mobile app ecosystems, OpenAI enables third-party developers to create ChatGPT plugins to further expand ChatGPT’s capabilities. These plugins are distributed through the OpenAI’s plugin store and are easily accessible to users. With ChatGPT as the powerful backbone, this app ecosystem has illustrated great business potential by offering users personalized services in a conversational manner. Nonetheless, this ecosystem is still in its nascent stage and undergoing dynamic evolution. Many crucial aspects regarding app development, deployment, and security have yet to be thoroughly studied in the research community, potentially hindering a wider adoption by both developers and users.
In this work, we conduct the first comprehensive study of the ChatGPT app ecosystem, aiming to unveil its landscape to our research community. Our study focuses on the distribution and deployment models in the integration of LLMs and third-party apps, and assesses their security and privacy implications. We investigate the runtime execution mechanism of ChatGPT apps and accordingly propose a three-layer security assessment model from the perspectives of users, app developers, and store operators. Our evaluation of all 1,038 plugins available in the store reveals their uneven distribution of functionality. Our security assessment also reveals a concerning status quo of security and privacy in the ChatGPT app ecosystem. We find that the authentication and user data protection for third-party app APIs integrated within LLMs contain severe flaws. For example, 173 plugins have broken access control vulnerabilities, 368 plugins are subject to leaking manifest files, and 271 plugins provide inaccessible legal document links. Our study for the first time highlights the immaturity of the ChatGPT app ecosystem. Our findings should especially raise an alert to OpenAI and third-party developers to collaboratively maintain the security and privacy compliance of this emerging ecosystem.
Wed 30 OctDisplayed time zone: Pacific Time (US & Canada) change
10:30 - 12:00 | AIWareResearch Papers / Journal-first Papers at Camellia Chair(s): Vladimir Filkov University of California at Davis, USA | ||
10:30 15mTalk | Imperceptible Content Poisoning in LLM-Powered Applications Research Papers Quan Zhang Tsinghua University, Chijin Zhou Tsinghua University, Gwihwan Go Tsinghua University, Binqi Zeng Central South University, Heyuan Shi Central South University, Zichen Xu The Nanchang University, Yu Jiang Tsinghua University | ||
10:45 15mTalk | What Makes a High-Quality Training Dataset for Large Language Models: A Practitioners’ Perspective Research Papers Xiao Yu Huawei, Zexian Zhang Wuhan University of Technology, Feifei Niu University of Ottawa, Xing Hu Zhejiang University, Xin Xia Huawei, John Grundy Monash University Media Attached | ||
11:00 15mTalk | Prompt Sapper: A LLM-Empowered Production Tool for Building AI Chains Journal-first Papers Yu Cheng Jiangxi Normal University, Jieshan Chen CSIRO's Data61, Qing Huang School of Computer Information Engineering, Jiangxi Normal University, Zhenchang Xing CSIRO's Data61, Xiwei (Sherry) Xu Data61, CSIRO, Qinghua Lu Data61, CSIRO | ||
11:15 15mTalk | Efficient Detection of Toxic Prompts in Large Language Models Research Papers Yi Liu Nanyang Technological University, Huijia Sun ShanghaiTech University, Ling Shi Nanyang Technological University, Gelei Deng Nanyang Technological University, Yuqi Chen ShanghaiTech University, Junzhe Yu ShanghaiTech University, Yang Liu Nanyang Technological University | ||
11:30 15mTalk | Exploring ChatGPT App Ecosystem: Distribution, Deployment and Security Research Papers Chuan Yan University of Queensland, Mark Huasong Meng National University of Singapore, Liuhuo Wan University of Queensland, Tian Yang Ooi University of Queensland, Ruomai Ren University of Queensland, Guangdong Bai University of Queensland | ||
11:45 15mTalk | DataRecipe — How to Cook the Data for CodeLLM? Research Papers Kisub Kim Singapore Management University, Singapore, Jounghoon Kim Chinese University of Hong Kong, Hong Kong, Byeongjo Park Chungbuk National University, Korea, Dongsun Kim Korea University, Chun Yong Chong Monash University Malaysia, Yuan Wang Independent Researcher, Hong Kong, Tiezhu Sun University of Luxembourg, Xunzhu Tang University of Luxembourg, Jacques Klein University of Luxembourg, Tegawendé F. Bissyandé University of Luxembourg | ||