BenGQL: An Extensible Benchmarking Framework for Automated GraphQL Testing
This program is tentative and subject to change.
GraphQL APIs provide a unified endpoint for retrieving and uploading data in a web application. Due to its efficient data-fetching strategy, which allows for the retrieval of only the required data, GraphQL is gaining popularity. Its software nature necessitates robust testing, both functional and non-functional. As automated testing tools, including load testers and fuzzers, are developed to assess GraphQL APIs, they lack a common set of case studies for rigorous evaluation and comparison.
To address this gap, we present BenGQL, a benchmarking framework containing 23 representative open-source GraphQL server applications, spanning different underlying engines and schema complexities. BenGQL provides an extensible infrastructure for running testing tools against these case studies, enabling developers and researchers to: (i) execute testing tools against the same case studies, (ii) analyse and compare results using custom analysis modules, and (iii) extend the benchmark with new case studies, tools, or metrics.
The ultimate goal of BenGQL is to foster more rigorous, reproducible research in automated GraphQL testing by providing both the case studies and the infrastructure for running experiments. As a consequence, we release the source code at https://github.com/marcellomaugeri/BenGQL, inviting other researchers to contribute. A video demonstration is also available at https://youtu.be/wZ06Xxa_Koo.
This program is tentative and subject to change.
Tue 18 NovDisplayed time zone: Seoul change
15:00 - 18:00 | |||
15:00 3hDemonstration | APIDA-Chat: Structured Synthesis of API Search Dialogues to Bootstrap Conversational Agents Tool Demonstration Track | ||
15:00 3hDemonstration | PROXiFY: A Bytecode Analysis Tool for Detecting and Classifying Proxy Contracts in Ethereum Smart Contracts Tool Demonstration Track Ilham Qasse Reykjavik University, Mohammad Hamdaqa Polytechnique Montreal, Björn Þór Jónsson Reykjavik University | ||
15:00 3hDemonstration | DeepTx: Real-Time Transaction Risk Analysis via Multi-Modal Features and LLM Reasoning Tool Demonstration Track Yixuan Liu Nanyang Technological University, Xinlei Li Nanyang Technological University, Yi Li Nanyang Technological University | ||
15:00 3hDemonstration | WIBE: Watermarks for generated Images - Benchmarking & Evaluation Tool Demonstration Track Aleksey Yakushev ISP RAS, Aleksandr Akimenkov ISP RAS, Khaled Abud MSU AI Institute, Dmitry Obydenkov ISP RAS, Irina Serzhenko MIPT, Kirill Aistov Huawei Research Center, Egor Kovalev MSU, Stanislav Fomin ISP RAS, Anastasia Antsiferova ISP RAS Research Center, MSU AI Institute, Kirill Lukianov ISP RAS Research Center, MIPT, Yury Markin ISP RAS | ||
15:00 3hDemonstration | EyeNav: Accessible Webpage Interaction and Testing using Eye-tracking and NLP Tool Demonstration Track Juan Diego Yepes-Parra Universidad de los Andes, Colombia, Camilo Escobar-Velásquez Universidad de los Andes, Colombia Link to publication Media Attached | ||
15:00 3hDemonstration | Quirx: A Mutation-Based Framework for Evaluating Prompt Robustness in LLM-based Software Tool Demonstration Track Souhaila Serbout University of Zurich, Zurich, Switzerland | ||
15:00 3hDemonstration | BenGQL: An Extensible Benchmarking Framework for Automated GraphQL Testing Tool Demonstration Track Media Attached | ||
15:00 3hDemonstration | evalSmarT: An LLM-Based Evaluation Framework for Smart Contract Comment Generation Tool Demonstration Track Fatou Ndiaye MBODJI SnT, University of Luxembourg, Mame Marieme Ciss SOUGOUFARA UCAD, Senegal, Wendkuuni Arzouma Marc Christian OUEDRAOGO SnT, University of Luxembourg, Alioune Diallo University of Luxembourg, Kui Liu Huawei, Jacques Klein University of Luxembourg, Tegawendé F. Bissyandé University of Luxembourg Pre-print | ||
15:00 3hDemonstration | LLMorph: Automated Metamorphic Testing of Large Language Models Tool Demonstration Track Steven Cho The University of Auckland, New Zealand, Stefano Ruberto JRC European Commission, Valerio Terragni University of Auckland | ||
15:00 3hDemonstration | TRUSTVIS: A Multi-Dimensional Trustworthiness Evaluation Framework for Large Language Models Tool Demonstration Track Ruoyu Sun University of Alberta, Canada, Da Song University of Alberta, Jiayang Song Macau University of Science and Technology, Yuheng Huang The University of Tokyo, Lei Ma The University of Tokyo & University of Alberta | ||
15:00 3hDemonstration | GUI-ReRank: Enhancing GUI Retrieval with Multi-Modal LLM-based Reranking Tool Demonstration Track Kristian Kolthoff Institute for Software and Systems Engineering, Clausthal University of Technology, Felix Kretzer human-centered systems Lab (h-lab), Karlsruhe Institute of Technology (KIT) , Christian Bartelt Institute for Software and Systems Engineering, TU Clausthal, Alexander Maedche human-centered systems Lab (h-lab), Karlsruhe Institute of Technology (KIT) , Simone Paolo Ponzetto Data and Web Science Group, University of Mannheim Pre-print Media Attached | ||
15:00 3hDemonstration | StackPlagger: A System for Identifying AI-Code Plagiarism on Stack Overflow Tool Demonstration Track Aman Swaraj Dept. of Computer Science & Engineering, Indian Institute of Technology, Roorkee, India, Harsh Goyal Indian Institute of Technology, Roorkee, Sumit Chadgal Indian Institute of Technology, Roorkee, Sandeep Kumar Dept. of Computer Science & Engineering, Indian Institute of Technology, Roorkee, India | ||
15:00 3hDemonstration | AgentDroid: A Multi-Agent Tool for Detecting Fraudulent Android Applications Tool Demonstration Track Ruwei Pan Chongqing University, Hongyu Zhang Chongqing University, Zhonghao Jiang , Ran Hou Chongqing University | ||