MioHint: LLM-Assisted Request Mutation for Whitebox REST API Testing
This program is tentative and subject to change.
Cloud applications heavily rely on APIs to communicate with each other and exchange data. To ensure the reliability of cloud applications, cloud providers widely adopt API testing techniques. Unfortunately, existing API testing approaches are insufficient to reach strict conditions, a problem known as fitness plateaus, due to the lack of a gradient provided by coverage metrics. To address this issue, we propose MioHint, a novel white-box API testing approach that leverages the code comprehension capabilities of LLM to boost API testing. The key challenge of LLM-based API testing lies in system-level testing, which emphasizes the dependencies between requests and targets across functions and files, thereby making the entire codebase the object of analysis. However, feeding the entire codebase to an LLM is impractical due to its limited context length and short memory. MioHint addresses this challenge by synergizing static analysis with LLMs. We retrieve relevant code with data-dependency analysis at the statement level, including def-use analysis for variables used in the target and function expansion for subfunctions called by the target.
To evaluate the effectiveness of our method, we conducted experiments across 16 real-world REST API services. The findings reveal that MioHint achieves an average increase of 4.95% in line coverage compared to the baseline, EvoMaster, alongside a remarkable factor of 67x improvement in mutation accuracy. Furthermore, our method successfully covers over 57% of hard-to-cover targets, while in the baseline, the coverage is less than 10%.
This program is tentative and subject to change.
Thu 16 AprDisplayed time zone: Brasilia, Distrito Federal, Brazil change
14:00 - 15:30 | Testing and Analysis 12Research Track at Oceania II Chair(s): Sam Malek University of California at Irvine | ||
14:00 15mTalk | Generator Solving for Symbolic Execution Research Track | ||
14:15 15mTalk | How Good are Input Grammar Miners? An Empirical Study Research Track Leon Bettscheider CISPA Helmholtz Center for Information Security, Andreas Zeller CISPA Helmholtz Center for Information Security | ||
14:30 15mTalk | LSPRAG: LSP-Guided RAG for Language-Agnostic Real-Time Unit Test Generation Research Track Gwihwan Go Tsinghua University, Quan Zhang East China Normal University, Chijin Zhou East China Normal University, Zhao Wei Tencent, Yu Jiang Tsinghua University | ||
14:45 15mTalk | Breaking Single-Tester Limits: Multi-Agent LLMs for Multi-User Feature Testing Research Track Sidong Feng Monash University, Changhao Du Jilin University, huaxiao liu Jilin University, Qingnan Wang Jilin University, Zhengwei Lv ByteDance, Mengfei Wang ByteDance, Chunyang Chen TU Munich | ||
15:00 15mTalk | Testing Deep Learning Libraries via Neurosymbolic Constraint Learning Research Track M M Abid Naziri North Carolina State University, Shinhae Kim Cornell University, Feiran Qin North Carolina State University, Saikat Dutta Cornell University, Marcelo d'Amorim North Carolina State University | ||
15:15 15mTalk | MioHint: LLM-Assisted Request Mutation for Whitebox REST API Testing Research Track Jia Li The Chinese University of Hong Kong, Jiacheng Shen Duke Kunshan University, Yuxin Su Sun Yat-sen University, Michael Lyu The Chinese University of Hong Kong | ||