Generating Software Tests for Mobile Applications Using Fine-Tuned Large Language Models
Software testing for mobile applications is a time-consuming and costly affair due to a high degree of manual testing. Consequently, researchers and corporate enterprises try to find solutions to automate the generation of test cases for mobile applications. Therefore, this work introduces fine-tuned Transformer models, called TestGen-Dart, for the downstream task of generating test cases for mobile applications in Dart. TestGen-Dart_v0.2 enhanced the generation of syntactically correct unit tests by 15.38% and functionally correct unit tests by 16.67%, compared to the underlying base model, Code Llama 13B. This evidenced that supervised fine-tuning (SFT) increases the capability of transformer-based LLMs in a specific downstream task, in this instance, generating test cases for mobile applications. TestGen- Dart_v0.2 also outperformed the state-of-the-art models LLaMA 2 13B and Mistral 7B in that task. Furthermore, the size of 13B parameters of the TestGen-Dart models enables it to run locally on standard consumer hardware, making it a cost-efficient and privacy-preserving testing assistant for software developers by avoiding an external server connection to run the model.
Mon 15 AprDisplayed time zone: Lisbon change
14:00 - 15:30 | Session 2: Test GenerationAST 2024 at Amália Rodrigues Chair(s): Sarmad Bashir RISE Research Institutes of Sweden | ||
14:00 20mFull-paper | Using GitHub Copilot for Test Generation in Python: An Empirical Study AST 2024 Khalid El Haji Delft University of Technology, Carolin Brandt Delft University of Technology, Andy Zaidman Delft University of Technology DOI Pre-print | ||
14:20 20mFull-paper | Grammar-Based Action Selection Rules for Scriptless Testing AST 2024 Lianne V. Hufkens Open Universiteit, Fernando Pastor Ricós Universitat Politècnica de València, Beatriz Marín Universitat Politècnica de València, Tanja E. J. Vos Universitat Politècnica de València and Open Universiteit | ||
14:40 20mFull-paper | Fences: Systematic Sample Generation for JSON Schemas using Boolean Algebra and Flow Graphs AST 2024 Björn Otto Institute for Automation and Communication, OVGU Magdeburg, Tobias Kleinert Chair of Information and Automation Systems for Process and Material Technology, RWTH Aachen | ||
15:00 10mPoster | Generating Software Tests for Mobile Applications Using Fine-Tuned Large Language Models AST 2024 Jacob Hoffmann Institute AIFB, Karlsruhe Institue of Technology (KIT), Demian Frister Institute of Applied Informatics and Formal Description Methods (AIFB) Karlsruhe Institue of Technology (KIT) DOI |