NavAI: A Generalizable LLM Framework for Navigation Tasks in Virtual Reality Environments
This program is tentative and subject to change.
Navigation is one of the fundamental tasks for automated exploration in Virtual Reality (VR). Existing technologies primarily focus on path optimization in 360-degree image datasets and 3D simulators, which cannot be directly applied to immersive VR environments. To address this gap, we present NavAI, a generalizable large language model (LLM)-based navigation framework that supports both basic actions and complex goal-directed tasks across diverse VR applications. We evaluate NavAI in three distinct VR environments through goal-oriented and exploratory tasks. Results show that it achieves high accuracy, with an 89% success rate in goal-oriented tasks. Our analysis also highlights current limitations of relying entirely on LLMs, particularly in scenarios that require dynamic goal assessment. Finally, we discuss the limitations observed during the experiments and offer insights for future research directions.
This program is tentative and subject to change.
Thu 20 NovDisplayed time zone: Seoul change
16:00 - 18:00 | VARSE - Second SessionVARSE at Grand Hall 6 Chair(s): Shuqing Li The Chinese University of Hong Kong | ||
16:00 24mResearch paper | A Test Automation Framework for User Interaction in Extended Reality Applications VARSE Pre-print | ||
16:24 24mResearch paper | ARTRIP: Automatic AR Testing with Randomized Interaction Patterns VARSE Maria Rivera University of Texas at San Antonio, Lisette Isais University of Texas at San Antonio, Xiaoyin Wang University of Texas at San Antonio | ||
16:48 24mResearch paper | NavAI: A Generalizable LLM Framework for Navigation Tasks in Virtual Reality Environments VARSE | ||
17:12 24mResearch paper | A Conformance Checking System for Interaction Testing in Virtual Reality VARSE Vijay Aravynthan Software Engineering Research Center International Institute of Information Technology, Raghu Reddy IIIT Hyderabad | ||
17:36 24mResearch paper | Toward Static Analysis of Immersive Attacks VARSE Kadiray Karakaya Heinz Nixdorf Institute at Paderborn University, Jonas Klauke Heinz Nixdorf Institut, Paderborn University, Enes Yigitbas Paderborn University | ||