ICSE 2026
Sun 12 - Sat 18 April 2026 Rio de Janeiro, Brazil

Extended Reality (XR) technologies have emerged as transformative tools across diverse domains, resulting in a rapidly growing market expected to exceed $200 billion by 2025. Despite this growth, the increasing complexity of XR software systems poses significant challenges for reliability and usability, particularly in automated testing and interaction processes. Unlike traditional software, XR applications operate in three-dimensional space with multi-modal inputs, making it difficult to translate natural language instructions into executable system commands. This paper addresses this fundamental challenge by introducing RealityScript, a novel domain-specific language designed specifically for XR applications that captures the unique spatial, temporal, and interactive characteristics of extended reality environments. We also present RealityCraft, an innovative approach that leverages large language models to analyze complex natural language instructions and translate them into executable RealityScript code through a process of semantic decomposition, rule-based parsing, and verification. Our evaluation demonstrates that this approach significantly outperforms existing methods in translating natural language instructions for XR applications, providing a robust foundation for automated testing and interaction in spatial computing environments. This work represents an important step toward enhancing the reliability and usability of XR software through improved automation of testing processes.