The integration of Large Language Models (LLMs) into software engineering has led to significant advancements, particularly in supporting development with well-established programming languages (e.g., Copilot for Java). LLMs face challenges when applied to domain-specific languages (DSLs), since often their training dataset does not include extensive sets of examples for these languages. This is also the case for languages that are either newly developed or under refinement.

In this paper, we introduce langium-llm, an extension of the Langium language workbench designed to facilitate interactions with LLMs during development with DSLs. Given a Langium-based DSL, we propose two approaches for the communication with the LLM: one based on the DSL concrete syntax and the other in its abstract syntax. Depending on the solution chosen, our extension generates a conversational interface that is aware of the grammar of the DSL concrete syntax or of the JSON Schema of its abstract syntax, enabling software engineers to collaborate with the LLM while developing in the DSL, using natural language instructions.

We first present an overview of the architecture underlying our proposed solutions. We then evaluate their effectiveness by assessing the LLM’s ability to interpret and execute development instructions via the graphical chat interface. We especially focus on the responsiveness and accuracy of our LLM-based approach in editing programs written in different DSLs.