Dynamic Graph Rewriting for User State-Based Dialogue Adaption in Real-Time: An Application in Personalized Interview Training
Interactive dialogue-based virtual systems have applications ranging from e-learning to social skills training. Personalized learning is most effective as individuals have different learning styles. Many e-learning systems rely on finite state machines for dialogue management which have less flexibility to adapt dynamically to user behaviors and open discourse, limiting their effectiveness. This paper proposes a dynamic graph rewriting framework to adapt dialogue flow to user states in real-time. We demonstrate its application using job interviews as a use case, a key step towards employment where personalized simulation training can help candidates better prepare for these scenarios. Our framework leverages real-time multimodal data—speech, eye gaze, and physiology (e.g., heart rate)—to infer user states. We identified three key states in interviews: engagement, stress, and being stuck. Dialogue scripts are modeled as attributed, directed graphs, incorporating constraints and properties to ensure consistency. Each interview question is represented as a node, while edges represent transitions influenced by user states. The interview structure, serving as the host graph, was designed in collaboration with experts. Using graph rewriting, the framework dynamically modifies dialogue flows to enable contextually appropriate adaptations such as—redirecting focus when the user goes off-topic, providing hints when stuck, or rephrasing questions for clarity—actions typically performed by coaches during mock interviews. Verification of the grammar is demonstrated using two interview scripts developed with domain experts, showcasing its feasibility in virtual job interview scenarios with simulated user states. This approach offers a personalized simulation that enhances realism and supports learning.
Thu 12 JunDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
13:30 - 15:00 | ICGT Session 5: Applications for ModelingICGT Research Papers at M 201 Session Chair: Berthold Hoffmann | ||
13:30 30mTalk | Dynamic Graph Rewriting for User State-Based Dialogue Adaption in Real-Time: An Application in Personalized Interview Training ICGT Research Papers Deeksha Adiani Vanderbilt University, Timothy J. Vogus Vanderbilt University, Nilanjan Sarkar Vanderbilt University, Medha Sarkar Middle Tennessee State University | ||
14:00 30mTalk | Graph-transformational Threat Modeling ICGT Research Papers Lars Friederichs German Aerospace Center, Institute for the Protection of Maritime Infrastructures, Aaron Lye German Aerospace Center, Institute for the Protection of Maritime Infrastructures | ||