A Two-Level Planning Framework for Mixed Reality Interactive Narratives with User Engagement

We present an event-based interactive storytelling system for virtual 3D environments that aims to offer free-form user experiences while constraining the narrative to follow author intent.

The characters of our stories are represented as smart objects, each having their own state and set of capabilities that they expose to the virtual world. Our narratives are represented as a collection of branching stories, where narrative flow is controlled by author-defined states. A user model is employed to evaluate the user’s engagement with smart objects and events, based on proximity, interaction patterns and visibility to the user. A two-level online planning system is designed to find the best narrative trajectory along pre-authored stories, according to the user model, and to generate a story sequence to the best trajectory with Monte Carlo Tree Search. We present the capabilities of our interactive storytelling system on an example story and describe the adaptations required for modeling user engagement in AR and VR applications.