Towards Emergent Play in Mixed Reality


This paper presents a system to experience emergent play within a mixed reality environment.

November 27, 2017                                            Workshop on Artificial Intelligence Meets Virtual and Augmented Worlds (AIVRAR) 2017



Patrick Misteli (Disney Research/ETH Joint M.Sc.)

Steven Poulakos (Disney Research)

Mubbasir Kapadia (Rutgers University)

Robert W. Sumner (Disney Research/ETH Zurich)

Towards Emergent Play in Mixed Reality


Real and virtual objects share a unified representation to allow joint interactions. These objects may optionally contain an internal mental model to act autonomously based on their beliefs about the world. The experience utilizes intuitive interaction patterns using voice, hand gestures and real object manipulation. We author experience by specifying dependency graphs and behavior models, which are extended to support player interactions. Feedback is provided to ensure the system and player share a common play experience, including awareness of obstacles and potential solutions. An author can mix features from game, story and agent-based experiences. We demonstrate our system through an example adventure game using the Microsoft HoloLens.

Copyright Notice