Digital Humans

Disney Research has been actively pushing the state-of-the-art in creating and animating high-quality digital humans for visual effects over the past decade. We have developed world leading technology to reconstruct facial performances, focusing on all aspects of the human face, including skin, eyes, teeth and hair.

 

Abstract:

Digital Humans are becoming omnipresent in entertainment. Almost every modern blockbuster movie relies on digital humans in some form, be it digital doubles that are indistinguishable from the real counterpart or fantasy creatures that are being puppeteered by a human actor. Many new forms of entertainment, such Virtual and Augmented Reality, are fully digital and hence rely entirely on digital assets, steadily increasing the need for digital humans.

To accommodate this need Disney Research has established the Digital Humans Strategic Program, with the vision to enable new ways for our guests and customers to meet and interact with our iconic characters. The program strives to facilitate the creation and animation of virtual characters that truly capture and preserve the personality of our world class talent, for use in film and experiences beyond the screen.

Medusa Performance Capture System

Much of our digital humans research has been combined into the Medusa system, which has been used in production since 2014 with a steadily increasing portfolio:

 

Other research, such as our work on eye and teeth reconstruction, is directly being applied to feature films by Disney Research staff, giving our researchers exposure to the world of production. Our work on eye reconstruction has for example been used on Star Wars: The Force Awakens and on Doctor Strange, and our work on teeth reconstruction has been used to create the teeth for Grand Moff Tarkin in Rogue One: A Star Wars Story from legacy footage of deceased actor Peter Cushing.



Publication Highlights

Appearance Capture and Modeling of Human Teeth

November 27, 2018
ACM SIGGRAPH Asia 2018

Zdravko Velinov (Disney Research/Rheinische Friedrich-Wilhelms-Universität Bonn) Marios Papas (Disney Research) Derek Bradley (Disney Research) Paulo Gotardo (Disney Research) Parsa Mirdehghan (Disney Research) Steve Marschner (Cornell University) Jan Novak (Disney Research) Thabo Beeler (Disney Research)

An Empirical Rig for Jaw Animation

July 30, 2018
ACM SIGGRAPH 2018

Gaspard Zoss (Disney Research) Derek Bradley (Disney Research) Pascal Bérard (Disney Research/ETH Joint PhD) Thabo Beeler (Disney Research)

Simulation-Ready Hair Capture

April 24, 2017
Eurographics 2017

Liwen Hu (Disney Research/Pinscreen/University of Southern California) Derek Bradley (Disney Research) Hao Li (Pinscreen/University of Southern California) Thabo Beeler (Disney Research)

Enriching Facial Blendshape Rigs with Physical Simulation

April 24, 2017
Eurographics 2017

Yeara Kozlov (Disney Research/ETH Joint PhD) Derek Bradley (Disney Research) Moritz Baecher (Disney Research) Bernhard Thomaszewski (Disney Research) Thabo Beeler (Disney Research) Markus Gross (Disney Research/ETH Zurich)

Corrective 3D Reconstruction of Lips from Monocular Video

November 11, 2016
ACM SIGGRAPH Asia 2016

Pablo Garrido (Max Planck Institute for Informatics) Michael Zollhöfer (Max Planck Institute for Informatics) Chenglei Wu (ETH Zurich) Derek Bradley (Disney Research) Patrick Pérez (Technicolor) Thabo Beeler (Disney Research) Christian Theobalt (Max Planck Institute for Informatics)

Synthetic Prior Design for Real-Time Face Tracking

October 25, 2016
3D International Conference on 3D Vision (3DV) 2016

Steven McDonagh (Disney Research) Martin Klaudiny (Disney Research) Thabo Beeler (Disney Research) Derek Bradley (Disney Research) Iain Matthews (Disney Research) Kenny Mitchell (Disney Research)

FaceDirector: Continuous Control of Facial Performance in Video

December 11, 2015
International Conference on Computer Vision (ICCV) 2015

Charles Malleson (Centre for Vision, Speech and Signal Processing, University of Surrey, UK) Jean-Charles Bazin (Disney Research) Oliver Wang (Disney Research) Derek Bradley (Disney Research) Thabo Beeler (Disney Research) Adrian Hilton (Centre for Vision, Speech and Signal Processing, University of Surrey, UK) Alexander Sorkine-Hornung (Disney Research)

Detailed Spatio-Temporal Reconstruction of Eyelids

July 27, 2015
ACM SIGGRAPH 2015

Amit Bermano (Disney Research/ETH Joint PhD) Thabo Beeler (Disney Research) Yeara Kozlov (Disney Research/ETH Joint PhD) Derek Bradley (Disney Research) Bernd Bickel (Disney Research/IST Austria) Markus Gross (Disney Research/ETH Zurich)

High-Quality Capture of Eyes

November 19, 2014
ACM SIGGRAPH Asia 2014

Pascal Berard (Disney Research/ETH Zürich) Derek Bradley (Disney Research) Maurizio Nitti (Disney Research) Thabo Beeler (Disney Research) Markus Gross (Disney Research/ETH Zurich)

Facial Performance Enhancement Using Dynamic Shape Space Analysis

March 1, 2014
ACM Transactions on Graphics (TOG) 2014

Amit Bermano (Disney Research/ETH Joint PhD) Derek Bradley (Disney Research) Thabo Beeler (Disney Research) Fabio Zund (Disney Research/ETH Joint PhD) Derek Nowrouzezahrai (Disney Research) Ilya Baran (Disney Research) Olga Sorkine-Hornung (ETH Zurich) Hanspeter Pfister (Disney Research/Harvard University) Robert W. Sumner (Disney Research) Bernd Bickel (Disney Research) Markus Gross (Disney Research/ETH Zurich)

Coupled 3D Reconstruction of Sparse Facial Hair and Skin

August 5, 2012
ACM SIGGRAPH 2012

Thabo Beeler (Disney Research/ETH Joint PhD) Bernd Bickel (Disney Research) Chino Noris (Disney Research/ETH Joint PhD) Paul Beardsley (Disney Research) Steve Marschner (Disney Research/Cornell University) Robert W. Sumner (Disney Research) Markus Gross (Disney Research/ETH Zurich)

Physical Face Cloning

August 5, 2012
SIGGRAPH 2012

Bernd Bickel (Disney Research) Peter Kaufmann (Disney Research) Melina Skouras (Disney Research/ETH Joint PhD) Bernhard Thomaszewski (Disney Research/ETH Zurich) Derek Bradley (Disney Research) Thabo Beeler (Disney Research) Phil Jackson (Walt Disney Imagineering) Steve Marschner (Cornell University) Wojciech Matusik (Disney Research) Markus Gross (Disney Research/ETH Zurich)

High-Quality Passive Facial Performance Capture using Anchor Frames

August 7, 2011
ACM SIGGRAPH 2011

Thabo Beeler (Disney Research) Fabian Hahn (Disney Research/ETH Joint PhD) Derek Bradley (Disney Research) Bernd Bickel (Disney Research) Paul Beardsley (Disney Research) Craig Gotsman (Technion – Israel Institute of Technology) Robert W. Sumner (Disney Research) Markus Gross (Disney Research/ETH Zurich)