Medusa Facial Capture System

The Medusa Facial Capture system, developed by DisneyResearch|Studios, consists of a mobile rig of cameras and lights coupled with proprietary software that can reconstruct actor’s faces in full motion, without using traditional motion-capture dots.
The technology comes as the result of many years worth of research and scientific advances in capturing and modeling of human faces.
Medusa delivers high-resolution 3D faces, with the ability to track individual pores and wrinkles over time, and can recover per-frame dynamic appearance (blood flow and shininess of the skin), providing a very realistic virtual face that is ideal for creating digital doubles for visual effects and computer games.
Medusa is useful in two primary scenarios: 1 – building an expression shape library, and 2 – reconstructing performance dialog. The following videos demonstrate examples of these two scenarios.
People
Medusa is the result of the hard work of many people:
Thabo Beeler
Derek Bradley
Paulo Gotardo
Jérémy Riviere
Pascal Bérard
Gaspard Zoss
Andreea Radoescu
Chenglei Wu
Prashanth Chandran
Sebastian Winberg
Loïc Ciccone
Irene Baeza
Max Grosse
Roman Cattaneo
Jan Wezel
Ronnie Gaensli
Bernd Bickel
Robert W. Sumner
Paul Beardsley
Abhijeet Ghosh
Markus Gross
Research Lead
Research Lead
Research (Appearance)
Research (Appearance)
Research (Eyes)
Research (Skull/Jaw)
Research (Ears)
Research (Teeth)
Research (Landmarks)
Research (Facial Hair)
Software Engineering
Software Engineering
Software Engineering
Software Engineering
Hardware Engineering
Hardware Engineering
Research Guidance
Research Guidance
Research Guidance
Research Guidance (Appearance)
Lab Director
And a special thanks to everyone at ILM for adopting Medusa and using it to make stunning digital humans!
Technology
The Medusa Performance Capture System is based on several recent technological advances created at Disney Research. We are continually improving the technology to provide the most state-of-the-art capture system in visual effects. For more details on the technology, please refer to the following research projects.
Facial Hair Tracking for High Fidelity Performance Capture
July 24, 2022
ACM SIGGRAPH 2022
Sebastian Winberg (DisneyResearch|Studios/ETH Zurich) Gaspard Zoss (DisneyResearch|Studios/ETH Joint PhD) Prashanth Chandran (DisneyResearch|Studios/ETH Joint PhD) Paulo Gotardo (DisneyResearch|Studios) Derek Bradley (DisneyResearch|Studios)
Single-Shot High-Quality Facial Geometry and Skin Appearance Capture
August 14, 2020
ACM Siggraph 2020
Jeremy Riviere (DisneyResearch|Studios) Paulo Gotardo (DisneyResearch|Studios) Derek Bradley (DisneyResearch|Studios) Abhijeet Ghosh (Imperial College London) Thabo Beeler (DisneyResearch|Studios)
Accurate Markerless Jaw Tracking for Facial Performance Capture
July 12, 2019
ACM Siggraph 2019
Gaspard Zoss (Disney Research/ETH Joint PhD) Thabo Beeler (Disney Research) Markus Gross (Disney Research/ETH Zurich) Derek Bradley (Disney Research)
Practical Dynamic Facial Appearance Modeling and Acquisition
November 27, 2018
ACM SIGGRAPH Asia 2018
Paulo Gotardo (Disney Research) Jeremy Riviere (Disney Research) Derek Bradley (Disney Research) Abhijeet Ghosh (Imperial College London) Thabo Beeler (Disney Research)
Model-Based Teeth Reconstruction
November 11, 2016
ACM SIGGRAPH Asia 2016
Chenglei Wu (ETH Zürich) Derek Bradley (Disney Research) Pablo Garrido (Max Planck Institute for Informatics) Michael Zollhöfer (Max Planck Institute for Informatics) Christian Theobolt (Max Planck Institute for Informatics)
Markus Gross (Disney Research/ETH Zürich)
Thabo Beeler (Disney Research)
Lightweight Eye Capture Using a Parametric Model
July 11, 2016
ACM SIGGRAPH 2016
Pascal Berard (Disney Research/ETH Joint PhD) Derek Bradley (Disney Research) Markus Gross (Disney Research/ETH Zurich) Thabo Beeler (Disney Research)
Rigid Stabilization of Facial Expressions
July 27, 2014
ACM SIGGRAPH 2014
Thabo Beeler (Disney Research) Derek Bradley (Disney Research)
High-Quality Passive Facial Performance Capture using Anchor Frames
August 7, 2011
ACM SIGGRAPH 2011
Thabo Beeler (Disney Research) Fabian Hahn (Disney Research/ETH Joint PhD) Derek Bradley (Disney Research) Bernd Bickel (Disney Research) Paul Beardsley (Disney Research) Craig Gotsman (Technion – Israel Institute of Technology) Robert W. Sumner (Disney Research) Markus Gross (Disney Research/ETH Zurich)
High-Quality Single-Shot Capture of Facial Geometry
July 26, 2010
ACM SIGGRAPH 2010
Thabo Beeler (Disney Research) Bernd Bickel (Disney Research) Paul Beardsley (Disney Research) Robert W. Sumner (Disney Research) Markus Gross (Disney Research/ETH Zurich)
For more information on the Medusa technology, please contact medusa@disneyresearch.com
Business Inquiries
For business inquiries, please email contact-facialcapture@ilm.com, or visit https://www.ilm.com/facial-capture/