Anyma Performance Capture System

The Anyma Performance Capture system, developed by DisneyResearch|Studios in Zurich, is a flexible solution for unconstrained, markerless facial performance capture. Anyma allows the actor freedom to move around, and still produces dense, consistent facial reconstruction at the quality of Medusa. The underlying actor model is built from a small number of Medusa reconstructions.

Anyma can be operated in several scenarios, ranging from a studio setup with an ADR-style booth, to on-set reconstruction with helmet-mounted cameras (HMCs). The flexibility of Anyma allows to operate from as little as one single camera, to many cameras placed around the scene.

Anyma delivers high quality, dense, facial geometry accurately tracked over time, including per-frame eye gazes.  Estimated per-frame skull positions are also provided automatically, removing the need to determine the head pose or HMC stabilization manually, and per-frame jaw positions are estimated from the deforming geometry.

The video on the right shows a production test of Anyma in action. Here we see the ADR-booth scenario, where 3 film cameras observe the actor from a distance. The video is illustrated in slow motion.

The top row shows the video plates, with the reconstructed face overlaid in blue. The bottom right shows a virtual texture mapped to the actor’s face, demonstrating the accuracy of the tracking. Since the skull is also tracked automatically, we can remove the rigid head motion and show only the expression deformation in the stabilized view in bottom left.

The video on the left demonstrates automatic eye gaze tracking in Anyma.  The top half shows the accuracy and stability of the gaze recovery over time, by overlaying the outer boundary of the captured iris (known as the limbus) on the input video, in a close-up view of the eyes.

The bottom half shows the same video, but this time with the complete 3D eye geometry tracked to match the actor’s eye gaze movements.  

Anyma has been used in the following productions

People

Anyma is the result of the hard work of many people:

Thabo Beeler
Derek Bradley
Chenglei Wu
Gaspard Zoss
Prashanth Chandran
Marco Fratarcangeli
Aurel Gruber
Pascal Bérard
Yury Gitman
Roman Cattaneo
Irene Baeza
Loïc Ciccone
Markus Gross

Research Lead
Research Lead
Research (Anatomical Face Modeling)
Research (Jaw Tracking)
Research (Landmarks)
Research (GPU Optimization)
Research (GPU Optimization)
Research (Eyes)
Software Engineering
Software Engineering
Software Engineering
Software Engineering
Lab Director

Technology

The Anyma Performance Capture System is based on the following main research projects, and is continually improved through ongoing research and technical innovation.

For more information on the Anyma technology, please contact anyma@disneyresearch.com

Business Inquiries

For business inquiries, please email contact-facialcapture@ilm.com, or visit https://www.ilm.com/facial-capture/

Research at Disney

 

 

 

 

 

Legal
MORE