SwapShop

Face swapping is the visual effect of transferring a source actor’s appearance to a target actor’s performance. It is often deployed when stunts are too dangerous for actors to perform, when an actor’s appearance needs to be de-aged, or when an actor indelibly associated with a character is deceased. Traditionally, this effect has been achieved with painstaking on-set preparation with stand-in performers, 3D CGI modeling of the human head, and months of dedicated work by effects artists. With the advent of deep neural networks (DNNs), however, face swapping can be an entirely data-driven process, trained in a fraction of the time required for the traditional pipeline.

With SwapShop, DisneyResearch|Studios has brought DNN-based face swapping into the high-resolution, production-quality domain. With a variety of innovations in model architecture and training methodology, SwapShop is capable of providing an output of extremely high quality while giving effects artists an unprecedented level of control over the final shot.

SwapShop technology has been used in the following productions

Technology

SwapShop is based on several recent technological advances created at Disney Research. We are continually improving the technology to provide the most state-of-the-art capture system in visual effects. For more details on the technology, please refer to the following research projects.

Learning Dynamic 3D Geometry and Texture for Video Face Swapping

October 05, 2022
Pacific Graphics (2022)

Christopher Otto (DisneyResearch|Studios/ETH Joint PhD) Jacek Naruniec (DisneyResearch|Studios) Leonhard Helminger (DisneyResearch|Studios/ETH Joint PhD) Thomas Etterlin (DisneyResearch|Studios/ETH Joint M.Sc.) Graziana Mignone (DisneyResearch|Studios) Prashanth Chandran (DisneyResearch|Studios/ETH Joint PhD) Gaspard Zoss (DisneyResearch|Studios) Christopher Schroers (DisneyResearch|Studios) Markus Gross (DisneyResearch|Studios/ETH) Paulo Gotardo (DisneyResearch|Studios) Derek Bradley (DisneyResearch|Studios) Romann Weber (DisneyResearch|Studios)

High-Resolution Neural Face Swapping for Visual Effects

June 29, 2020
Eurographics Symposium on Rendering (2020)

Jacek Naruniec (DisneyResearch|Studios) Leonhard Helminger (DisneyResearch|Studios/ETH Joint PhD) Christopher Schroers (DisneyResearch|Studios) Romann M. Weber (DisneyResearch|Studios)

For more information on the SwapShop technology, please contact swapshop@disneyresearch.com