Deep Video Processing

Disney Research is actively pushing the limits of deep learning based technologies for video processing with the goal to improve or enable new post production processes as well as to advance the handling and distribution of image and video data.

Abstract:

At the Disney Studios, we are confronted with extreme amounts of visual data. Often, a single movie alone exceeds one petabyte of data which has to be generated, processed, analyzed, distributed, and archived. In our research, we explore opportunities to leverage large amounts of visual data using deep learning to tackle the aforementioned tasks. For example, we have developed approaches for rate conversion, denoising, and image scaling which are commonly required video processing tasks in a post production pipeline. We are also focussing on learning optimal video compression based on data with the goal to support transfer and distribution of video content. The field of deep video processing offers many further exciting opportunities, such as reducing the complexity of video editing tasks by leveraging a more principled video understanding and giving rise to deep learning based visual effects.

Tech Transfer:

We have transferred technology for rate conversion that relies on optical flow based frame interpolation to all of Disney’s Studios in form of a Nuke plug in which has been frequently used for retiming tasks. Our technology for denoising has for example been applied to denoise footage for Pirates of the Caribbean: Dead Men Tell No Tales.

Publication Highlights