StereoBrush: Interactive 2D to 3D Conversion Using Discontinuous Warps

 

We introduce a novel workflow for stereoscopic 2D to 3D conversion in which the user “paints” depth onto a 2D image via sparse scribbles, instantaneously receiving intuitive 3D feedback.

August 5, 2011
Symposium on Sketch-Based Interfaces and Modeling (SBIM) 2011

 

Authors

Oliver Wang (Disney Research)

Manuel Lang (Disney Research/ETH Joint PhD)

Matthias Frei (Disney Research)

Alexander Sorkine-Hornung (Disney Research)

Aljoscha Smolic (Disney Research)

Markus Gross (Disney Research/ETH Zurich)

StereoBrush: Interactive 2D to 3D Conversion Using Discontinuous Warps

Abstract

We introduce a novel workflow for stereoscopic 2D to 3D conversion in which the user “paints” depth onto a 2D image via sparse scribbles, instantaneously receiving intuitive 3D feedback. This workflow is enabled by the introduction of a discontinuous warping technique that creates stereoscopic pairs from sparse, possibly erroneous user input. Our method assumes a piecewise continuous depth representation, preserving visual continuity in most areas, while creating sharp depth discontinuities at important object boundaries. As opposed to prior work that relies strictly on a per pixel depth map, our scribbles are processed as soft constraints in a global solve and operate entirely on image domain disparity, allowing for relaxed input requirements. This formulation also allows us to simultaneously compute a disparity-and-content-aware stretching of background areas to automatically fill disoccluded regions with valid stereo information. We tightly integrate all steps of stereo content conversion into a single optimization framework, which can then be solved on a GPU at interactive rates. The instant feedback received while painting depth allows even untrained users to quickly create compelling 3D scenes from single-view footage.