Adaptive Convolutions for Structure-Aware Style Transfer

 

We propose Adaptive convolutions; a generic extension of AdaIN, which allows for the simultaneous transfer of both statistical and structural styles in real time.

June 19, 2021
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2021

 

Authors

Prashanth Chandran (DisneyResearch|Studios/ETH Joint PhD)

Gaspard Zoss (DisneyResearch|Studios/ETH Joint PhD)

Paulo Gotardo (DisneyResearch|Studios)

Markus Gross (DisneyResearch|Studios/ETH Zurich)

Derek Bradley (DisneyResearch|Studios)

 

Adaptive Convolutions for Structure-Aware Style Transfer

Abstract

Style transfer between images is an artistic application of CNNs, where the ‘style’ of one image is transferred onto another image without modifying its content. The current state-of-the-art in neural style transfer uses a technique called Adaptive Instance Normalization (AdaIN), which transfers the statistical properties of style features to a content image, and can transfer an infinite number of styles in real time. However, AdaIN is a global operation, and thus local geometric structures in the style image are often ignored during the transfer. We propose Adaptive convolutions; a generic extension of AdaIN, which allows for the simultaneous transfer of both statistical and structural styles in real time. Apart from style transfer, our method can also be readily extended to style-based image generation, and other tasks where AdaIN has already been adopted.

Copyright Notice