2019/04/09 by Yulun Zhang, Chen Fang, Fang Chen +12 · 9 citations
Computer Science · Mathematics · #Advanced Image Processing Techniques #Artificial intelligence #Artificial neural network #Computer Vision and Pattern Recognition (cs.CV) #Computer science #FOS: Computer and information sciences #Flexibility (engineering) #Generative Adversarial Networks and Image Synthesis #Image Enhancement Techniques #Mathematics #Pattern recognition (psychology) #Robustness (evolution) #Style (visual arts) #Stylized fact #cs.CV
paper · pdf · doi:10.48550/arxiv.1904.04443
published in arXiv (Cornell University) 2019, 5942-5950 (Cornell University) · Accepted to ICCV 2019. Typos in Eqs. (11) and (12) have been fixed in arXiv V2 and this version (V6). Code: https://github.com/yulunzhang/MST
openalex publication_date 2019/04/09 · arxiv created 2020/01/07 · arxiv updated 2020/01/08 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/08
An assumption widely used in recent neural style transfer methods is that image styles can be described by global statics of deep features like Gram or covariance matrices. Alternative approaches have represented styles by decomposing them into local pixel or neural patches. Despite the recent progress, most existing methods treat the semantic patterns of style image uniformly, resulting unpleasing results on complex styles. In this paper, we introduce a more flexible and general universal style transfer technique: multimodal style transfer (MST). MST explicitly considers the matching of semantic patterns in content and style images. Specifically, the style image features are clustered into sub-style components, which are matched with local content features under a graph cut formulation. A reconstruction network is trained to transfer each sub-style and render the final stylized result. We also generalize MST to improve some existing methods. Extensive experiments demonstrate the superior effectiveness, robustness, and flexibility of MST.