Spatially Controllable Image Synthesis with Internal Representation Collaging

November 26, 2018 · Entered Twilight · + Add venue

"Last commit was 6.0 years ago (≥5 year threshold)"

Evidence collected by the PWNC Scanner

Repo contents: .gitmodules, LICENSE, NeuralCollage_demo.ipynb, README.md, README_advanced.md, __init__.py, configs, datasets, demo_feature_blending.py, demo_spatial_translation.py, dis_models, enc_models, evaluation.py, evaluations, examples, gen_models, images, optimization.py, poissonblending, requirements.txt, source, static, templates, train_GAN.py, train_auxab.py, train_enc.py, updater.py, updater_auxab.py, updater_enc.py

Authors Ryohei Suzuki, Masanori Koyama, Takeru Miyato, Taizan Yonetsuji, Huachun Zhu arXiv ID 1811.10153 Category cs.CV: Computer Vision Cross-listed cs.LG Citations 43 Repository https://github.com/pfnet-research/neural-collage ⭐ 562 Last Checked 1 month ago

Abstract

We present a novel CNN-based image editing strategy that allows the user to change the semantic information of an image over an arbitrary region by manipulating the feature-space representation of the image in a trained GAN model. We will present two variants of our strategy: (1) spatial conditional batch normalization (sCBN), a type of conditional batch normalization with user-specifiable spatial weight maps, and (2) feature-blending, a method of directly modifying the intermediate features. Our methods can be used to edit both artificial image and real image, and they both can be used together with any GAN with conditional normalization layers. We will demonstrate the power of our method through experiments on various types of GANs trained on different datasets. Code will be available at https://github.com/pfnet-research/neural-collage.