One Model to Reconstruct Them All: A Novel Way to Use the Stochastic Noise in StyleGAN

October 21, 2020 · Entered Twilight · 🏛 British Machine Vision Conference

"Last commit was 5.0 years ago (≥5 year threshold)"

Evidence collected by the PWNC Scanner

Repo contents: .dockerignore, .gitignore, .gitmodules, Dockerfile, LICENSE, README.md, analysis, analyze_latent_code.py, configs, data, embeddings, evaluate_all_denoising_checkpoints.py, evaluate_checkpoints.py, evaluate_denoising.py, evaluation, extensions, file_based_simple_style_transfer.py, interpolate_between_embeddings.py, latent_projecting, losses, networks, project.py, pytest.ini, reconstruct_image.py, requirements.txt, tests, train_code_finder.py, training_tools, updater, utils

Authors Christian Bartz, Joseph Bethge, Haojin Yang, Christoph Meinel arXiv ID 2010.11113 Category cs.CV: Computer Vision Cross-listed cs.LG, eess.IV Citations 16 Venue British Machine Vision Conference Repository https://github.com/Bartzi/one-model-to-reconstruct-them-all ⭐ 73 Last Checked 1 month ago

Abstract

Generative Adversarial Networks (GANs) have achieved state-of-the-art performance for several image generation and manipulation tasks. Different works have improved the limited understanding of the latent space of GANs by embedding images into specific GAN architectures to reconstruct the original images. We present a novel StyleGAN-based autoencoder architecture, which can reconstruct images with very high quality across several data domains. We demonstrate a previously unknown grade of generalizablility by training the encoder and decoder independently and on different datasets. Furthermore, we provide new insights about the significance and capabilities of noise inputs of the well-known StyleGAN architecture. Our proposed architecture can handle up to 40 images per second on a single GPU, which is approximately 28x faster than previous approaches. Finally, our model also shows promising results, when compared to the state-of-the-art on the image denoising task, although it was not explicitly designed for this task.