DemoFusion: Democratising High-Resolution Image Generation With No $$$

November 24, 2023 · Entered Twilight · 🏛 Computer Vision and Pattern Recognition

"No code URL or promise found in abstract"
"Derived repo from GitHub Pages (backfill)"

Evidence collected by the PWNC Scanner

Repo contents: README.md, demo.ipynb, illustration.jpg, pipeline_demofusion_sdxl.py, progressive_process.jpg

Authors Ruoyi Du, Dongliang Chang, Timothy Hospedales, Yi-Zhe Song, Zhanyu Ma arXiv ID 2311.16973 Category cs.CV: Computer Vision Cross-listed cs.AI, cs.LG Citations 90 Venue Computer Vision and Pattern Recognition Repository https://github.com/ruoyidu/demofusion ⭐ 3 Last Checked 9 days ago

Abstract

High-resolution image generation with Generative Artificial Intelligence (GenAI) has immense potential but, due to the enormous capital investment required for training, it is increasingly centralised to a few large corporations, and hidden behind paywalls. This paper aims to democratise high-resolution GenAI by advancing the frontier of high-resolution generation while remaining accessible to a broad audience. We demonstrate that existing Latent Diffusion Models (LDMs) possess untapped potential for higher-resolution image generation. Our novel DemoFusion framework seamlessly extends open-source GenAI models, employing Progressive Upscaling, Skip Residual, and Dilated Sampling mechanisms to achieve higher-resolution image generation. The progressive nature of DemoFusion requires more passes, but the intermediate results can serve as "previews", facilitating rapid prompt iteration.