Non-Local Video Denoising by CNN

November 30, 2018 · Entered Twilight · 🏛 arXiv.org

"Last commit was 6.0 years ago (≥5 year threshold)"

Evidence collected by the PWNC Scanner

Repo contents: .gitignore, LICENSE, README.md, dataset.py, models.py, psnr.py, requirements.txt, ssim.py, test.py, train.py, video_patch_search.cl, video_patch_search.py, video_patch_search_acc.pyx, vnlnet_color_05.pth, vnlnet_color_10.pth, vnlnet_color_15.pth, vnlnet_color_20.pth, vnlnet_color_25.pth, vnlnet_color_35.pth, vnlnet_color_40.pth, vnlnet_gray_10.pth, vnlnet_gray_15.pth, vnlnet_gray_20.pth, vnlnet_gray_25.pth, vnlnet_gray_30.pth, vnlnet_gray_40.pth, vnlnet_gray_5.pth, vnlnet_gray_50.pth

Authors Axel Davy, Thibaud Ehret, Jean-Michel Morel, Pablo Arias, Gabriele Facciolo arXiv ID 1811.12758 Category cs.CV: Computer Vision Citations 39 Venue arXiv.org Repository https://github.com/axeldavy/vnlnet ⭐ 86 Last Checked 1 month ago

Abstract

Non-local patch based methods were until recently state-of-the-art for image denoising but are now outperformed by CNNs. Yet they are still the state-of-the-art for video denoising, as video redundancy is a key factor to attain high denoising performance. The problem is that CNN architectures are hardly compatible with the search for self-similarities. In this work we propose a new and efficient way to feed video self-similarities to a CNN. The non-locality is incorporated into the network via a first non-trainable layer which finds for each patch in the input image its most similar patches in a search region. The central values of these patches are then gathered in a feature vector which is assigned to each image pixel. This information is presented to a CNN which is trained to predict the clean image. We apply the proposed architecture to image and video denoising. For the latter patches are searched for in a 3D spatio-temporal volume. The proposed architecture achieves state-of-the-art results. To the best of our knowledge, this is the first successful application of a CNN to video denoising.