Identity Mappings in Deep Residual Networks

March 16, 2016 · Entered Twilight · 🏛 European Conference on Computer Vision

"Last commit was 9.0 years ago (≥5 year threshold)"

Evidence collected by the PWNC Scanner

Repo contents: .gitignore, README.md, resnet-pre-act.lua

Authors Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun arXiv ID 1603.05027 Category cs.CV: Computer Vision Cross-listed cs.LG Citations 11.0K Venue European Conference on Computer Vision Repository https://github.com/KaimingHe/resnet-1k-layers ⭐ 930 Last Checked 1 month ago

Abstract

Deep residual networks have emerged as a family of extremely deep architectures showing compelling accuracy and nice convergence behaviors. In this paper, we analyze the propagation formulations behind the residual building blocks, which suggest that the forward and backward signals can be directly propagated from one block to any other block, when using identity mappings as the skip connections and after-addition activation. A series of ablation experiments support the importance of these identity mappings. This motivates us to propose a new residual unit, which makes training easier and improves generalization. We report improved results using a 1001-layer ResNet on CIFAR-10 (4.62% error) and CIFAR-100, and a 200-layer ResNet on ImageNet. Code is available at: https://github.com/KaimingHe/resnet-1k-layers