Hidden Two-Stream Convolutional Networks for Action Recognition

April 02, 2017 · Entered Twilight · 🏛 Asian Conference on Computer Vision

"Last commit was 8.0 years ago (≥5 year threshold)"

Evidence collected by the PWNC Scanner

Repo contents: CMakeLists.txt, CONTRIBUTING.md, CONTRIBUTORS.md, INSTALL.md, LICENSE, Makefile, Makefile.config, Makefile.config.example, README.md, docker, docs, include, matlab, models, python, scripts, src, test, tools

Authors Yi Zhu, Zhenzhong Lan, Shawn Newsam, Alexander G. Hauptmann arXiv ID 1704.00389 Category cs.CV: Computer Vision Cross-listed cs.LG, cs.MM Citations 305 Venue Asian Conference on Computer Vision Repository https://github.com/bryanyzhu/Hidden-Two-Stream ⭐ 191 Last Checked 1 month ago

Abstract

Analyzing videos of human actions involves understanding the temporal relationships among video frames. State-of-the-art action recognition approaches rely on traditional optical flow estimation methods to pre-compute motion information for CNNs. Such a two-stage approach is computationally expensive, storage demanding, and not end-to-end trainable. In this paper, we present a novel CNN architecture that implicitly captures motion information between adjacent frames. We name our approach hidden two-stream CNNs because it only takes raw video frames as input and directly predicts action classes without explicitly computing optical flow. Our end-to-end approach is 10x faster than its two-stage baseline. Experimental results on four challenging action recognition datasets: UCF101, HMDB51, THUMOS14 and ActivityNet v1.2 show that our approach significantly outperforms the previous best real-time approaches.