Oops! Predicting Unintentional Action in Video

November 25, 2019 · Entered Twilight · 🏛 Computer Vision and Pattern Recognition

"No code URL or promise found in abstract"
"Code repo scraped from project page (backfill)"

Evidence collected by the PWNC Scanner

Repo contents: LICENSE, README.md, binary_classify.py, dataloader.py, eval.py, eval_results.py, fails_classify.py, modules.py, nets.py, plotting.py, py12transforms.py, run, sampler.py, transforms.py, utils

Authors Dave Epstein, Boyuan Chen, Carl Vondrick arXiv ID 1911.11206 Category cs.CV: Computer Vision Cross-listed cs.LG, eess.IV Citations 111 Venue Computer Vision and Pattern Recognition Repository https://github.com/cvlab-columbia/oops ⭐ 80 Last Checked 6 days ago

Abstract

From just a short glance at a video, we can often tell whether a person's action is intentional or not. Can we train a model to recognize this? We introduce a dataset of in-the-wild videos of unintentional action, as well as a suite of tasks for recognizing, localizing, and anticipating its onset. We train a supervised neural network as a baseline and analyze its performance compared to human consistency on the tasks. We also investigate self-supervised representations that leverage natural signals in our dataset, and show the effectiveness of an approach that uses the intrinsic speed of video to perform competitively with highly-supervised pretraining. However, a significant gap between machine and human performance remains. The project website is available at https://oops.cs.columbia.edu