MovieQA: Understanding Stories in Movies through Question-Answering

December 09, 2015 · Entered Twilight · 🏛 Computer Vision and Pattern Recognition

"Last commit was 9.0 years ago (≥5 year threshold)"

Evidence collected by the PWNC Scanner

Repo contents: .gitignore, README.md, cosine_similarity.py, descriptor_cache, encode_qa_and_text.py, hasty_machine.py, logs, memN2N_text.py, memory_network_text.py, models, movieqa_importer.py, results, sscb.py, sscb_cache, tfidf.py, train_split.json, upload_these, utils.py

Authors Makarand Tapaswi, Yukun Zhu, Rainer Stiefelhagen, Antonio Torralba, Raquel Urtasun, Sanja Fidler arXiv ID 1512.02902 Category cs.CV: Computer Vision Cross-listed cs.CL Citations 795 Venue Computer Vision and Pattern Recognition Repository https://github.com/makarandtapaswi/MovieQA_CVPR2016/ ⭐ 78 Last Checked 1 month ago

Abstract

We introduce the MovieQA dataset which aims to evaluate automatic story comprehension from both video and text. The dataset consists of 14,944 questions about 408 movies with high semantic diversity. The questions range from simpler "Who" did "What" to "Whom", to "Why" and "How" certain events occurred. Each question comes with a set of five possible answers; a correct one and four deceiving answers provided by human annotators. Our dataset is unique in that it contains multiple sources of information -- video clips, plots, subtitles, scripts, and DVS. We analyze our data through various statistics and methods. We further extend existing QA techniques to show that question-answering with such open-ended semantics is hard. We make this data set public along with an evaluation benchmark to encourage inspiring work in this challenging domain.