Open-Ended Visual Question-Answering

October 09, 2016 · Entered Twilight · 🏛 arXiv.org

"No code URL or promise found in abstract"
"Derived repo from GitHub Pages (backfill)"

Evidence collected by the PWNC Scanner

Repo contents: .gitignore, README.md, bin, data, requirements.txt, setup.py, tests, vqa

Authors Issey Masuda, Santiago Pascual de la Puente, Xavier Giro-i-Nieto arXiv ID 1610.02692 Category cs.CL: Computation & Language Cross-listed cs.CV, cs.MM Citations 9 Venue arXiv.org Repository https://github.com/imatge-upc/vqa-2016-cvprw ⭐ 41 Last Checked 1 month ago

Abstract

This thesis report studies methods to solve Visual Question-Answering (VQA) tasks with a Deep Learning framework. As a preliminary step, we explore Long Short-Term Memory (LSTM) networks used in Natural Language Processing (NLP) to tackle Question-Answering (text based). We then modify the previous model to accept an image as an input in addition to the question. For this purpose, we explore the VGG-16 and K-CNN convolutional neural networks to extract visual features from the image. These are merged with the word embedding or with a sentence embedding of the question to predict the answer. This work was successfully submitted to the Visual Question Answering Challenge 2016, where it achieved a 53,62% of accuracy in the test dataset. The developed software has followed the best programming practices and Python code style, providing a consistent baseline in Keras for different configurations.