SoPa: Bridging CNNs, RNNs, and Weighted Finite-State Machines

May 15, 2018 · Entered Twilight · 🏛 Annual Meeting of the Association for Computational Linguistics

"Last commit was 7.0 years ago (≥5 year threshold)"

Evidence collected by the PWNC Scanner

Repo contents: .gitignore, LICENSE, README.md, __init__.py, baselines, data.py, data, environment_linux.yml, environment_osx.yml, install.sh, interpret_classification_results.py, mlp.py, rnn.py, scripts, soft_patterns.py, soft_patterns_test.py, test, util.py, visualize.py, visualize_efficiently.py

Authors Roy Schwartz, Sam Thomson, Noah A. Smith arXiv ID 1805.06061 Category cs.CL: Computation & Language Cross-listed cs.AI, cs.LG Citations 24 Venue Annual Meeting of the Association for Computational Linguistics Repository https://github.com/Noahs-ARK/soft_patterns ⭐ 54 Last Checked 1 month ago

Abstract

Recurrent and convolutional neural networks comprise two distinct families of models that have proven to be useful for encoding natural language utterances. In this paper we present SoPa, a new model that aims to bridge these two approaches. SoPa combines neural representation learning with weighted finite-state automata (WFSAs) to learn a soft version of traditional surface patterns. We show that SoPa is an extension of a one-layer CNN, and that such CNNs are equivalent to a restricted version of SoPa, and accordingly, to a restricted form of WFSA. Empirically, on three text classification tasks, SoPa is comparable or better than both a BiLSTM (RNN) baseline and a CNN baseline, and is particularly useful in small data settings.