Contrastive Attention Mechanism for Abstractive Sentence Summarization

October 29, 2019 · Entered Twilight · 🏛 Conference on Empirical Methods in Natural Language Processing

"Last commit was 6.0 years ago (≥5 year threshold)"

Evidence collected by the PWNC Scanner

Repo contents: CONTRIBUTING.md, LICENSE, PATENTS, README.md, distributed_train.py, docs, eval_lm.py, fairseq.gif, fairseq, generate.py, interactive.py, multiprocessing_train.py, preprocess.py, requirements.txt, score.py, scripts, setup.py, tests, train.py

Authors Xiangyu Duan, Hoongfei Yu, Mingming Yin, Min Zhang, Weihua Luo, Yue Zhang arXiv ID 1910.13114 Category cs.CL: Computation & Language Cross-listed cs.AI Citations 53 Venue Conference on Empirical Methods in Natural Language Processing Repository https://github.com/travel-go/Abstractive-Text-Summarization ⭐ 40 Last Checked 1 month ago

Abstract

We propose a contrastive attention mechanism to extend the sequence-to-sequence framework for abstractive sentence summarization task, which aims to generate a brief summary of a given source sentence. The proposed contrastive attention mechanism accommodates two categories of attention: one is the conventional attention that attends to relevant parts of the source sentence, the other is the opponent attention that attends to irrelevant or less relevant parts of the source sentence. Both attentions are trained in an opposite way so that the contribution from the conventional attention is encouraged and the contribution from the opponent attention is discouraged through a novel softmax and softmin functionality. Experiments on benchmark datasets show that, the proposed contrastive attention mechanism is more focused on the relevant parts for the summary than the conventional attention mechanism, and greatly advances the state-of-the-art performance on the abstractive sentence summarization task. We release the code at https://github.com/travel-go/Abstractive-Text-Summarization