A Shortcut-Stacked Document Encoder for Extractive Text Summarization

Peng Yan, Linjing Li, Daniel Zeng

Research output: Chapter in Book/Report/Conference proceedingConference contribution


While doing summarization, human needs to understand the whole document, rather than separately understanding each sentence in the document. However, inter-sentence features within one document are not adequately modeled by previous neural network-based models that almost use only one layer recurrent neural network as document encoder. To learn high quality context-aware representation, we propose a shortcut-stacked document encoder for extractive summarization. We use multiple stacked bidirectional long short-term memory (LSTM) layers and add shortcut connections between LSTM layers to increase representation capacity. The shortcut-stacked document encoder is built on a temporal convolutional neural network-based sentence encoder to capture the hierarchical structure of the document. Then sentence representations encoded by document encoder are fed to a sentence selection classifier for summary extraction. Experiments on the well-known CNN/Daily Mail dataset show that the proposed model outperforms several recently proposed strong baselines, including both extractive and abstractive neural network-based models. Furthermore, the ablation analysis and position analysis also demonstrate the effectiveness of the proposed shortcut-stacked document encoder.

Original languageEnglish (US)
Title of host publication2019 International Joint Conference on Neural Networks, IJCNN 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781728119854
StatePublished - Jul 2019
Externally publishedYes
Event2019 International Joint Conference on Neural Networks, IJCNN 2019 - Budapest, Hungary
Duration: Jul 14 2019Jul 19 2019

Publication series

NameProceedings of the International Joint Conference on Neural Networks


Conference2019 International Joint Conference on Neural Networks, IJCNN 2019

ASJC Scopus subject areas

  • Software
  • Artificial Intelligence

Fingerprint Dive into the research topics of 'A Shortcut-Stacked Document Encoder for Extractive Text Summarization'. Together they form a unique fingerprint.

Cite this