Fusion of heterogeneous attention mechanisms in multi-view convolutional neural network for text classification

Yunji Liang, Huihui Li, Bin Guo, Zhiwen Yu, Xiaolong Zheng, Sagar Samtani, Daniel D. Zeng

Research output: Contribution to journalArticlepeer-review

1 Scopus citations

Abstract

The rapid proliferation of user generated content has given rise to large volumes of text corpora. Increasingly, scholars, researchers, and organizations employ text classification to mine novel insights for high-impact applications. Despite their prevalence, conventional text classification methods rely on labor-intensive feature engineering efforts that are task specific, omit long-term relationships, and are not suitable for the rapidly evolving domains. While an increasing body of deep learning and attention mechanism literature aim to address these issues, extant methods often represent text as a single view and omit multiple sets of features at varying levels of granularity. Recognizing that these issues often result in performance degradations, we propose a novel Spatial View Attention Convolutional Neural Network (SVA-CNN). SVA-CNN leverages an innovative and carefully designed set of multi-view representation learning, a combination of heterogeneous attention mechanisms and CNN-based operations to automatically extract and weight multiple granularities and fine-grained representations. Rigorously evaluating SVA-CNN against prevailing text classification methods on five large-scale benchmark datasets indicates its ability to outperform extant deep learning-based classification methods in both performance and training time for document classification, sentiment analysis, and thematic identification applications. To facilitate model reproducibility and extensions, SVA-CNN's source code is also available via GitHub.

Original languageEnglish (US)
Pages (from-to)295-312
Number of pages18
JournalInformation Sciences
Volume548
DOIs
StatePublished - Feb 16 2021
Externally publishedYes

Keywords

  • Conventional neural network
  • Multi-view representation
  • Series and parallel connection
  • Spatial attention
  • Text classification
  • View attention

ASJC Scopus subject areas

  • Software
  • Control and Systems Engineering
  • Theoretical Computer Science
  • Computer Science Applications
  • Information Systems and Management
  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'Fusion of heterogeneous attention mechanisms in multi-view convolutional neural network for text classification'. Together they form a unique fingerprint.

Cite this