TY - JOUR
T1 - Fusion of heterogeneous attention mechanisms in multi-view convolutional neural network for text classification
AU - Liang, Yunji
AU - Li, Huihui
AU - Guo, Bin
AU - Yu, Zhiwen
AU - Zheng, Xiaolong
AU - Samtani, Sagar
AU - Zeng, Daniel D.
N1 - Funding Information:
This work is supported by the National Key Research and Development Program of China under Grant No.: 2019YFB2102200 , by the ministry of health of China under Grant No.: 2017ZX10303401-002 and 2017YFC1200302, by the natural science foundation of China under Grant No.: 61902320, 71472175, 71602184, 71621002, by national science foundation under Grant No. CNS-1850362 and OAC-1917117, and by the fundamental research funds for the central universities under Grant No.:31020180QD140.
Publisher Copyright:
© 2020 Elsevier Inc.
PY - 2021/2/16
Y1 - 2021/2/16
N2 - The rapid proliferation of user generated content has given rise to large volumes of text corpora. Increasingly, scholars, researchers, and organizations employ text classification to mine novel insights for high-impact applications. Despite their prevalence, conventional text classification methods rely on labor-intensive feature engineering efforts that are task specific, omit long-term relationships, and are not suitable for the rapidly evolving domains. While an increasing body of deep learning and attention mechanism literature aim to address these issues, extant methods often represent text as a single view and omit multiple sets of features at varying levels of granularity. Recognizing that these issues often result in performance degradations, we propose a novel Spatial View Attention Convolutional Neural Network (SVA-CNN). SVA-CNN leverages an innovative and carefully designed set of multi-view representation learning, a combination of heterogeneous attention mechanisms and CNN-based operations to automatically extract and weight multiple granularities and fine-grained representations. Rigorously evaluating SVA-CNN against prevailing text classification methods on five large-scale benchmark datasets indicates its ability to outperform extant deep learning-based classification methods in both performance and training time for document classification, sentiment analysis, and thematic identification applications. To facilitate model reproducibility and extensions, SVA-CNN's source code is also available via GitHub.
AB - The rapid proliferation of user generated content has given rise to large volumes of text corpora. Increasingly, scholars, researchers, and organizations employ text classification to mine novel insights for high-impact applications. Despite their prevalence, conventional text classification methods rely on labor-intensive feature engineering efforts that are task specific, omit long-term relationships, and are not suitable for the rapidly evolving domains. While an increasing body of deep learning and attention mechanism literature aim to address these issues, extant methods often represent text as a single view and omit multiple sets of features at varying levels of granularity. Recognizing that these issues often result in performance degradations, we propose a novel Spatial View Attention Convolutional Neural Network (SVA-CNN). SVA-CNN leverages an innovative and carefully designed set of multi-view representation learning, a combination of heterogeneous attention mechanisms and CNN-based operations to automatically extract and weight multiple granularities and fine-grained representations. Rigorously evaluating SVA-CNN against prevailing text classification methods on five large-scale benchmark datasets indicates its ability to outperform extant deep learning-based classification methods in both performance and training time for document classification, sentiment analysis, and thematic identification applications. To facilitate model reproducibility and extensions, SVA-CNN's source code is also available via GitHub.
KW - Conventional neural network
KW - Multi-view representation
KW - Series and parallel connection
KW - Spatial attention
KW - Text classification
KW - View attention
UR - http://www.scopus.com/inward/record.url?scp=85093960486&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85093960486&partnerID=8YFLogxK
U2 - 10.1016/j.ins.2020.10.021
DO - 10.1016/j.ins.2020.10.021
M3 - Article
AN - SCOPUS:85093960486
VL - 548
SP - 295
EP - 312
JO - Information Sciences
JF - Information Sciences
SN - 0020-0255
ER -