The state-of-the-art in twitter sentiment analysis: A review and benchmark evaluation

David Zimbra, Ahmed Abbasi, Dajun Zeng, Hsinchun Chen

Research output: Contribution to journalReview article

2 Citations (Scopus)

Abstract

Twitter has emerged as a major social media platform and generated great interest from sentiment analysis researchers. Despite this attention, state-of-the-art Twitter sentiment analysis approaches perform relatively poorly with reported classification accuracies often below 70%, adversely impacting applications of the derived sentiment information. In this research, we investigate the unique challenges presented by Twitter sentiment analysis and review the literature to determine how the devised approaches have addressed these challenges. To assess the state-of-the-art in Twitter sentiment analysis, we conduct a benchmark evaluation of 28 top academic and commercial systems in tweet sentiment classification across five distinctive data sets. We perform an error analysis to uncover the causes of commonly occurring classification errors. To further the evaluation, we apply select systems in an event detection case study. Finally, we summarize the key trends and takeaways from the review and benchmark evaluation and provide suggestions to guide the design of the next generation of approaches.

Original languageEnglish (US)
Article number5
JournalACM Transactions on Management Information Systems
Volume9
Issue number2
DOIs
StatePublished - Apr 1 2018

Fingerprint

Error analysis
Sentiment analysis
Benchmark
Evaluation
Twitter
Sentiment
Event detection
Social media
Sentiment classification

Keywords

  • Benchmark evaluation
  • Natural language processing
  • Opinion mining
  • Sentiment analysis
  • Social media
  • Text mining
  • Twitter

ASJC Scopus subject areas

  • Management Information Systems
  • Computer Science(all)

Cite this

The state-of-the-art in twitter sentiment analysis : A review and benchmark evaluation. / Zimbra, David; Abbasi, Ahmed; Zeng, Dajun; Chen, Hsinchun.

In: ACM Transactions on Management Information Systems, Vol. 9, No. 2, 5, 01.04.2018.

Research output: Contribution to journalReview article

@article{56622554082144859b6c5cf5ff238ad8,
title = "The state-of-the-art in twitter sentiment analysis: A review and benchmark evaluation",
abstract = "Twitter has emerged as a major social media platform and generated great interest from sentiment analysis researchers. Despite this attention, state-of-the-art Twitter sentiment analysis approaches perform relatively poorly with reported classification accuracies often below 70{\%}, adversely impacting applications of the derived sentiment information. In this research, we investigate the unique challenges presented by Twitter sentiment analysis and review the literature to determine how the devised approaches have addressed these challenges. To assess the state-of-the-art in Twitter sentiment analysis, we conduct a benchmark evaluation of 28 top academic and commercial systems in tweet sentiment classification across five distinctive data sets. We perform an error analysis to uncover the causes of commonly occurring classification errors. To further the evaluation, we apply select systems in an event detection case study. Finally, we summarize the key trends and takeaways from the review and benchmark evaluation and provide suggestions to guide the design of the next generation of approaches.",
keywords = "Benchmark evaluation, Natural language processing, Opinion mining, Sentiment analysis, Social media, Text mining, Twitter",
author = "David Zimbra and Ahmed Abbasi and Dajun Zeng and Hsinchun Chen",
year = "2018",
month = "4",
day = "1",
doi = "10.1145/3185045",
language = "English (US)",
volume = "9",
journal = "ACM Transactions on Management Information Systems",
issn = "2158-656X",
publisher = "Association for Computing Machinery (ACM)",
number = "2",

}

TY - JOUR

T1 - The state-of-the-art in twitter sentiment analysis

T2 - A review and benchmark evaluation

AU - Zimbra, David

AU - Abbasi, Ahmed

AU - Zeng, Dajun

AU - Chen, Hsinchun

PY - 2018/4/1

Y1 - 2018/4/1

N2 - Twitter has emerged as a major social media platform and generated great interest from sentiment analysis researchers. Despite this attention, state-of-the-art Twitter sentiment analysis approaches perform relatively poorly with reported classification accuracies often below 70%, adversely impacting applications of the derived sentiment information. In this research, we investigate the unique challenges presented by Twitter sentiment analysis and review the literature to determine how the devised approaches have addressed these challenges. To assess the state-of-the-art in Twitter sentiment analysis, we conduct a benchmark evaluation of 28 top academic and commercial systems in tweet sentiment classification across five distinctive data sets. We perform an error analysis to uncover the causes of commonly occurring classification errors. To further the evaluation, we apply select systems in an event detection case study. Finally, we summarize the key trends and takeaways from the review and benchmark evaluation and provide suggestions to guide the design of the next generation of approaches.

AB - Twitter has emerged as a major social media platform and generated great interest from sentiment analysis researchers. Despite this attention, state-of-the-art Twitter sentiment analysis approaches perform relatively poorly with reported classification accuracies often below 70%, adversely impacting applications of the derived sentiment information. In this research, we investigate the unique challenges presented by Twitter sentiment analysis and review the literature to determine how the devised approaches have addressed these challenges. To assess the state-of-the-art in Twitter sentiment analysis, we conduct a benchmark evaluation of 28 top academic and commercial systems in tweet sentiment classification across five distinctive data sets. We perform an error analysis to uncover the causes of commonly occurring classification errors. To further the evaluation, we apply select systems in an event detection case study. Finally, we summarize the key trends and takeaways from the review and benchmark evaluation and provide suggestions to guide the design of the next generation of approaches.

KW - Benchmark evaluation

KW - Natural language processing

KW - Opinion mining

KW - Sentiment analysis

KW - Social media

KW - Text mining

KW - Twitter

UR - http://www.scopus.com/inward/record.url?scp=85053528947&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85053528947&partnerID=8YFLogxK

U2 - 10.1145/3185045

DO - 10.1145/3185045

M3 - Review article

AN - SCOPUS:85053528947

VL - 9

JO - ACM Transactions on Management Information Systems

JF - ACM Transactions on Management Information Systems

SN - 2158-656X

IS - 2

M1 - 5

ER -