Textual analysis of stock market prediction using breaking financial news: The AZFin text system

Robert P. Schumaker, Hsinchun Chen

Research output: Contribution to journalArticle

312 Citations (Scopus)

Abstract

Our research examines a predictive machine learning approach for financial news articles analysis using several different textual representations: bag of words, noun phrases, and named entities. Through this approach, we investigated 9,211 financial news articles and 10,259,042 stock quotes covering the S&P 500 stocks during a five week period. We applied our analysis to estimate a discrete stock price twenty minutes after a news article was released. Using a support vector machine (SVM) derivative specially tailored for discrete numeric prediction and models containing different stock-specific variables, we show that the model containing both article terms and stock price at the time of article release had the best performance in closeness to the actual future stock price (MSE 0.04261), the same direction of price movement as the future price (57.1% directional accuracy) and the highest return using a simulated trading engine (2.06% return). We further investigated the different textual representations and found that a Proper Noun scheme performs better than the de facto standard of Bag of Words in all three metrics.

Original languageEnglish (US)
Article numbera12
JournalACM Transactions on Information Systems
Volume27
Issue number2
DOIs
StatePublished - Feb 1 2009

Fingerprint

Support vector machines
Learning systems
Engines
Derivatives
Financial markets
Prediction
Stock market
Stock prices
News
Textual analysis
Directional accuracy
Futures prices
Machine learning
Named entity
Support vector machine
Closeness

Keywords

  • Prediction
  • Stock market
  • SVM

ASJC Scopus subject areas

  • Information Systems
  • Business, Management and Accounting(all)
  • Computer Science Applications

Cite this

Textual analysis of stock market prediction using breaking financial news : The AZFin text system. / Schumaker, Robert P.; Chen, Hsinchun.

In: ACM Transactions on Information Systems, Vol. 27, No. 2, a12, 01.02.2009.

Research output: Contribution to journalArticle

@article{999abb56de63450f86fb01e488b5566c,
title = "Textual analysis of stock market prediction using breaking financial news: The AZFin text system",
abstract = "Our research examines a predictive machine learning approach for financial news articles analysis using several different textual representations: bag of words, noun phrases, and named entities. Through this approach, we investigated 9,211 financial news articles and 10,259,042 stock quotes covering the S&P 500 stocks during a five week period. We applied our analysis to estimate a discrete stock price twenty minutes after a news article was released. Using a support vector machine (SVM) derivative specially tailored for discrete numeric prediction and models containing different stock-specific variables, we show that the model containing both article terms and stock price at the time of article release had the best performance in closeness to the actual future stock price (MSE 0.04261), the same direction of price movement as the future price (57.1{\%} directional accuracy) and the highest return using a simulated trading engine (2.06{\%} return). We further investigated the different textual representations and found that a Proper Noun scheme performs better than the de facto standard of Bag of Words in all three metrics.",
keywords = "Prediction, Stock market, SVM",
author = "Schumaker, {Robert P.} and Hsinchun Chen",
year = "2009",
month = "2",
day = "1",
doi = "10.1145/1462198.1462204",
language = "English (US)",
volume = "27",
journal = "ACM Transactions on Information Systems",
issn = "1046-8188",
publisher = "Association for Computing Machinery (ACM)",
number = "2",

}

TY - JOUR

T1 - Textual analysis of stock market prediction using breaking financial news

T2 - The AZFin text system

AU - Schumaker, Robert P.

AU - Chen, Hsinchun

PY - 2009/2/1

Y1 - 2009/2/1

N2 - Our research examines a predictive machine learning approach for financial news articles analysis using several different textual representations: bag of words, noun phrases, and named entities. Through this approach, we investigated 9,211 financial news articles and 10,259,042 stock quotes covering the S&P 500 stocks during a five week period. We applied our analysis to estimate a discrete stock price twenty minutes after a news article was released. Using a support vector machine (SVM) derivative specially tailored for discrete numeric prediction and models containing different stock-specific variables, we show that the model containing both article terms and stock price at the time of article release had the best performance in closeness to the actual future stock price (MSE 0.04261), the same direction of price movement as the future price (57.1% directional accuracy) and the highest return using a simulated trading engine (2.06% return). We further investigated the different textual representations and found that a Proper Noun scheme performs better than the de facto standard of Bag of Words in all three metrics.

AB - Our research examines a predictive machine learning approach for financial news articles analysis using several different textual representations: bag of words, noun phrases, and named entities. Through this approach, we investigated 9,211 financial news articles and 10,259,042 stock quotes covering the S&P 500 stocks during a five week period. We applied our analysis to estimate a discrete stock price twenty minutes after a news article was released. Using a support vector machine (SVM) derivative specially tailored for discrete numeric prediction and models containing different stock-specific variables, we show that the model containing both article terms and stock price at the time of article release had the best performance in closeness to the actual future stock price (MSE 0.04261), the same direction of price movement as the future price (57.1% directional accuracy) and the highest return using a simulated trading engine (2.06% return). We further investigated the different textual representations and found that a Proper Noun scheme performs better than the de facto standard of Bag of Words in all three metrics.

KW - Prediction

KW - Stock market

KW - SVM

UR - http://www.scopus.com/inward/record.url?scp=67649304832&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=67649304832&partnerID=8YFLogxK

U2 - 10.1145/1462198.1462204

DO - 10.1145/1462198.1462204

M3 - Article

AN - SCOPUS:67649304832

VL - 27

JO - ACM Transactions on Information Systems

JF - ACM Transactions on Information Systems

SN - 1046-8188

IS - 2

M1 - a12

ER -