Prospective infectious disease outbreak detection using markov switching models

Research output: Contribution to journalArticle

32 Citations (Scopus)

Abstract

Accurate and timely detection of infectious disease outbreaks provides valuable information which can enable public health officials to respond to major public health threats in a timely fashion. However, disease outbreaks are often not directly observable. For surveillance systems used to detect outbreaks, noises caused by routine behavioral patterns and by special events can further complicate the detection task. Most existing detection methods combine a time series filtering procedure followed by a statistical surveillance method. The performance of this "two-step detection method is hampered by the unrealistic assumption that the training data are outbreak-free. Moreover, existing approaches are sensitive to extreme values, which are common in real-world data sets. We considered the problem of identifying outbreak patterns in a syndrome count time series using Markov switching models. The disease outbreak states are modeled as hidden state variables which control the observed time series. A jump component is introduced to absorb sporadic extreme values that may otherwise weaken the ability to detect slow-moving disease outbreaks. Our approach outperformed several state-of-the-art detection methods in terms of detection sensitivity using both simulated and real-world data.

Original languageEnglish (US)
Article number4912199
Pages (from-to)565-577
Number of pages13
JournalIEEE Transactions on Knowledge and Data Engineering
Volume22
Issue number4
DOIs
StatePublished - Apr 2010

Fingerprint

Time series
Public health

Keywords

  • Gibbs sampling
  • Markov switching models
  • Outbreak detection
  • Syndromic surveillance

ASJC Scopus subject areas

  • Computational Theory and Mathematics
  • Information Systems
  • Computer Science Applications

Cite this

Prospective infectious disease outbreak detection using markov switching models. / Lu, Hsin Min; Zeng, Dajun; Chen, Hsinchun.

In: IEEE Transactions on Knowledge and Data Engineering, Vol. 22, No. 4, 4912199, 04.2010, p. 565-577.

Research output: Contribution to journalArticle

@article{031a04c230f84278b925bd874977c8ed,
title = "Prospective infectious disease outbreak detection using markov switching models",
abstract = "Accurate and timely detection of infectious disease outbreaks provides valuable information which can enable public health officials to respond to major public health threats in a timely fashion. However, disease outbreaks are often not directly observable. For surveillance systems used to detect outbreaks, noises caused by routine behavioral patterns and by special events can further complicate the detection task. Most existing detection methods combine a time series filtering procedure followed by a statistical surveillance method. The performance of this {"}two-step detection method is hampered by the unrealistic assumption that the training data are outbreak-free. Moreover, existing approaches are sensitive to extreme values, which are common in real-world data sets. We considered the problem of identifying outbreak patterns in a syndrome count time series using Markov switching models. The disease outbreak states are modeled as hidden state variables which control the observed time series. A jump component is introduced to absorb sporadic extreme values that may otherwise weaken the ability to detect slow-moving disease outbreaks. Our approach outperformed several state-of-the-art detection methods in terms of detection sensitivity using both simulated and real-world data.",
keywords = "Gibbs sampling, Markov switching models, Outbreak detection, Syndromic surveillance",
author = "Lu, {Hsin Min} and Dajun Zeng and Hsinchun Chen",
year = "2010",
month = "4",
doi = "10.1109/TKDE.2009.115",
language = "English (US)",
volume = "22",
pages = "565--577",
journal = "IEEE Transactions on Knowledge and Data Engineering",
issn = "1041-4347",
publisher = "IEEE Computer Society",
number = "4",

}

TY - JOUR

T1 - Prospective infectious disease outbreak detection using markov switching models

AU - Lu, Hsin Min

AU - Zeng, Dajun

AU - Chen, Hsinchun

PY - 2010/4

Y1 - 2010/4

N2 - Accurate and timely detection of infectious disease outbreaks provides valuable information which can enable public health officials to respond to major public health threats in a timely fashion. However, disease outbreaks are often not directly observable. For surveillance systems used to detect outbreaks, noises caused by routine behavioral patterns and by special events can further complicate the detection task. Most existing detection methods combine a time series filtering procedure followed by a statistical surveillance method. The performance of this "two-step detection method is hampered by the unrealistic assumption that the training data are outbreak-free. Moreover, existing approaches are sensitive to extreme values, which are common in real-world data sets. We considered the problem of identifying outbreak patterns in a syndrome count time series using Markov switching models. The disease outbreak states are modeled as hidden state variables which control the observed time series. A jump component is introduced to absorb sporadic extreme values that may otherwise weaken the ability to detect slow-moving disease outbreaks. Our approach outperformed several state-of-the-art detection methods in terms of detection sensitivity using both simulated and real-world data.

AB - Accurate and timely detection of infectious disease outbreaks provides valuable information which can enable public health officials to respond to major public health threats in a timely fashion. However, disease outbreaks are often not directly observable. For surveillance systems used to detect outbreaks, noises caused by routine behavioral patterns and by special events can further complicate the detection task. Most existing detection methods combine a time series filtering procedure followed by a statistical surveillance method. The performance of this "two-step detection method is hampered by the unrealistic assumption that the training data are outbreak-free. Moreover, existing approaches are sensitive to extreme values, which are common in real-world data sets. We considered the problem of identifying outbreak patterns in a syndrome count time series using Markov switching models. The disease outbreak states are modeled as hidden state variables which control the observed time series. A jump component is introduced to absorb sporadic extreme values that may otherwise weaken the ability to detect slow-moving disease outbreaks. Our approach outperformed several state-of-the-art detection methods in terms of detection sensitivity using both simulated and real-world data.

KW - Gibbs sampling

KW - Markov switching models

KW - Outbreak detection

KW - Syndromic surveillance

UR - http://www.scopus.com/inward/record.url?scp=77649264193&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77649264193&partnerID=8YFLogxK

U2 - 10.1109/TKDE.2009.115

DO - 10.1109/TKDE.2009.115

M3 - Article

VL - 22

SP - 565

EP - 577

JO - IEEE Transactions on Knowledge and Data Engineering

JF - IEEE Transactions on Knowledge and Data Engineering

SN - 1041-4347

IS - 4

M1 - 4912199

ER -