A noise-driven strategy for background estimation and event detection in data streams

Bruce R. Copeland, Min Chen, Brad D. Wade, Linda S Powers

Research output: Contribution to journalArticle

5 Citations (Scopus)

Abstract

A theory is presented for a discrete, finite-horizon, H filter that estimates background in a data stream. Threshold rejection is introduced into the theory by way of an approximation for the H observation innovation. The threshold is simply related to a basis variance that can either be provided as input or accumulated over the data stream. This framework identifies background as the portion of a data stream that varies within the bulk of the noise in the data. Unexpected events in the data stream are therefore synonymous with statistical outliers-especially successive outliers of the same direction. The resulting methodology is robust and suitable for real-time applications. It can handle types of background variation in which smoothing and band pass filtering are ineffective. There are no adjustable parameters because all such quantities either have universal values or are selected using well-defined principles. The performance of the filter is demonstrated using computer simulated data sets and arbitrary instrumental data. Examples of its application are also presented in the fields of finance and computer security.

Original languageEnglish (US)
Pages (from-to)3739-3751
Number of pages13
JournalSignal Processing
Volume86
Issue number12
DOIs
StatePublished - Dec 2006
Externally publishedYes

Fingerprint

Finance
Security of data
Innovation

Keywords

  • Background estimation
  • Kalman filtering
  • Statistical threshold
  • Threshold rejection

ASJC Scopus subject areas

  • Signal Processing
  • Electrical and Electronic Engineering

Cite this

A noise-driven strategy for background estimation and event detection in data streams. / Copeland, Bruce R.; Chen, Min; Wade, Brad D.; Powers, Linda S.

In: Signal Processing, Vol. 86, No. 12, 12.2006, p. 3739-3751.

Research output: Contribution to journalArticle

Copeland, Bruce R. ; Chen, Min ; Wade, Brad D. ; Powers, Linda S. / A noise-driven strategy for background estimation and event detection in data streams. In: Signal Processing. 2006 ; Vol. 86, No. 12. pp. 3739-3751.
@article{66235bcde9ac49b2b750ab57ffc9392c,
title = "A noise-driven strategy for background estimation and event detection in data streams",
abstract = "A theory is presented for a discrete, finite-horizon, H∞ filter that estimates background in a data stream. Threshold rejection is introduced into the theory by way of an approximation for the H∞ observation innovation. The threshold is simply related to a basis variance that can either be provided as input or accumulated over the data stream. This framework identifies background as the portion of a data stream that varies within the bulk of the noise in the data. Unexpected events in the data stream are therefore synonymous with statistical outliers-especially successive outliers of the same direction. The resulting methodology is robust and suitable for real-time applications. It can handle types of background variation in which smoothing and band pass filtering are ineffective. There are no adjustable parameters because all such quantities either have universal values or are selected using well-defined principles. The performance of the filter is demonstrated using computer simulated data sets and arbitrary instrumental data. Examples of its application are also presented in the fields of finance and computer security.",
keywords = "Background estimation, Kalman filtering, Statistical threshold, Threshold rejection",
author = "Copeland, {Bruce R.} and Min Chen and Wade, {Brad D.} and Powers, {Linda S}",
year = "2006",
month = "12",
doi = "10.1016/j.sigpro.2006.03.002",
language = "English (US)",
volume = "86",
pages = "3739--3751",
journal = "Signal Processing",
issn = "0165-1684",
publisher = "Elsevier",
number = "12",

}

TY - JOUR

T1 - A noise-driven strategy for background estimation and event detection in data streams

AU - Copeland, Bruce R.

AU - Chen, Min

AU - Wade, Brad D.

AU - Powers, Linda S

PY - 2006/12

Y1 - 2006/12

N2 - A theory is presented for a discrete, finite-horizon, H∞ filter that estimates background in a data stream. Threshold rejection is introduced into the theory by way of an approximation for the H∞ observation innovation. The threshold is simply related to a basis variance that can either be provided as input or accumulated over the data stream. This framework identifies background as the portion of a data stream that varies within the bulk of the noise in the data. Unexpected events in the data stream are therefore synonymous with statistical outliers-especially successive outliers of the same direction. The resulting methodology is robust and suitable for real-time applications. It can handle types of background variation in which smoothing and band pass filtering are ineffective. There are no adjustable parameters because all such quantities either have universal values or are selected using well-defined principles. The performance of the filter is demonstrated using computer simulated data sets and arbitrary instrumental data. Examples of its application are also presented in the fields of finance and computer security.

AB - A theory is presented for a discrete, finite-horizon, H∞ filter that estimates background in a data stream. Threshold rejection is introduced into the theory by way of an approximation for the H∞ observation innovation. The threshold is simply related to a basis variance that can either be provided as input or accumulated over the data stream. This framework identifies background as the portion of a data stream that varies within the bulk of the noise in the data. Unexpected events in the data stream are therefore synonymous with statistical outliers-especially successive outliers of the same direction. The resulting methodology is robust and suitable for real-time applications. It can handle types of background variation in which smoothing and band pass filtering are ineffective. There are no adjustable parameters because all such quantities either have universal values or are selected using well-defined principles. The performance of the filter is demonstrated using computer simulated data sets and arbitrary instrumental data. Examples of its application are also presented in the fields of finance and computer security.

KW - Background estimation

KW - Kalman filtering

KW - Statistical threshold

KW - Threshold rejection

UR - http://www.scopus.com/inward/record.url?scp=33749445629&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33749445629&partnerID=8YFLogxK

U2 - 10.1016/j.sigpro.2006.03.002

DO - 10.1016/j.sigpro.2006.03.002

M3 - Article

AN - SCOPUS:33749445629

VL - 86

SP - 3739

EP - 3751

JO - Signal Processing

JF - Signal Processing

SN - 0165-1684

IS - 12

ER -