Performance evaluation of an automated single-channel sleep-wake detection algorithm

Richard F. Kaplan, Ying Wang, Kenneth A. Loparo, Monica R. Kelly, Richard R Bootzin

Research output: Contribution to journalArticle

18 Citations (Scopus)

Abstract

Background: A need exists, from both a clinical and a research standpoint, for objective sleep measurement systems that are both easy to use and can accurately assess sleep and wake. This study evaluates the output of an automated sleep-wake detection algorithm (Z-ALG) used in the Zmachine (a portable, single-channel, electroencephalographic [EEG] acquisition and analysis system) against laboratory polysomnography (PSG) using a consensus of expert visual scorers.

Methods: Overnight laboratory PSG studies from 99 subjects (52 females/47 males, 18-60 years, median age 32.7 years), including both normal sleepers and those with a variety of sleep disorders, were assessed. PSG data obtained from the differential mastoids (A1-A2) were assessed by Z-ALG, which determines sleep versus wake every 30 seconds using low-frequency, intermediate-frequency, and high-frequency and time domain EEG features. PSG data were independently scored by two to four certified PSG technologists, using standard Rechtschaffen and Kales guidelines, and these score files were combined on an epoch-by-epoch basis, using a majority voting rule, to generate a single score file per subject to compare against the Z-ALG output. Both epoch-by-epoch and standard sleep indices (eg, total sleep time, sleep efficiency, latency to persistent sleep, and wake after sleep onset) were compared between the Z-ALG output and the technologist consensus score files.

Results: Overall, the sensitivity and specificity for detecting sleep using the Z-ALG as compared to the technologist consensus are 95.5% and 92.5%, respectively, across all subjects, and the positive predictive value and the negative predictive value for detecting sleep are 98.0% and 84.2%, respectively. Overall κ agreement is 0.85 (approaching the level of agreement observed among sleep technologists). These results persist when the sleep disorder subgroups are analyzed separately.

Conclusion: This study demonstrates that the Z-ALG automated sleep-wake detection algorithm, using the single A1-A2 EEG channel, has a level of accuracy that is similar to PSG technologists in the scoring of sleep and wake, thereby making it suitable for a variety of in-home monitoring applications, such as in conjunction with the Zmachine system.

Original languageEnglish (US)
Pages (from-to)113-122
Number of pages10
JournalNature and Science of Sleep
Volume6
DOIs
StatePublished - 2014

Fingerprint

Sleep
Polysomnography
Consensus
varespladib methyl
Mastoid
Brassica
Politics
Guidelines
Sensitivity and Specificity

Keywords

  • Algorithm
  • Automatic sleep scoring
  • EEG
  • Single channel
  • Sleep-wake detection
  • Zmachine

ASJC Scopus subject areas

  • Applied Psychology
  • Behavioral Neuroscience

Cite this

Performance evaluation of an automated single-channel sleep-wake detection algorithm. / Kaplan, Richard F.; Wang, Ying; Loparo, Kenneth A.; Kelly, Monica R.; Bootzin, Richard R.

In: Nature and Science of Sleep, Vol. 6, 2014, p. 113-122.

Research output: Contribution to journalArticle

Kaplan, Richard F. ; Wang, Ying ; Loparo, Kenneth A. ; Kelly, Monica R. ; Bootzin, Richard R. / Performance evaluation of an automated single-channel sleep-wake detection algorithm. In: Nature and Science of Sleep. 2014 ; Vol. 6. pp. 113-122.
@article{2a9a60f29d8843ebad5670fd09f247a5,
title = "Performance evaluation of an automated single-channel sleep-wake detection algorithm",
abstract = "Background: A need exists, from both a clinical and a research standpoint, for objective sleep measurement systems that are both easy to use and can accurately assess sleep and wake. This study evaluates the output of an automated sleep-wake detection algorithm (Z-ALG) used in the Zmachine (a portable, single-channel, electroencephalographic [EEG] acquisition and analysis system) against laboratory polysomnography (PSG) using a consensus of expert visual scorers.Methods: Overnight laboratory PSG studies from 99 subjects (52 females/47 males, 18-60 years, median age 32.7 years), including both normal sleepers and those with a variety of sleep disorders, were assessed. PSG data obtained from the differential mastoids (A1-A2) were assessed by Z-ALG, which determines sleep versus wake every 30 seconds using low-frequency, intermediate-frequency, and high-frequency and time domain EEG features. PSG data were independently scored by two to four certified PSG technologists, using standard Rechtschaffen and Kales guidelines, and these score files were combined on an epoch-by-epoch basis, using a majority voting rule, to generate a single score file per subject to compare against the Z-ALG output. Both epoch-by-epoch and standard sleep indices (eg, total sleep time, sleep efficiency, latency to persistent sleep, and wake after sleep onset) were compared between the Z-ALG output and the technologist consensus score files.Results: Overall, the sensitivity and specificity for detecting sleep using the Z-ALG as compared to the technologist consensus are 95.5{\%} and 92.5{\%}, respectively, across all subjects, and the positive predictive value and the negative predictive value for detecting sleep are 98.0{\%} and 84.2{\%}, respectively. Overall κ agreement is 0.85 (approaching the level of agreement observed among sleep technologists). These results persist when the sleep disorder subgroups are analyzed separately.Conclusion: This study demonstrates that the Z-ALG automated sleep-wake detection algorithm, using the single A1-A2 EEG channel, has a level of accuracy that is similar to PSG technologists in the scoring of sleep and wake, thereby making it suitable for a variety of in-home monitoring applications, such as in conjunction with the Zmachine system.",
keywords = "Algorithm, Automatic sleep scoring, EEG, Single channel, Sleep-wake detection, Zmachine",
author = "Kaplan, {Richard F.} and Ying Wang and Loparo, {Kenneth A.} and Kelly, {Monica R.} and Bootzin, {Richard R}",
year = "2014",
doi = "10.2147/NSS.S71159",
language = "English (US)",
volume = "6",
pages = "113--122",
journal = "Nature and Science of Sleep",
issn = "1179-1608",
publisher = "Dove Medical Press Ltd.",

}

TY - JOUR

T1 - Performance evaluation of an automated single-channel sleep-wake detection algorithm

AU - Kaplan, Richard F.

AU - Wang, Ying

AU - Loparo, Kenneth A.

AU - Kelly, Monica R.

AU - Bootzin, Richard R

PY - 2014

Y1 - 2014

N2 - Background: A need exists, from both a clinical and a research standpoint, for objective sleep measurement systems that are both easy to use and can accurately assess sleep and wake. This study evaluates the output of an automated sleep-wake detection algorithm (Z-ALG) used in the Zmachine (a portable, single-channel, electroencephalographic [EEG] acquisition and analysis system) against laboratory polysomnography (PSG) using a consensus of expert visual scorers.Methods: Overnight laboratory PSG studies from 99 subjects (52 females/47 males, 18-60 years, median age 32.7 years), including both normal sleepers and those with a variety of sleep disorders, were assessed. PSG data obtained from the differential mastoids (A1-A2) were assessed by Z-ALG, which determines sleep versus wake every 30 seconds using low-frequency, intermediate-frequency, and high-frequency and time domain EEG features. PSG data were independently scored by two to four certified PSG technologists, using standard Rechtschaffen and Kales guidelines, and these score files were combined on an epoch-by-epoch basis, using a majority voting rule, to generate a single score file per subject to compare against the Z-ALG output. Both epoch-by-epoch and standard sleep indices (eg, total sleep time, sleep efficiency, latency to persistent sleep, and wake after sleep onset) were compared between the Z-ALG output and the technologist consensus score files.Results: Overall, the sensitivity and specificity for detecting sleep using the Z-ALG as compared to the technologist consensus are 95.5% and 92.5%, respectively, across all subjects, and the positive predictive value and the negative predictive value for detecting sleep are 98.0% and 84.2%, respectively. Overall κ agreement is 0.85 (approaching the level of agreement observed among sleep technologists). These results persist when the sleep disorder subgroups are analyzed separately.Conclusion: This study demonstrates that the Z-ALG automated sleep-wake detection algorithm, using the single A1-A2 EEG channel, has a level of accuracy that is similar to PSG technologists in the scoring of sleep and wake, thereby making it suitable for a variety of in-home monitoring applications, such as in conjunction with the Zmachine system.

AB - Background: A need exists, from both a clinical and a research standpoint, for objective sleep measurement systems that are both easy to use and can accurately assess sleep and wake. This study evaluates the output of an automated sleep-wake detection algorithm (Z-ALG) used in the Zmachine (a portable, single-channel, electroencephalographic [EEG] acquisition and analysis system) against laboratory polysomnography (PSG) using a consensus of expert visual scorers.Methods: Overnight laboratory PSG studies from 99 subjects (52 females/47 males, 18-60 years, median age 32.7 years), including both normal sleepers and those with a variety of sleep disorders, were assessed. PSG data obtained from the differential mastoids (A1-A2) were assessed by Z-ALG, which determines sleep versus wake every 30 seconds using low-frequency, intermediate-frequency, and high-frequency and time domain EEG features. PSG data were independently scored by two to four certified PSG technologists, using standard Rechtschaffen and Kales guidelines, and these score files were combined on an epoch-by-epoch basis, using a majority voting rule, to generate a single score file per subject to compare against the Z-ALG output. Both epoch-by-epoch and standard sleep indices (eg, total sleep time, sleep efficiency, latency to persistent sleep, and wake after sleep onset) were compared between the Z-ALG output and the technologist consensus score files.Results: Overall, the sensitivity and specificity for detecting sleep using the Z-ALG as compared to the technologist consensus are 95.5% and 92.5%, respectively, across all subjects, and the positive predictive value and the negative predictive value for detecting sleep are 98.0% and 84.2%, respectively. Overall κ agreement is 0.85 (approaching the level of agreement observed among sleep technologists). These results persist when the sleep disorder subgroups are analyzed separately.Conclusion: This study demonstrates that the Z-ALG automated sleep-wake detection algorithm, using the single A1-A2 EEG channel, has a level of accuracy that is similar to PSG technologists in the scoring of sleep and wake, thereby making it suitable for a variety of in-home monitoring applications, such as in conjunction with the Zmachine system.

KW - Algorithm

KW - Automatic sleep scoring

KW - EEG

KW - Single channel

KW - Sleep-wake detection

KW - Zmachine

UR - http://www.scopus.com/inward/record.url?scp=84912036007&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84912036007&partnerID=8YFLogxK

U2 - 10.2147/NSS.S71159

DO - 10.2147/NSS.S71159

M3 - Article

AN - SCOPUS:84912036007

VL - 6

SP - 113

EP - 122

JO - Nature and Science of Sleep

JF - Nature and Science of Sleep

SN - 1179-1608

ER -