Comparing cardiac ejection fraction estimation algorithms without a gold standard

Matthew A Kupinski, John W. Hoppin, Joshua Krasnow, Seth Dahlberg, Jeffrey A. Leppo, Michael A. King, Eric W Clarkson, Harrison H Barrett

Research output: Contribution to journalArticle

16 Citations (Scopus)

Abstract

Rationale and Objectives. Imaging and estimation of left ventricular function have major diagnostic and prognostic importance in patients with coronary artery disease. It is vital that the method used to estimate cardiac ejection fraction (EF) allows the observer to best perform this task. To measure task-based performance, one must clearly define the task in question, the observer performing the task, and the patient population being imaged. In this report, the task is to accurately and precisely measure cardiac EF, and the observers are human-assisted computer algorithms that analyze the images and estimate cardiac EF. It is very difficult to measure the performance of an observer by using clinical data because estimation tasks typically lack a gold standard. A solution to this "no-gold-standard" problem recently was proposed, called regression without truth (RWT). Materials and Methods. Results of three different software packages used to analyze gated, cardiac, and nuclear medicine images, each of which uses a different algorithm to estimate a patient's cardiac EF, are compared. The three methods are the Emory method, Quantitative Gated Single-Photon Emission Computed Tomographic method, and the Wackers-Liu Circumferential Quantification method. The same set of images is used as input to each of the three algorithms. Data were analyzed from the three different algorithms by using RWT to determine which produces the best estimates of cardiac EF in terms of accuracy and precision. Results and Discussion. In performing this study, three different consistency checks were developed to ensure that the RWT method is working properly. The Emory method of estimating EF slightly outperformed the other two methods. In addition, the RWT method passed all three consistency checks, garnering confidence in the method and its application to clinical data.

Original languageEnglish (US)
Pages (from-to)329-337
Number of pages9
JournalAcademic Radiology
Volume13
Issue number3
DOIs
StatePublished - Mar 2006

Fingerprint

Computer-Assisted Image Processing
Nuclear Medicine
Task Performance and Analysis
Photons
Left Ventricular Function
Gold
Coronary Artery Disease
Software
Population

Keywords

  • Cardiac ejection fraction
  • Gold standard
  • Image quality

ASJC Scopus subject areas

  • Radiology Nuclear Medicine and imaging

Cite this

Comparing cardiac ejection fraction estimation algorithms without a gold standard. / Kupinski, Matthew A; Hoppin, John W.; Krasnow, Joshua; Dahlberg, Seth; Leppo, Jeffrey A.; King, Michael A.; Clarkson, Eric W; Barrett, Harrison H.

In: Academic Radiology, Vol. 13, No. 3, 03.2006, p. 329-337.

Research output: Contribution to journalArticle

Kupinski, Matthew A ; Hoppin, John W. ; Krasnow, Joshua ; Dahlberg, Seth ; Leppo, Jeffrey A. ; King, Michael A. ; Clarkson, Eric W ; Barrett, Harrison H. / Comparing cardiac ejection fraction estimation algorithms without a gold standard. In: Academic Radiology. 2006 ; Vol. 13, No. 3. pp. 329-337.
@article{f6ccd5b0d6504b0aa401671f346154f0,
title = "Comparing cardiac ejection fraction estimation algorithms without a gold standard",
abstract = "Rationale and Objectives. Imaging and estimation of left ventricular function have major diagnostic and prognostic importance in patients with coronary artery disease. It is vital that the method used to estimate cardiac ejection fraction (EF) allows the observer to best perform this task. To measure task-based performance, one must clearly define the task in question, the observer performing the task, and the patient population being imaged. In this report, the task is to accurately and precisely measure cardiac EF, and the observers are human-assisted computer algorithms that analyze the images and estimate cardiac EF. It is very difficult to measure the performance of an observer by using clinical data because estimation tasks typically lack a gold standard. A solution to this {"}no-gold-standard{"} problem recently was proposed, called regression without truth (RWT). Materials and Methods. Results of three different software packages used to analyze gated, cardiac, and nuclear medicine images, each of which uses a different algorithm to estimate a patient's cardiac EF, are compared. The three methods are the Emory method, Quantitative Gated Single-Photon Emission Computed Tomographic method, and the Wackers-Liu Circumferential Quantification method. The same set of images is used as input to each of the three algorithms. Data were analyzed from the three different algorithms by using RWT to determine which produces the best estimates of cardiac EF in terms of accuracy and precision. Results and Discussion. In performing this study, three different consistency checks were developed to ensure that the RWT method is working properly. The Emory method of estimating EF slightly outperformed the other two methods. In addition, the RWT method passed all three consistency checks, garnering confidence in the method and its application to clinical data.",
keywords = "Cardiac ejection fraction, Gold standard, Image quality",
author = "Kupinski, {Matthew A} and Hoppin, {John W.} and Joshua Krasnow and Seth Dahlberg and Leppo, {Jeffrey A.} and King, {Michael A.} and Clarkson, {Eric W} and Barrett, {Harrison H}",
year = "2006",
month = "3",
doi = "10.1016/j.acra.2005.12.005",
language = "English (US)",
volume = "13",
pages = "329--337",
journal = "Academic Radiology",
issn = "1076-6332",
publisher = "Elsevier USA",
number = "3",

}

TY - JOUR

T1 - Comparing cardiac ejection fraction estimation algorithms without a gold standard

AU - Kupinski, Matthew A

AU - Hoppin, John W.

AU - Krasnow, Joshua

AU - Dahlberg, Seth

AU - Leppo, Jeffrey A.

AU - King, Michael A.

AU - Clarkson, Eric W

AU - Barrett, Harrison H

PY - 2006/3

Y1 - 2006/3

N2 - Rationale and Objectives. Imaging and estimation of left ventricular function have major diagnostic and prognostic importance in patients with coronary artery disease. It is vital that the method used to estimate cardiac ejection fraction (EF) allows the observer to best perform this task. To measure task-based performance, one must clearly define the task in question, the observer performing the task, and the patient population being imaged. In this report, the task is to accurately and precisely measure cardiac EF, and the observers are human-assisted computer algorithms that analyze the images and estimate cardiac EF. It is very difficult to measure the performance of an observer by using clinical data because estimation tasks typically lack a gold standard. A solution to this "no-gold-standard" problem recently was proposed, called regression without truth (RWT). Materials and Methods. Results of three different software packages used to analyze gated, cardiac, and nuclear medicine images, each of which uses a different algorithm to estimate a patient's cardiac EF, are compared. The three methods are the Emory method, Quantitative Gated Single-Photon Emission Computed Tomographic method, and the Wackers-Liu Circumferential Quantification method. The same set of images is used as input to each of the three algorithms. Data were analyzed from the three different algorithms by using RWT to determine which produces the best estimates of cardiac EF in terms of accuracy and precision. Results and Discussion. In performing this study, three different consistency checks were developed to ensure that the RWT method is working properly. The Emory method of estimating EF slightly outperformed the other two methods. In addition, the RWT method passed all three consistency checks, garnering confidence in the method and its application to clinical data.

AB - Rationale and Objectives. Imaging and estimation of left ventricular function have major diagnostic and prognostic importance in patients with coronary artery disease. It is vital that the method used to estimate cardiac ejection fraction (EF) allows the observer to best perform this task. To measure task-based performance, one must clearly define the task in question, the observer performing the task, and the patient population being imaged. In this report, the task is to accurately and precisely measure cardiac EF, and the observers are human-assisted computer algorithms that analyze the images and estimate cardiac EF. It is very difficult to measure the performance of an observer by using clinical data because estimation tasks typically lack a gold standard. A solution to this "no-gold-standard" problem recently was proposed, called regression without truth (RWT). Materials and Methods. Results of three different software packages used to analyze gated, cardiac, and nuclear medicine images, each of which uses a different algorithm to estimate a patient's cardiac EF, are compared. The three methods are the Emory method, Quantitative Gated Single-Photon Emission Computed Tomographic method, and the Wackers-Liu Circumferential Quantification method. The same set of images is used as input to each of the three algorithms. Data were analyzed from the three different algorithms by using RWT to determine which produces the best estimates of cardiac EF in terms of accuracy and precision. Results and Discussion. In performing this study, three different consistency checks were developed to ensure that the RWT method is working properly. The Emory method of estimating EF slightly outperformed the other two methods. In addition, the RWT method passed all three consistency checks, garnering confidence in the method and its application to clinical data.

KW - Cardiac ejection fraction

KW - Gold standard

KW - Image quality

UR - http://www.scopus.com/inward/record.url?scp=32644479192&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=32644479192&partnerID=8YFLogxK

U2 - 10.1016/j.acra.2005.12.005

DO - 10.1016/j.acra.2005.12.005

M3 - Article

C2 - 16488845

AN - SCOPUS:32644479192

VL - 13

SP - 329

EP - 337

JO - Academic Radiology

JF - Academic Radiology

SN - 1076-6332

IS - 3

ER -