Using Patient Care Quality Measures to Assess Educational Outcomes

Susan R. Swing, Sandra Schneider, Ken Bizovi, Dane Chapman, Louis G. Graff, Cherri Hobgood, Thomas Lukens, Martha J. Radford, Arthur B Sanders, Rebecca Smith-Coggins, Linda Spillane, Laura Hruska, Robert L. Wears

Research output: Contribution to journalArticle

18 Citations (Scopus)

Abstract

Objectives: To report the results of a project designed to develop and implement a prototype methodology for identifying candidate patient care quality measures for potential use in assessing the outcomes and effectiveness of graduate medical education in emergency medicine. Methods: A workgroup composed of experts in emergency medicine residency education and patient care quality measurement was convened. Workgroup members performed a modified Delphi process that included iterative review of potential measures; individual expert rating of the measures on four dimensions, including measures quality of care and educational effectiveness; development of consensus on measures to be retained; external stakeholder rating of measures followed by a final workgroup review; and a post hoc stratification of measures. The workgroup completed a structured exercise to examine the linkage of patient care process and outcome measures to educational effectiveness. Results: The workgroup selected 62 measures for inclusion in its final set, including 43 measures for 21 clinical conditions, eight medication measures, seven measures for procedures, and four measures for department efficiency. Twenty-six measures met the more stringent criteria applied post hoc to further stratify and prioritize measures for development. Nineteen of these measures received high ratings from 75% of the workgroup and external stakeholder raters on importance for care in the ED, measures quality of care, and measures educational effectiveness; the majority of the raters considered these indicators feasible to measure. The workgroup utilized a simple framework for exploring the relationship of residency program educational activities, competencies from the six Accreditation Council for Graduate Medical Education general competency domains, patient care quality measures, and external factors that could intervene to affect care quality. Conclusions: Numerous patient care quality measures have potential for use in assessing the educational effectiveness and performance of graduate medical education programs in emergency medicine. The measures identified in this report can be used as a starter set for further development, implementation, and study. Implementation of the measures, especially for high-stakes use, will require resolution of significant measurement issues.

Original languageEnglish (US)
Pages (from-to)463-473
Number of pages11
JournalAcademic Emergency Medicine
Volume14
Issue number5
DOIs
StatePublished - May 2007
Externally publishedYes

Fingerprint

Patient Care
Graduate Medical Education
Emergency Medicine
Quality of Health Care
Internship and Residency
Process Assessment (Health Care)
Accreditation
Patient Education
Consensus
Outcome Assessment (Health Care)
Education

Keywords

  • educational measurement
  • emergency medicine
  • internship and residency
  • outcome and process assessment (health care)
  • program evaluation
  • quality indicators

ASJC Scopus subject areas

  • Emergency Medicine

Cite this

Swing, S. R., Schneider, S., Bizovi, K., Chapman, D., Graff, L. G., Hobgood, C., ... Wears, R. L. (2007). Using Patient Care Quality Measures to Assess Educational Outcomes. Academic Emergency Medicine, 14(5), 463-473. https://doi.org/10.1197/j.aem.2006.12.011

Using Patient Care Quality Measures to Assess Educational Outcomes. / Swing, Susan R.; Schneider, Sandra; Bizovi, Ken; Chapman, Dane; Graff, Louis G.; Hobgood, Cherri; Lukens, Thomas; Radford, Martha J.; Sanders, Arthur B; Smith-Coggins, Rebecca; Spillane, Linda; Hruska, Laura; Wears, Robert L.

In: Academic Emergency Medicine, Vol. 14, No. 5, 05.2007, p. 463-473.

Research output: Contribution to journalArticle

Swing, SR, Schneider, S, Bizovi, K, Chapman, D, Graff, LG, Hobgood, C, Lukens, T, Radford, MJ, Sanders, AB, Smith-Coggins, R, Spillane, L, Hruska, L & Wears, RL 2007, 'Using Patient Care Quality Measures to Assess Educational Outcomes', Academic Emergency Medicine, vol. 14, no. 5, pp. 463-473. https://doi.org/10.1197/j.aem.2006.12.011
Swing SR, Schneider S, Bizovi K, Chapman D, Graff LG, Hobgood C et al. Using Patient Care Quality Measures to Assess Educational Outcomes. Academic Emergency Medicine. 2007 May;14(5):463-473. https://doi.org/10.1197/j.aem.2006.12.011
Swing, Susan R. ; Schneider, Sandra ; Bizovi, Ken ; Chapman, Dane ; Graff, Louis G. ; Hobgood, Cherri ; Lukens, Thomas ; Radford, Martha J. ; Sanders, Arthur B ; Smith-Coggins, Rebecca ; Spillane, Linda ; Hruska, Laura ; Wears, Robert L. / Using Patient Care Quality Measures to Assess Educational Outcomes. In: Academic Emergency Medicine. 2007 ; Vol. 14, No. 5. pp. 463-473.
@article{26dc24760bf643c791ec8830cbd8cffe,
title = "Using Patient Care Quality Measures to Assess Educational Outcomes",
abstract = "Objectives: To report the results of a project designed to develop and implement a prototype methodology for identifying candidate patient care quality measures for potential use in assessing the outcomes and effectiveness of graduate medical education in emergency medicine. Methods: A workgroup composed of experts in emergency medicine residency education and patient care quality measurement was convened. Workgroup members performed a modified Delphi process that included iterative review of potential measures; individual expert rating of the measures on four dimensions, including measures quality of care and educational effectiveness; development of consensus on measures to be retained; external stakeholder rating of measures followed by a final workgroup review; and a post hoc stratification of measures. The workgroup completed a structured exercise to examine the linkage of patient care process and outcome measures to educational effectiveness. Results: The workgroup selected 62 measures for inclusion in its final set, including 43 measures for 21 clinical conditions, eight medication measures, seven measures for procedures, and four measures for department efficiency. Twenty-six measures met the more stringent criteria applied post hoc to further stratify and prioritize measures for development. Nineteen of these measures received high ratings from 75{\%} of the workgroup and external stakeholder raters on importance for care in the ED, measures quality of care, and measures educational effectiveness; the majority of the raters considered these indicators feasible to measure. The workgroup utilized a simple framework for exploring the relationship of residency program educational activities, competencies from the six Accreditation Council for Graduate Medical Education general competency domains, patient care quality measures, and external factors that could intervene to affect care quality. Conclusions: Numerous patient care quality measures have potential for use in assessing the educational effectiveness and performance of graduate medical education programs in emergency medicine. The measures identified in this report can be used as a starter set for further development, implementation, and study. Implementation of the measures, especially for high-stakes use, will require resolution of significant measurement issues.",
keywords = "educational measurement, emergency medicine, internship and residency, outcome and process assessment (health care), program evaluation, quality indicators",
author = "Swing, {Susan R.} and Sandra Schneider and Ken Bizovi and Dane Chapman and Graff, {Louis G.} and Cherri Hobgood and Thomas Lukens and Radford, {Martha J.} and Sanders, {Arthur B} and Rebecca Smith-Coggins and Linda Spillane and Laura Hruska and Wears, {Robert L.}",
year = "2007",
month = "5",
doi = "10.1197/j.aem.2006.12.011",
language = "English (US)",
volume = "14",
pages = "463--473",
journal = "Academic Emergency Medicine",
issn = "1069-6563",
publisher = "Wiley-Blackwell",
number = "5",

}

TY - JOUR

T1 - Using Patient Care Quality Measures to Assess Educational Outcomes

AU - Swing, Susan R.

AU - Schneider, Sandra

AU - Bizovi, Ken

AU - Chapman, Dane

AU - Graff, Louis G.

AU - Hobgood, Cherri

AU - Lukens, Thomas

AU - Radford, Martha J.

AU - Sanders, Arthur B

AU - Smith-Coggins, Rebecca

AU - Spillane, Linda

AU - Hruska, Laura

AU - Wears, Robert L.

PY - 2007/5

Y1 - 2007/5

N2 - Objectives: To report the results of a project designed to develop and implement a prototype methodology for identifying candidate patient care quality measures for potential use in assessing the outcomes and effectiveness of graduate medical education in emergency medicine. Methods: A workgroup composed of experts in emergency medicine residency education and patient care quality measurement was convened. Workgroup members performed a modified Delphi process that included iterative review of potential measures; individual expert rating of the measures on four dimensions, including measures quality of care and educational effectiveness; development of consensus on measures to be retained; external stakeholder rating of measures followed by a final workgroup review; and a post hoc stratification of measures. The workgroup completed a structured exercise to examine the linkage of patient care process and outcome measures to educational effectiveness. Results: The workgroup selected 62 measures for inclusion in its final set, including 43 measures for 21 clinical conditions, eight medication measures, seven measures for procedures, and four measures for department efficiency. Twenty-six measures met the more stringent criteria applied post hoc to further stratify and prioritize measures for development. Nineteen of these measures received high ratings from 75% of the workgroup and external stakeholder raters on importance for care in the ED, measures quality of care, and measures educational effectiveness; the majority of the raters considered these indicators feasible to measure. The workgroup utilized a simple framework for exploring the relationship of residency program educational activities, competencies from the six Accreditation Council for Graduate Medical Education general competency domains, patient care quality measures, and external factors that could intervene to affect care quality. Conclusions: Numerous patient care quality measures have potential for use in assessing the educational effectiveness and performance of graduate medical education programs in emergency medicine. The measures identified in this report can be used as a starter set for further development, implementation, and study. Implementation of the measures, especially for high-stakes use, will require resolution of significant measurement issues.

AB - Objectives: To report the results of a project designed to develop and implement a prototype methodology for identifying candidate patient care quality measures for potential use in assessing the outcomes and effectiveness of graduate medical education in emergency medicine. Methods: A workgroup composed of experts in emergency medicine residency education and patient care quality measurement was convened. Workgroup members performed a modified Delphi process that included iterative review of potential measures; individual expert rating of the measures on four dimensions, including measures quality of care and educational effectiveness; development of consensus on measures to be retained; external stakeholder rating of measures followed by a final workgroup review; and a post hoc stratification of measures. The workgroup completed a structured exercise to examine the linkage of patient care process and outcome measures to educational effectiveness. Results: The workgroup selected 62 measures for inclusion in its final set, including 43 measures for 21 clinical conditions, eight medication measures, seven measures for procedures, and four measures for department efficiency. Twenty-six measures met the more stringent criteria applied post hoc to further stratify and prioritize measures for development. Nineteen of these measures received high ratings from 75% of the workgroup and external stakeholder raters on importance for care in the ED, measures quality of care, and measures educational effectiveness; the majority of the raters considered these indicators feasible to measure. The workgroup utilized a simple framework for exploring the relationship of residency program educational activities, competencies from the six Accreditation Council for Graduate Medical Education general competency domains, patient care quality measures, and external factors that could intervene to affect care quality. Conclusions: Numerous patient care quality measures have potential for use in assessing the educational effectiveness and performance of graduate medical education programs in emergency medicine. The measures identified in this report can be used as a starter set for further development, implementation, and study. Implementation of the measures, especially for high-stakes use, will require resolution of significant measurement issues.

KW - educational measurement

KW - emergency medicine

KW - internship and residency

KW - outcome and process assessment (health care)

KW - program evaluation

KW - quality indicators

UR - http://www.scopus.com/inward/record.url?scp=34247161200&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=34247161200&partnerID=8YFLogxK

U2 - 10.1197/j.aem.2006.12.011

DO - 10.1197/j.aem.2006.12.011

M3 - Article

VL - 14

SP - 463

EP - 473

JO - Academic Emergency Medicine

JF - Academic Emergency Medicine

SN - 1069-6563

IS - 5

ER -