Does delayed measurement affect patient reports of provider performance? Implications for performance measurement of medical assistance with tobacco cessation: A Dental PBRN study

Thomas K. Houston, Joshua S. Richman, Heather L. Coley, Midge N. Ray, Jeroan J. Allison, Gregg H. Gilbert, Judith S Gordon, Catarina I. Kiefe

Research output: Contribution to journalArticle

9 Citations (Scopus)

Abstract

Background: We compared two methods of measuring provider performance of tobacco control activities: immediate "exit cards" versus delayed telephone follow-up surveys. Current standards, e.g. HEDIS, use delayed patient measures that may over or under-estimate overall performance. Methods: Patients completed exit cards in 60 dental practices immediately after a visit to measure whether the provider "asked" about tobacco use, and "advised" the patient to quit. One to six months later patients were asked the same questions by telephone survey. Using the exit cards as the standard, we quantified performance and calculated sensitivity (agreement of those responding yes on telephone surveys compared with exit cards) and specificity (agreement of those responding no) of the delayed measurement. Results: Among 150 patients, 21% reporting being asked about tobacco use on the exit cards and 30% reporting being asked in the delayed surveys. The sensitivity and specificity were 50% and 75%, respectively. Similarly, among 182 tobacco users, 38% reported being advised to quit on the exit cards and this increased to 51% on the delayed surveys. The sensitivity and specificity were 75% and 64%, respectively. Increasing the delay from the visit to the telephone survey resulted in increasing disagreement. Conclusion: Patient reports differed considerably in immediate versus delayed measures. These results have important implications because they suggest that our delayed measures may over-estimate performance. The immediate exit cards should be included in the armamentarium of tools for measuring providers' performance of tobacco control, and perhaps other service delivery.

Original languageEnglish (US)
Article number100
JournalBMC Health Services Research
Volume8
DOIs
StatePublished - 2008
Externally publishedYes

Fingerprint

Tobacco Use Cessation
Medical Assistance
Tooth
Telephone
Tobacco
Tobacco Use
Sensitivity and Specificity
Surveys and Questionnaires

ASJC Scopus subject areas

  • Nursing(all)

Cite this

Does delayed measurement affect patient reports of provider performance? Implications for performance measurement of medical assistance with tobacco cessation : A Dental PBRN study. / Houston, Thomas K.; Richman, Joshua S.; Coley, Heather L.; Ray, Midge N.; Allison, Jeroan J.; Gilbert, Gregg H.; Gordon, Judith S; Kiefe, Catarina I.

In: BMC Health Services Research, Vol. 8, 100, 2008.

Research output: Contribution to journalArticle

Houston, Thomas K. ; Richman, Joshua S. ; Coley, Heather L. ; Ray, Midge N. ; Allison, Jeroan J. ; Gilbert, Gregg H. ; Gordon, Judith S ; Kiefe, Catarina I. / Does delayed measurement affect patient reports of provider performance? Implications for performance measurement of medical assistance with tobacco cessation : A Dental PBRN study. In: BMC Health Services Research. 2008 ; Vol. 8.
@article{6df5640e1d764b39bab3e0aa0110d76a,
title = "Does delayed measurement affect patient reports of provider performance? Implications for performance measurement of medical assistance with tobacco cessation: A Dental PBRN study",
abstract = "Background: We compared two methods of measuring provider performance of tobacco control activities: immediate {"}exit cards{"} versus delayed telephone follow-up surveys. Current standards, e.g. HEDIS, use delayed patient measures that may over or under-estimate overall performance. Methods: Patients completed exit cards in 60 dental practices immediately after a visit to measure whether the provider {"}asked{"} about tobacco use, and {"}advised{"} the patient to quit. One to six months later patients were asked the same questions by telephone survey. Using the exit cards as the standard, we quantified performance and calculated sensitivity (agreement of those responding yes on telephone surveys compared with exit cards) and specificity (agreement of those responding no) of the delayed measurement. Results: Among 150 patients, 21{\%} reporting being asked about tobacco use on the exit cards and 30{\%} reporting being asked in the delayed surveys. The sensitivity and specificity were 50{\%} and 75{\%}, respectively. Similarly, among 182 tobacco users, 38{\%} reported being advised to quit on the exit cards and this increased to 51{\%} on the delayed surveys. The sensitivity and specificity were 75{\%} and 64{\%}, respectively. Increasing the delay from the visit to the telephone survey resulted in increasing disagreement. Conclusion: Patient reports differed considerably in immediate versus delayed measures. These results have important implications because they suggest that our delayed measures may over-estimate performance. The immediate exit cards should be included in the armamentarium of tools for measuring providers' performance of tobacco control, and perhaps other service delivery.",
author = "Houston, {Thomas K.} and Richman, {Joshua S.} and Coley, {Heather L.} and Ray, {Midge N.} and Allison, {Jeroan J.} and Gilbert, {Gregg H.} and Gordon, {Judith S} and Kiefe, {Catarina I.}",
year = "2008",
doi = "10.1186/1472-6963-8-100",
language = "English (US)",
volume = "8",
journal = "BMC Health Services Research",
issn = "1472-6963",
publisher = "BioMed Central",

}

TY - JOUR

T1 - Does delayed measurement affect patient reports of provider performance? Implications for performance measurement of medical assistance with tobacco cessation

T2 - A Dental PBRN study

AU - Houston, Thomas K.

AU - Richman, Joshua S.

AU - Coley, Heather L.

AU - Ray, Midge N.

AU - Allison, Jeroan J.

AU - Gilbert, Gregg H.

AU - Gordon, Judith S

AU - Kiefe, Catarina I.

PY - 2008

Y1 - 2008

N2 - Background: We compared two methods of measuring provider performance of tobacco control activities: immediate "exit cards" versus delayed telephone follow-up surveys. Current standards, e.g. HEDIS, use delayed patient measures that may over or under-estimate overall performance. Methods: Patients completed exit cards in 60 dental practices immediately after a visit to measure whether the provider "asked" about tobacco use, and "advised" the patient to quit. One to six months later patients were asked the same questions by telephone survey. Using the exit cards as the standard, we quantified performance and calculated sensitivity (agreement of those responding yes on telephone surveys compared with exit cards) and specificity (agreement of those responding no) of the delayed measurement. Results: Among 150 patients, 21% reporting being asked about tobacco use on the exit cards and 30% reporting being asked in the delayed surveys. The sensitivity and specificity were 50% and 75%, respectively. Similarly, among 182 tobacco users, 38% reported being advised to quit on the exit cards and this increased to 51% on the delayed surveys. The sensitivity and specificity were 75% and 64%, respectively. Increasing the delay from the visit to the telephone survey resulted in increasing disagreement. Conclusion: Patient reports differed considerably in immediate versus delayed measures. These results have important implications because they suggest that our delayed measures may over-estimate performance. The immediate exit cards should be included in the armamentarium of tools for measuring providers' performance of tobacco control, and perhaps other service delivery.

AB - Background: We compared two methods of measuring provider performance of tobacco control activities: immediate "exit cards" versus delayed telephone follow-up surveys. Current standards, e.g. HEDIS, use delayed patient measures that may over or under-estimate overall performance. Methods: Patients completed exit cards in 60 dental practices immediately after a visit to measure whether the provider "asked" about tobacco use, and "advised" the patient to quit. One to six months later patients were asked the same questions by telephone survey. Using the exit cards as the standard, we quantified performance and calculated sensitivity (agreement of those responding yes on telephone surveys compared with exit cards) and specificity (agreement of those responding no) of the delayed measurement. Results: Among 150 patients, 21% reporting being asked about tobacco use on the exit cards and 30% reporting being asked in the delayed surveys. The sensitivity and specificity were 50% and 75%, respectively. Similarly, among 182 tobacco users, 38% reported being advised to quit on the exit cards and this increased to 51% on the delayed surveys. The sensitivity and specificity were 75% and 64%, respectively. Increasing the delay from the visit to the telephone survey resulted in increasing disagreement. Conclusion: Patient reports differed considerably in immediate versus delayed measures. These results have important implications because they suggest that our delayed measures may over-estimate performance. The immediate exit cards should be included in the armamentarium of tools for measuring providers' performance of tobacco control, and perhaps other service delivery.

UR - http://www.scopus.com/inward/record.url?scp=44449178856&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=44449178856&partnerID=8YFLogxK

U2 - 10.1186/1472-6963-8-100

DO - 10.1186/1472-6963-8-100

M3 - Article

C2 - 18466617

AN - SCOPUS:44449178856

VL - 8

JO - BMC Health Services Research

JF - BMC Health Services Research

SN - 1472-6963

M1 - 100

ER -