An evaluation of the validity of inferences made from 3 diabetes assessment instruments

A Rasch analysis

Lisa Izzo Handley, Terri L Warholak, Terrence R. Jackson

Research output: Contribution to journalArticle

9 Citations (Scopus)

Abstract

Background: Many patients who have diabetes lack adequate knowledge, positive attitudes, and sufficient resources to achieve optimal outcomes in the management of their disease. A key to assessing the impact of pharmacists' interactions with diabetes patients is the resultant impact on patient knowledge, attitudes, and satisfaction. As such, it is important that quality instruments be used to obtain accurate and dependable measures of these outcomes. Objective: To evaluate the validity of inferences made from 3 separate diabetes instruments used in the assessment of patient knowledge, attitudes, and satisfaction. Methods: This pilot study enrolled 30 patients with diabetes mellitus to evaluate the following 3 instruments: (1) Check Your Hemoglobin A1CIQ, (2) the revised Diabetes Questionnaire, and (3) a satisfaction questionnaire. The instruments were used to assess patient knowledge of diabetes and diabetes-related complications, attitudes toward having diabetes mellitus, and satisfaction with the services provided in a pharmacist-run diabetes clinic, respectively. Rasch analysis was used to determine if the instruments were able to measure the concepts they are intended to measure when used in this sample. Results: After evaluating the 3 instruments, it was determined that Instrument 1 displayed construct underrepresentation and some mistargeting. Moreover, Instrument 2 demonstrated reasonably good rating scale function but exhibited construct underrepresentation and ceiling effects. Finally, Instrument 3 did not meet the necessary requirements for proper rating scale function and displayed ceiling effects and mistargeting. Conclusions: This pilot evaluation suggested that none of the instruments were useful in this population, which reinforces the need for researchers to use item response theories to examine the psychometric properties of instruments used in reporting various patient outcome measures. Pharmacists and other health care professionals should be alerted to potential problems with the validity of inferences made from underperforming instruments, so as to prevent inaccurate conclusions.

Original languageEnglish (US)
Pages (from-to)67-81
Number of pages15
JournalResearch in Social and Administrative Pharmacy
Volume4
Issue number1
DOIs
StatePublished - Mar 2008

Fingerprint

Medical problems
Pharmacists
Diabetes Mellitus
Outcome Assessment (Health Care)
Ceilings
Diabetes Complications
Disease Management
Psychometrics
Hemoglobins
Research Personnel
Delivery of Health Care
Bioelectric potentials
Health care
Population

Keywords

  • Diabetes
  • Pharmacy
  • Rasch analysis

ASJC Scopus subject areas

  • Pharmacy
  • Pharmaceutical Science

Cite this

An evaluation of the validity of inferences made from 3 diabetes assessment instruments : A Rasch analysis. / Handley, Lisa Izzo; Warholak, Terri L; Jackson, Terrence R.

In: Research in Social and Administrative Pharmacy, Vol. 4, No. 1, 03.2008, p. 67-81.

Research output: Contribution to journalArticle

@article{d51a929a052a46aea66c31172192b1d9,
title = "An evaluation of the validity of inferences made from 3 diabetes assessment instruments: A Rasch analysis",
abstract = "Background: Many patients who have diabetes lack adequate knowledge, positive attitudes, and sufficient resources to achieve optimal outcomes in the management of their disease. A key to assessing the impact of pharmacists' interactions with diabetes patients is the resultant impact on patient knowledge, attitudes, and satisfaction. As such, it is important that quality instruments be used to obtain accurate and dependable measures of these outcomes. Objective: To evaluate the validity of inferences made from 3 separate diabetes instruments used in the assessment of patient knowledge, attitudes, and satisfaction. Methods: This pilot study enrolled 30 patients with diabetes mellitus to evaluate the following 3 instruments: (1) Check Your Hemoglobin A1CIQ, (2) the revised Diabetes Questionnaire, and (3) a satisfaction questionnaire. The instruments were used to assess patient knowledge of diabetes and diabetes-related complications, attitudes toward having diabetes mellitus, and satisfaction with the services provided in a pharmacist-run diabetes clinic, respectively. Rasch analysis was used to determine if the instruments were able to measure the concepts they are intended to measure when used in this sample. Results: After evaluating the 3 instruments, it was determined that Instrument 1 displayed construct underrepresentation and some mistargeting. Moreover, Instrument 2 demonstrated reasonably good rating scale function but exhibited construct underrepresentation and ceiling effects. Finally, Instrument 3 did not meet the necessary requirements for proper rating scale function and displayed ceiling effects and mistargeting. Conclusions: This pilot evaluation suggested that none of the instruments were useful in this population, which reinforces the need for researchers to use item response theories to examine the psychometric properties of instruments used in reporting various patient outcome measures. Pharmacists and other health care professionals should be alerted to potential problems with the validity of inferences made from underperforming instruments, so as to prevent inaccurate conclusions.",
keywords = "Diabetes, Pharmacy, Rasch analysis",
author = "Handley, {Lisa Izzo} and Warholak, {Terri L} and Jackson, {Terrence R.}",
year = "2008",
month = "3",
doi = "10.1016/j.sapharm.2007.04.002",
language = "English (US)",
volume = "4",
pages = "67--81",
journal = "Research in Social and Administrative Pharmacy",
issn = "1551-7411",
publisher = "Elsevier Inc.",
number = "1",

}

TY - JOUR

T1 - An evaluation of the validity of inferences made from 3 diabetes assessment instruments

T2 - A Rasch analysis

AU - Handley, Lisa Izzo

AU - Warholak, Terri L

AU - Jackson, Terrence R.

PY - 2008/3

Y1 - 2008/3

N2 - Background: Many patients who have diabetes lack adequate knowledge, positive attitudes, and sufficient resources to achieve optimal outcomes in the management of their disease. A key to assessing the impact of pharmacists' interactions with diabetes patients is the resultant impact on patient knowledge, attitudes, and satisfaction. As such, it is important that quality instruments be used to obtain accurate and dependable measures of these outcomes. Objective: To evaluate the validity of inferences made from 3 separate diabetes instruments used in the assessment of patient knowledge, attitudes, and satisfaction. Methods: This pilot study enrolled 30 patients with diabetes mellitus to evaluate the following 3 instruments: (1) Check Your Hemoglobin A1CIQ, (2) the revised Diabetes Questionnaire, and (3) a satisfaction questionnaire. The instruments were used to assess patient knowledge of diabetes and diabetes-related complications, attitudes toward having diabetes mellitus, and satisfaction with the services provided in a pharmacist-run diabetes clinic, respectively. Rasch analysis was used to determine if the instruments were able to measure the concepts they are intended to measure when used in this sample. Results: After evaluating the 3 instruments, it was determined that Instrument 1 displayed construct underrepresentation and some mistargeting. Moreover, Instrument 2 demonstrated reasonably good rating scale function but exhibited construct underrepresentation and ceiling effects. Finally, Instrument 3 did not meet the necessary requirements for proper rating scale function and displayed ceiling effects and mistargeting. Conclusions: This pilot evaluation suggested that none of the instruments were useful in this population, which reinforces the need for researchers to use item response theories to examine the psychometric properties of instruments used in reporting various patient outcome measures. Pharmacists and other health care professionals should be alerted to potential problems with the validity of inferences made from underperforming instruments, so as to prevent inaccurate conclusions.

AB - Background: Many patients who have diabetes lack adequate knowledge, positive attitudes, and sufficient resources to achieve optimal outcomes in the management of their disease. A key to assessing the impact of pharmacists' interactions with diabetes patients is the resultant impact on patient knowledge, attitudes, and satisfaction. As such, it is important that quality instruments be used to obtain accurate and dependable measures of these outcomes. Objective: To evaluate the validity of inferences made from 3 separate diabetes instruments used in the assessment of patient knowledge, attitudes, and satisfaction. Methods: This pilot study enrolled 30 patients with diabetes mellitus to evaluate the following 3 instruments: (1) Check Your Hemoglobin A1CIQ, (2) the revised Diabetes Questionnaire, and (3) a satisfaction questionnaire. The instruments were used to assess patient knowledge of diabetes and diabetes-related complications, attitudes toward having diabetes mellitus, and satisfaction with the services provided in a pharmacist-run diabetes clinic, respectively. Rasch analysis was used to determine if the instruments were able to measure the concepts they are intended to measure when used in this sample. Results: After evaluating the 3 instruments, it was determined that Instrument 1 displayed construct underrepresentation and some mistargeting. Moreover, Instrument 2 demonstrated reasonably good rating scale function but exhibited construct underrepresentation and ceiling effects. Finally, Instrument 3 did not meet the necessary requirements for proper rating scale function and displayed ceiling effects and mistargeting. Conclusions: This pilot evaluation suggested that none of the instruments were useful in this population, which reinforces the need for researchers to use item response theories to examine the psychometric properties of instruments used in reporting various patient outcome measures. Pharmacists and other health care professionals should be alerted to potential problems with the validity of inferences made from underperforming instruments, so as to prevent inaccurate conclusions.

KW - Diabetes

KW - Pharmacy

KW - Rasch analysis

UR - http://www.scopus.com/inward/record.url?scp=40649110281&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=40649110281&partnerID=8YFLogxK

U2 - 10.1016/j.sapharm.2007.04.002

DO - 10.1016/j.sapharm.2007.04.002

M3 - Article

VL - 4

SP - 67

EP - 81

JO - Research in Social and Administrative Pharmacy

JF - Research in Social and Administrative Pharmacy

SN - 1551-7411

IS - 1

ER -