Developing a patient-centered outcome measure for complementary and alternative medicine therapies II: Refining content validity through cognitive interviews

Jennifer J. Thompson, Kimberly L Kelly, Cheryl Ritenbaugh, Allison L. Hopkins, Colette M. Sims, Stephen J. Coons

Research output: Contribution to journalArticle

17 Citations (Scopus)

Abstract

Background: Available measures of patient-reported outcomes for complementary and alternative medicine (CAM) inadequately capture the range of patient-reported treatment effects. The Self-Assessment of Change questionnaire was developed to measure multi-dimensional shifts in well-being for CAM users. With content derived from patient narratives, items were subsequently focused through interviews on a new cohort of participants. Here we present the development of the final version in which the content and format is refined through cognitive interviews.Methods: We conducted cognitive interviews across five iterations of questionnaire refinement with a culturally diverse sample of 28 CAM users. In each iteration, participant critiques were used to revise the questionnaire, which was then re-tested in subsequent rounds of cognitive interviews. Following all five iterations, transcripts of cognitive interviews were systematically coded and analyzed to examine participants' understanding of the format and content of the final questionnaire. Based on this data, we established summary descriptions and selected exemplar quotations for each word pair on the final questionnaire.Results: The final version of the Self-Assessment of Change questionnaire (SAC) includes 16 word pairs, nine of which remained unchanged from the original draft. Participants consistently said that these stable word pairs represented opposite ends of the same domain of experience and the meanings of these terms were stable across the participant pool. Five pairs underwent revision and two word pairs were added. Four word pairs were eliminated for redundancy or because participants did not agree on the meaning of the terms. Cognitive interviews indicate that participants understood the format of the questionnaire and considered each word pair to represent opposite poles of a shared domain of experience.Conclusions: We have placed lay language and direct experience at the center of questionnaire revision and refinement. In so doing, we provide an innovative model for the development of truly patient-centered outcome measures. Although this instrument was designed and tested in a CAM-specific population, it may be useful in assessing multi-dimensional shifts in well-being across a broader patient population.

Original languageEnglish (US)
Article number136
JournalBMC Complementary and Alternative Medicine
Volume11
DOIs
StatePublished - Dec 29 2011

Fingerprint

Complementary Therapies
Outcome Assessment (Health Care)
Interviews
Surveys and Questionnaires
Population
Language

Keywords

  • Cognitive interviewing
  • Complementary and alternative medicine (CAM)
  • Non-specific outcomes
  • Patient-centered care
  • Patient-reported outcomes (PROs)
  • Questionnaire development
  • Retrospective pre-test
  • Well-being

ASJC Scopus subject areas

  • Complementary and alternative medicine

Cite this

Developing a patient-centered outcome measure for complementary and alternative medicine therapies II : Refining content validity through cognitive interviews. / Thompson, Jennifer J.; Kelly, Kimberly L; Ritenbaugh, Cheryl; Hopkins, Allison L.; Sims, Colette M.; Coons, Stephen J.

In: BMC Complementary and Alternative Medicine, Vol. 11, 136, 29.12.2011.

Research output: Contribution to journalArticle

@article{ae11ef334d5748c782cd49c146cf3c38,
title = "Developing a patient-centered outcome measure for complementary and alternative medicine therapies II: Refining content validity through cognitive interviews",
abstract = "Background: Available measures of patient-reported outcomes for complementary and alternative medicine (CAM) inadequately capture the range of patient-reported treatment effects. The Self-Assessment of Change questionnaire was developed to measure multi-dimensional shifts in well-being for CAM users. With content derived from patient narratives, items were subsequently focused through interviews on a new cohort of participants. Here we present the development of the final version in which the content and format is refined through cognitive interviews.Methods: We conducted cognitive interviews across five iterations of questionnaire refinement with a culturally diverse sample of 28 CAM users. In each iteration, participant critiques were used to revise the questionnaire, which was then re-tested in subsequent rounds of cognitive interviews. Following all five iterations, transcripts of cognitive interviews were systematically coded and analyzed to examine participants' understanding of the format and content of the final questionnaire. Based on this data, we established summary descriptions and selected exemplar quotations for each word pair on the final questionnaire.Results: The final version of the Self-Assessment of Change questionnaire (SAC) includes 16 word pairs, nine of which remained unchanged from the original draft. Participants consistently said that these stable word pairs represented opposite ends of the same domain of experience and the meanings of these terms were stable across the participant pool. Five pairs underwent revision and two word pairs were added. Four word pairs were eliminated for redundancy or because participants did not agree on the meaning of the terms. Cognitive interviews indicate that participants understood the format of the questionnaire and considered each word pair to represent opposite poles of a shared domain of experience.Conclusions: We have placed lay language and direct experience at the center of questionnaire revision and refinement. In so doing, we provide an innovative model for the development of truly patient-centered outcome measures. Although this instrument was designed and tested in a CAM-specific population, it may be useful in assessing multi-dimensional shifts in well-being across a broader patient population.",
keywords = "Cognitive interviewing, Complementary and alternative medicine (CAM), Non-specific outcomes, Patient-centered care, Patient-reported outcomes (PROs), Questionnaire development, Retrospective pre-test, Well-being",
author = "Thompson, {Jennifer J.} and Kelly, {Kimberly L} and Cheryl Ritenbaugh and Hopkins, {Allison L.} and Sims, {Colette M.} and Coons, {Stephen J.}",
year = "2011",
month = "12",
day = "29",
doi = "10.1186/1472-6882-11-136",
language = "English (US)",
volume = "11",
journal = "BMC Complementary and Alternative Medicine",
issn = "1472-6882",
publisher = "BioMed Central",

}

TY - JOUR

T1 - Developing a patient-centered outcome measure for complementary and alternative medicine therapies II

T2 - Refining content validity through cognitive interviews

AU - Thompson, Jennifer J.

AU - Kelly, Kimberly L

AU - Ritenbaugh, Cheryl

AU - Hopkins, Allison L.

AU - Sims, Colette M.

AU - Coons, Stephen J.

PY - 2011/12/29

Y1 - 2011/12/29

N2 - Background: Available measures of patient-reported outcomes for complementary and alternative medicine (CAM) inadequately capture the range of patient-reported treatment effects. The Self-Assessment of Change questionnaire was developed to measure multi-dimensional shifts in well-being for CAM users. With content derived from patient narratives, items were subsequently focused through interviews on a new cohort of participants. Here we present the development of the final version in which the content and format is refined through cognitive interviews.Methods: We conducted cognitive interviews across five iterations of questionnaire refinement with a culturally diverse sample of 28 CAM users. In each iteration, participant critiques were used to revise the questionnaire, which was then re-tested in subsequent rounds of cognitive interviews. Following all five iterations, transcripts of cognitive interviews were systematically coded and analyzed to examine participants' understanding of the format and content of the final questionnaire. Based on this data, we established summary descriptions and selected exemplar quotations for each word pair on the final questionnaire.Results: The final version of the Self-Assessment of Change questionnaire (SAC) includes 16 word pairs, nine of which remained unchanged from the original draft. Participants consistently said that these stable word pairs represented opposite ends of the same domain of experience and the meanings of these terms were stable across the participant pool. Five pairs underwent revision and two word pairs were added. Four word pairs were eliminated for redundancy or because participants did not agree on the meaning of the terms. Cognitive interviews indicate that participants understood the format of the questionnaire and considered each word pair to represent opposite poles of a shared domain of experience.Conclusions: We have placed lay language and direct experience at the center of questionnaire revision and refinement. In so doing, we provide an innovative model for the development of truly patient-centered outcome measures. Although this instrument was designed and tested in a CAM-specific population, it may be useful in assessing multi-dimensional shifts in well-being across a broader patient population.

AB - Background: Available measures of patient-reported outcomes for complementary and alternative medicine (CAM) inadequately capture the range of patient-reported treatment effects. The Self-Assessment of Change questionnaire was developed to measure multi-dimensional shifts in well-being for CAM users. With content derived from patient narratives, items were subsequently focused through interviews on a new cohort of participants. Here we present the development of the final version in which the content and format is refined through cognitive interviews.Methods: We conducted cognitive interviews across five iterations of questionnaire refinement with a culturally diverse sample of 28 CAM users. In each iteration, participant critiques were used to revise the questionnaire, which was then re-tested in subsequent rounds of cognitive interviews. Following all five iterations, transcripts of cognitive interviews were systematically coded and analyzed to examine participants' understanding of the format and content of the final questionnaire. Based on this data, we established summary descriptions and selected exemplar quotations for each word pair on the final questionnaire.Results: The final version of the Self-Assessment of Change questionnaire (SAC) includes 16 word pairs, nine of which remained unchanged from the original draft. Participants consistently said that these stable word pairs represented opposite ends of the same domain of experience and the meanings of these terms were stable across the participant pool. Five pairs underwent revision and two word pairs were added. Four word pairs were eliminated for redundancy or because participants did not agree on the meaning of the terms. Cognitive interviews indicate that participants understood the format of the questionnaire and considered each word pair to represent opposite poles of a shared domain of experience.Conclusions: We have placed lay language and direct experience at the center of questionnaire revision and refinement. In so doing, we provide an innovative model for the development of truly patient-centered outcome measures. Although this instrument was designed and tested in a CAM-specific population, it may be useful in assessing multi-dimensional shifts in well-being across a broader patient population.

KW - Cognitive interviewing

KW - Complementary and alternative medicine (CAM)

KW - Non-specific outcomes

KW - Patient-centered care

KW - Patient-reported outcomes (PROs)

KW - Questionnaire development

KW - Retrospective pre-test

KW - Well-being

UR - http://www.scopus.com/inward/record.url?scp=84855161767&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84855161767&partnerID=8YFLogxK

U2 - 10.1186/1472-6882-11-136

DO - 10.1186/1472-6882-11-136

M3 - Article

C2 - 22206409

AN - SCOPUS:84855161767

VL - 11

JO - BMC Complementary and Alternative Medicine

JF - BMC Complementary and Alternative Medicine

SN - 1472-6882

M1 - 136

ER -