Competency-based evaluation tools for integrative medicine training in family medicine residency

A pilot study

Benjamin Kligler, Mary S Koithan, Victoria Maizes, Meg Hayes, Craig Schneider, Patricia Lebensohn, Susan Monica Keating Hadley

Research output: Contribution to journalArticle

22 Citations (Scopus)

Abstract

Background. As more integrative medicine educational content is integrated into conventional family medicine teaching, the need for effective evaluation strategies grows. Through the Integrative Family Medicine program, a six site pilot program of a four year residency training model combining integrative medicine and family medicine training, we have developed and tested a set of competency-based evaluation tools to assess residents' skills in integrative medicine history-taking and treatment planning. This paper presents the results from the implementation of direct observation and treatment plan evaluation tools, as well as the results of two Objective Structured Clinical Examinations (OSCEs) developed for the program. Methods. The direct observation (DO) and treatment plan (TP) evaluation tools developed for the IFM program were implemented by faculty at each of the six sites during the PGY-4 year (n = 11 on DO and n = 8 on TP). The OSCE I was implemented first in 2005 (n = 6), revised and then implemented with a second class of IFM participants in 2006 (n = 7). OSCE II was implemented in fall 2005 with only one class of IFM participants (n = 6). Data from the initial implementation of these tools are described using descriptive statistics. Results. Results from the implementation of these tools at the IFM sites suggest that we need more emphasis in our curriculum on incorporating spirituality into history-taking and treatment planning, and more training for IFM residents on effective assessment of readiness for change and strategies for delivering integrative medicine treatment recommendations. Focusing our OSCE assessment more narrowly on integrative medicine history-taking skills was much more effective in delineating strengths and weaknesses in our residents' performance than using the OSCE for both integrative and more basic communication competencies. Conclusion. As these tools are refined further they will be of value both in improving our teaching in the IFM program and as competency-based evaluation resources for the expanding number of family medicine residency programs incorporating integrative medicine into their curriculum. The next stages of work on these instruments will involve establishing inter-rater reliability and defining more clearly the specific behaviors which we believe establish competency in the integrative medicine skills defined for the program.

Original languageEnglish (US)
Article number7
JournalBMC Medical Education
Volume7
DOIs
StatePublished - 2007

Fingerprint

Integrative Medicine
Internship and Residency
Medicine
medicine
evaluation
History
examination
Observation
Curriculum
Teaching
Therapeutics
resident
Spirituality
history
educational content
curriculum
planning
Communication
descriptive statistics
spirituality

ASJC Scopus subject areas

  • Medicine(all)

Cite this

Competency-based evaluation tools for integrative medicine training in family medicine residency : A pilot study. / Kligler, Benjamin; Koithan, Mary S; Maizes, Victoria; Hayes, Meg; Schneider, Craig; Lebensohn, Patricia; Hadley, Susan Monica Keating.

In: BMC Medical Education, Vol. 7, 7, 2007.

Research output: Contribution to journalArticle

@article{5e17dd39089a43818a60be73543cb0bd,
title = "Competency-based evaluation tools for integrative medicine training in family medicine residency: A pilot study",
abstract = "Background. As more integrative medicine educational content is integrated into conventional family medicine teaching, the need for effective evaluation strategies grows. Through the Integrative Family Medicine program, a six site pilot program of a four year residency training model combining integrative medicine and family medicine training, we have developed and tested a set of competency-based evaluation tools to assess residents' skills in integrative medicine history-taking and treatment planning. This paper presents the results from the implementation of direct observation and treatment plan evaluation tools, as well as the results of two Objective Structured Clinical Examinations (OSCEs) developed for the program. Methods. The direct observation (DO) and treatment plan (TP) evaluation tools developed for the IFM program were implemented by faculty at each of the six sites during the PGY-4 year (n = 11 on DO and n = 8 on TP). The OSCE I was implemented first in 2005 (n = 6), revised and then implemented with a second class of IFM participants in 2006 (n = 7). OSCE II was implemented in fall 2005 with only one class of IFM participants (n = 6). Data from the initial implementation of these tools are described using descriptive statistics. Results. Results from the implementation of these tools at the IFM sites suggest that we need more emphasis in our curriculum on incorporating spirituality into history-taking and treatment planning, and more training for IFM residents on effective assessment of readiness for change and strategies for delivering integrative medicine treatment recommendations. Focusing our OSCE assessment more narrowly on integrative medicine history-taking skills was much more effective in delineating strengths and weaknesses in our residents' performance than using the OSCE for both integrative and more basic communication competencies. Conclusion. As these tools are refined further they will be of value both in improving our teaching in the IFM program and as competency-based evaluation resources for the expanding number of family medicine residency programs incorporating integrative medicine into their curriculum. The next stages of work on these instruments will involve establishing inter-rater reliability and defining more clearly the specific behaviors which we believe establish competency in the integrative medicine skills defined for the program.",
author = "Benjamin Kligler and Koithan, {Mary S} and Victoria Maizes and Meg Hayes and Craig Schneider and Patricia Lebensohn and Hadley, {Susan Monica Keating}",
year = "2007",
doi = "10.1186/1472-6920-7-7",
language = "English (US)",
volume = "7",
journal = "BMC Medical Education",
issn = "1472-6920",
publisher = "BioMed Central",

}

TY - JOUR

T1 - Competency-based evaluation tools for integrative medicine training in family medicine residency

T2 - A pilot study

AU - Kligler, Benjamin

AU - Koithan, Mary S

AU - Maizes, Victoria

AU - Hayes, Meg

AU - Schneider, Craig

AU - Lebensohn, Patricia

AU - Hadley, Susan Monica Keating

PY - 2007

Y1 - 2007

N2 - Background. As more integrative medicine educational content is integrated into conventional family medicine teaching, the need for effective evaluation strategies grows. Through the Integrative Family Medicine program, a six site pilot program of a four year residency training model combining integrative medicine and family medicine training, we have developed and tested a set of competency-based evaluation tools to assess residents' skills in integrative medicine history-taking and treatment planning. This paper presents the results from the implementation of direct observation and treatment plan evaluation tools, as well as the results of two Objective Structured Clinical Examinations (OSCEs) developed for the program. Methods. The direct observation (DO) and treatment plan (TP) evaluation tools developed for the IFM program were implemented by faculty at each of the six sites during the PGY-4 year (n = 11 on DO and n = 8 on TP). The OSCE I was implemented first in 2005 (n = 6), revised and then implemented with a second class of IFM participants in 2006 (n = 7). OSCE II was implemented in fall 2005 with only one class of IFM participants (n = 6). Data from the initial implementation of these tools are described using descriptive statistics. Results. Results from the implementation of these tools at the IFM sites suggest that we need more emphasis in our curriculum on incorporating spirituality into history-taking and treatment planning, and more training for IFM residents on effective assessment of readiness for change and strategies for delivering integrative medicine treatment recommendations. Focusing our OSCE assessment more narrowly on integrative medicine history-taking skills was much more effective in delineating strengths and weaknesses in our residents' performance than using the OSCE for both integrative and more basic communication competencies. Conclusion. As these tools are refined further they will be of value both in improving our teaching in the IFM program and as competency-based evaluation resources for the expanding number of family medicine residency programs incorporating integrative medicine into their curriculum. The next stages of work on these instruments will involve establishing inter-rater reliability and defining more clearly the specific behaviors which we believe establish competency in the integrative medicine skills defined for the program.

AB - Background. As more integrative medicine educational content is integrated into conventional family medicine teaching, the need for effective evaluation strategies grows. Through the Integrative Family Medicine program, a six site pilot program of a four year residency training model combining integrative medicine and family medicine training, we have developed and tested a set of competency-based evaluation tools to assess residents' skills in integrative medicine history-taking and treatment planning. This paper presents the results from the implementation of direct observation and treatment plan evaluation tools, as well as the results of two Objective Structured Clinical Examinations (OSCEs) developed for the program. Methods. The direct observation (DO) and treatment plan (TP) evaluation tools developed for the IFM program were implemented by faculty at each of the six sites during the PGY-4 year (n = 11 on DO and n = 8 on TP). The OSCE I was implemented first in 2005 (n = 6), revised and then implemented with a second class of IFM participants in 2006 (n = 7). OSCE II was implemented in fall 2005 with only one class of IFM participants (n = 6). Data from the initial implementation of these tools are described using descriptive statistics. Results. Results from the implementation of these tools at the IFM sites suggest that we need more emphasis in our curriculum on incorporating spirituality into history-taking and treatment planning, and more training for IFM residents on effective assessment of readiness for change and strategies for delivering integrative medicine treatment recommendations. Focusing our OSCE assessment more narrowly on integrative medicine history-taking skills was much more effective in delineating strengths and weaknesses in our residents' performance than using the OSCE for both integrative and more basic communication competencies. Conclusion. As these tools are refined further they will be of value both in improving our teaching in the IFM program and as competency-based evaluation resources for the expanding number of family medicine residency programs incorporating integrative medicine into their curriculum. The next stages of work on these instruments will involve establishing inter-rater reliability and defining more clearly the specific behaviors which we believe establish competency in the integrative medicine skills defined for the program.

UR - http://www.scopus.com/inward/record.url?scp=34247538895&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=34247538895&partnerID=8YFLogxK

U2 - 10.1186/1472-6920-7-7

DO - 10.1186/1472-6920-7-7

M3 - Article

VL - 7

JO - BMC Medical Education

JF - BMC Medical Education

SN - 1472-6920

M1 - 7

ER -