Measurement characteristics of a concept classification exam using multiple case examples: A Rasch analysis

Nicholas B. Jennings, Marion K. Slack, Lea E. Mollon, Terri L. Warholak

Research output: Contribution to journalArticlepeer-review

Abstract

Objective: To determine if an exam using multiple cases to test research design concepts measured only one cognitive skill, concept classification, and to determine if item difficulty varied according to the research design used for the case. Methods: The exam consisted of 50 multiple choice items associated with five example abstracts: a randomized controlled trial, pretest-posttest, crossover, retrospective cohort, and descriptive designs. A Rasch analysis was conducted to determine dimensionality (i.e., measured a single skill). Items were stratified by design to explore the relationship between item difficulty and study design. Overall difficulty was assessed using an item-person map. Results: The exam was administered to 101 students; the mean was 88.4% (mean score = 44.2; SD = 3.5). The Rasch analysis indicated the exam primarily measured one cognitive skill, presumably concept classification. The stratified analysis indicated that overall no single research design was more difficult than other designs; however, the type of research design and item topic interacted so that an easy item for one design could be difficult when associated with a different study design. Conclusions: The exam appeared to function more like a mastery exam documenting that most students performed well rather than as an exam for ranking students by ability. That item topic interacted with study design to affect item difficulty, indicates that items on the same topic are needed to test basic design concepts across study designs.

Original languageEnglish (US)
Pages (from-to)31-38
Number of pages8
JournalCurrents in Pharmacy Teaching and Learning
Volume8
Issue number1
DOIs
StatePublished - 2016

Keywords

  • Concepts
  • Multiple choice exam
  • Rasch analysis
  • Research design

ASJC Scopus subject areas

  • Pharmacy
  • Pharmacology, Toxicology and Pharmaceutics(all)

Fingerprint Dive into the research topics of 'Measurement characteristics of a concept classification exam using multiple case examples: A Rasch analysis'. Together they form a unique fingerprint.

Cite this