The rating and matching item-objective alignment methods

Jerome V. D'Agostino, Megan E. Welsh, Adriana D. Cimetta, Lia D. Falco, Shannon Smith, Waverely Hester VanWinkle, Sonya J. Powers

Research output: Contribution to journalArticlepeer-review

7 Scopus citations

Abstract

Central to the standards-based assessment validation process is an examination of the alignment between state standards and test items. Several alignment analysis systems have emerged recently, but most rely on either traditional rating or matching techniques. Little, if any, analyses have been reported on the degree of consistency between the two methods and on the item and objective characteristics that influence judges' decisions. We randomly assigned judges to either rate item-objective links or match items to objectives while reviewing the 2004 Arizona high school mathematics standards and assessment. Across items we found moderate convergence between methods, and we detected apparent reasons for divergently scored items. We also found that judges relied on item and objective content and intellectual skill features to render decisions. Based on our evidence, we contend that a thorough alignment analysis would involve judges using both rating and matching, while focusing on both content and intellectual skill. The findings have important implications for states when examining the alignment between their standards and assessments.

Original languageEnglish (US)
Pages (from-to)1-21
Number of pages21
JournalApplied Measurement in Education
Volume21
Issue number1
DOIs
StatePublished - Jan 1 2008

ASJC Scopus subject areas

  • Education
  • Developmental and Educational Psychology

Fingerprint Dive into the research topics of 'The rating and matching item-objective alignment methods'. Together they form a unique fingerprint.

Cite this