Rating performance assessments of students with disabilities: A study of reliability and bias

Ann M. Mastergeorge, José Felipe Martínez

Research output: Contribution to journalArticle

2 Scopus citations

Abstract

Inclusion of students with disabilities in district-wide and state assessments is mandated by federal regulations, and teachers sometimes play an important role in rating these students' work. In this study, trained teachers rated student proficiency in performance assessments in language arts and mathematics in third, fifth, and ninth grades. The scores assigned by teacher raters to students with and without disabilities in an initial blind rating were compared with the ratings assigned in a second occasion when raters were aware of each student's disability status. A series of generalizability studies was used to determine if there are differences in the patterns of variability across groups and whether rater bias may play a role in these differences. Although knowledge of a student's disability status did not increase or decrease the scores assigned by raters on average, the findings point to differences in the sources of variability across groups and specifically to greater inconsistency when rating papers from students with disabilities. The findings suggest that individual teachers may behave differently when scoring students with disabilities. A survey was also used to investigate rater perceptions of one's own and other teacher's bias when grading papers of students with disabilities. Implications for decision making in rating assessments are discussed.

Original languageEnglish (US)
Pages (from-to)536-550
Number of pages15
JournalJournal of Psychoeducational Assessment
Volume28
Issue number6
DOIs
StatePublished - Dec 1 2010

Keywords

  • bias
  • disabilities
  • generalizability theory
  • performance assessment
  • teacher judgment

ASJC Scopus subject areas

  • Education
  • Clinical Psychology
  • Psychology(all)

Fingerprint Dive into the research topics of 'Rating performance assessments of students with disabilities: A study of reliability and bias'. Together they form a unique fingerprint.

  • Cite this