The composition of normative groups and diagnostic decision making

Shooting ourselves in the foot

Elizabeth D. Peña, Tammie J. Spaulding, Elena M Plante

Research output: Contribution to journalArticle

56 Citations (Scopus)

Abstract

Purpose: The normative group of a norm-referenced test is intended to provide a basis for interpreting test scores. However, the composition of the normative group may facilitate or impede different types of diagnostic interpretations. This article considers who should be included in a normative sample and how this decision must be made relative to the purpose for which a test is intended. Method: The way in which the composition of the normative sample affects classification accuracy is demonstrated through a test review followed by a simulation study. The test review examined the descriptions of the normative group in a sample of 32 child language tests. The mean performance reported in the test manual for the sample of language impaired children was compared with the sample's norms, which either included or excluded children with language impairment. For the simulation, 2 contrasting normative procedures were modeled. The first procedure included a mixed group of representative cases (language impaired and normal cases). The second procedure excluded the language impaired cases from the norm. Results: Both the data obtained from test manuals and the data simulation based on population characteristics supported our claim that use of mixed normative groups decreases the ability to accurately identify language impairment. Tests that used mixed norms had smaller differences between the normative and language impaired groups in comparison with tests that excluded children with impairment within the normative sample. The simulation demonstrated mixed norms that lowered the group mean and increased the standard deviation, resulting in decreased classification accuracy. Conclusions: When the purpose of testing is to identify children with impaired language skills, including children with language impairment in the normative sample can reduce identification accuracy.

Original languageEnglish (US)
Pages (from-to)247-254
Number of pages8
JournalAmerican Journal of Speech-Language Pathology
Volume15
Issue number3
DOIs
StatePublished - Aug 2006

Fingerprint

Child Language
Foot
Decision Making
diagnostic
Language
decision making
language
Group
Language Tests
simulation
Aptitude
Population Characteristics
language group
interpretation
ability

Keywords

  • Assessment
  • Classification
  • Evidence-based practice
  • Language impairment

ASJC Scopus subject areas

  • Rehabilitation
  • Health Professions(all)
  • Linguistics and Language

Cite this

The composition of normative groups and diagnostic decision making : Shooting ourselves in the foot. / Peña, Elizabeth D.; Spaulding, Tammie J.; Plante, Elena M.

In: American Journal of Speech-Language Pathology, Vol. 15, No. 3, 08.2006, p. 247-254.

Research output: Contribution to journalArticle

@article{0298c4edac724ef0bcbfc4f97691785c,
title = "The composition of normative groups and diagnostic decision making: Shooting ourselves in the foot",
abstract = "Purpose: The normative group of a norm-referenced test is intended to provide a basis for interpreting test scores. However, the composition of the normative group may facilitate or impede different types of diagnostic interpretations. This article considers who should be included in a normative sample and how this decision must be made relative to the purpose for which a test is intended. Method: The way in which the composition of the normative sample affects classification accuracy is demonstrated through a test review followed by a simulation study. The test review examined the descriptions of the normative group in a sample of 32 child language tests. The mean performance reported in the test manual for the sample of language impaired children was compared with the sample's norms, which either included or excluded children with language impairment. For the simulation, 2 contrasting normative procedures were modeled. The first procedure included a mixed group of representative cases (language impaired and normal cases). The second procedure excluded the language impaired cases from the norm. Results: Both the data obtained from test manuals and the data simulation based on population characteristics supported our claim that use of mixed normative groups decreases the ability to accurately identify language impairment. Tests that used mixed norms had smaller differences between the normative and language impaired groups in comparison with tests that excluded children with impairment within the normative sample. The simulation demonstrated mixed norms that lowered the group mean and increased the standard deviation, resulting in decreased classification accuracy. Conclusions: When the purpose of testing is to identify children with impaired language skills, including children with language impairment in the normative sample can reduce identification accuracy.",
keywords = "Assessment, Classification, Evidence-based practice, Language impairment",
author = "Pe{\~n}a, {Elizabeth D.} and Spaulding, {Tammie J.} and Plante, {Elena M}",
year = "2006",
month = "8",
doi = "10.1044/1058-0360(2006/023)",
language = "English (US)",
volume = "15",
pages = "247--254",
journal = "American Journal of Speech-Language Pathology",
issn = "1058-0360",
publisher = "American Speech-Language-Hearing Association (ASHA)",
number = "3",

}

TY - JOUR

T1 - The composition of normative groups and diagnostic decision making

T2 - Shooting ourselves in the foot

AU - Peña, Elizabeth D.

AU - Spaulding, Tammie J.

AU - Plante, Elena M

PY - 2006/8

Y1 - 2006/8

N2 - Purpose: The normative group of a norm-referenced test is intended to provide a basis for interpreting test scores. However, the composition of the normative group may facilitate or impede different types of diagnostic interpretations. This article considers who should be included in a normative sample and how this decision must be made relative to the purpose for which a test is intended. Method: The way in which the composition of the normative sample affects classification accuracy is demonstrated through a test review followed by a simulation study. The test review examined the descriptions of the normative group in a sample of 32 child language tests. The mean performance reported in the test manual for the sample of language impaired children was compared with the sample's norms, which either included or excluded children with language impairment. For the simulation, 2 contrasting normative procedures were modeled. The first procedure included a mixed group of representative cases (language impaired and normal cases). The second procedure excluded the language impaired cases from the norm. Results: Both the data obtained from test manuals and the data simulation based on population characteristics supported our claim that use of mixed normative groups decreases the ability to accurately identify language impairment. Tests that used mixed norms had smaller differences between the normative and language impaired groups in comparison with tests that excluded children with impairment within the normative sample. The simulation demonstrated mixed norms that lowered the group mean and increased the standard deviation, resulting in decreased classification accuracy. Conclusions: When the purpose of testing is to identify children with impaired language skills, including children with language impairment in the normative sample can reduce identification accuracy.

AB - Purpose: The normative group of a norm-referenced test is intended to provide a basis for interpreting test scores. However, the composition of the normative group may facilitate or impede different types of diagnostic interpretations. This article considers who should be included in a normative sample and how this decision must be made relative to the purpose for which a test is intended. Method: The way in which the composition of the normative sample affects classification accuracy is demonstrated through a test review followed by a simulation study. The test review examined the descriptions of the normative group in a sample of 32 child language tests. The mean performance reported in the test manual for the sample of language impaired children was compared with the sample's norms, which either included or excluded children with language impairment. For the simulation, 2 contrasting normative procedures were modeled. The first procedure included a mixed group of representative cases (language impaired and normal cases). The second procedure excluded the language impaired cases from the norm. Results: Both the data obtained from test manuals and the data simulation based on population characteristics supported our claim that use of mixed normative groups decreases the ability to accurately identify language impairment. Tests that used mixed norms had smaller differences between the normative and language impaired groups in comparison with tests that excluded children with impairment within the normative sample. The simulation demonstrated mixed norms that lowered the group mean and increased the standard deviation, resulting in decreased classification accuracy. Conclusions: When the purpose of testing is to identify children with impaired language skills, including children with language impairment in the normative sample can reduce identification accuracy.

KW - Assessment

KW - Classification

KW - Evidence-based practice

KW - Language impairment

UR - http://www.scopus.com/inward/record.url?scp=33748870007&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33748870007&partnerID=8YFLogxK

U2 - 10.1044/1058-0360(2006/023)

DO - 10.1044/1058-0360(2006/023)

M3 - Article

VL - 15

SP - 247

EP - 254

JO - American Journal of Speech-Language Pathology

JF - American Journal of Speech-Language Pathology

SN - 1058-0360

IS - 3

ER -