Limits of data analysis in scientific inference

Reply to Sleep et al.

Research output: Contribution to journalArticle

4 Citations (Scopus)

Abstract

Statistical inference is an important element of science, but these inferences are constrained within the framework established by the objectives and design of a study. The choice of approach to data analysis, while important, has far less consequence on scientific inference than claimed by Sleep et al. (2007). Their principal assertion - that when model selection is used as the approach to data analysis, all studies provide a reliable foundation for distinguishing among mechanistic explanatory hypotheses - is incorrect and encourages faulty inferences. Sleep et al. (2007) overlook the critical distinction between inferences that result from studies designed a priori to discriminate among a set of candidate explanations versus inferences that result from exploring data post hoc from studies designed originally to meet pattern-based objectives. No approach to data analysis, including model selection, has the power to overcome fundamental limitations on inferences imposed by study design. The comments by Sleep et al. (2007) reinforce the need for scientists to understand clearly the inferential basis for their scientific claims, including the roles and limitations of data analysis.

Original languageEnglish (US)
Pages (from-to)2122-2124
Number of pages3
JournalJournal of Wildlife Management
Volume71
Issue number7
DOIs
StatePublished - Sep 2007

Fingerprint

sleep
data analysis
experimental design

Keywords

  • Candidate models
  • Data analysis
  • Hypothesis testing
  • Information theory
  • Model selection
  • Null hypotheses
  • Research hypotheses

ASJC Scopus subject areas

  • Animal Science and Zoology
  • Ecology

Cite this

Limits of data analysis in scientific inference : Reply to Sleep et al. / Steidl, Robert J.

In: Journal of Wildlife Management, Vol. 71, No. 7, 09.2007, p. 2122-2124.

Research output: Contribution to journalArticle

@article{8dcfa419886c4154a26922a888344a3f,
title = "Limits of data analysis in scientific inference: Reply to Sleep et al.",
abstract = "Statistical inference is an important element of science, but these inferences are constrained within the framework established by the objectives and design of a study. The choice of approach to data analysis, while important, has far less consequence on scientific inference than claimed by Sleep et al. (2007). Their principal assertion - that when model selection is used as the approach to data analysis, all studies provide a reliable foundation for distinguishing among mechanistic explanatory hypotheses - is incorrect and encourages faulty inferences. Sleep et al. (2007) overlook the critical distinction between inferences that result from studies designed a priori to discriminate among a set of candidate explanations versus inferences that result from exploring data post hoc from studies designed originally to meet pattern-based objectives. No approach to data analysis, including model selection, has the power to overcome fundamental limitations on inferences imposed by study design. The comments by Sleep et al. (2007) reinforce the need for scientists to understand clearly the inferential basis for their scientific claims, including the roles and limitations of data analysis.",
keywords = "Candidate models, Data analysis, Hypothesis testing, Information theory, Model selection, Null hypotheses, Research hypotheses",
author = "Steidl, {Robert J}",
year = "2007",
month = "9",
doi = "10.2193/2007-187",
language = "English (US)",
volume = "71",
pages = "2122--2124",
journal = "Journal of Wildlife Management",
issn = "0022-541X",
publisher = "Wiley-Blackwell",
number = "7",

}

TY - JOUR

T1 - Limits of data analysis in scientific inference

T2 - Reply to Sleep et al.

AU - Steidl, Robert J

PY - 2007/9

Y1 - 2007/9

N2 - Statistical inference is an important element of science, but these inferences are constrained within the framework established by the objectives and design of a study. The choice of approach to data analysis, while important, has far less consequence on scientific inference than claimed by Sleep et al. (2007). Their principal assertion - that when model selection is used as the approach to data analysis, all studies provide a reliable foundation for distinguishing among mechanistic explanatory hypotheses - is incorrect and encourages faulty inferences. Sleep et al. (2007) overlook the critical distinction between inferences that result from studies designed a priori to discriminate among a set of candidate explanations versus inferences that result from exploring data post hoc from studies designed originally to meet pattern-based objectives. No approach to data analysis, including model selection, has the power to overcome fundamental limitations on inferences imposed by study design. The comments by Sleep et al. (2007) reinforce the need for scientists to understand clearly the inferential basis for their scientific claims, including the roles and limitations of data analysis.

AB - Statistical inference is an important element of science, but these inferences are constrained within the framework established by the objectives and design of a study. The choice of approach to data analysis, while important, has far less consequence on scientific inference than claimed by Sleep et al. (2007). Their principal assertion - that when model selection is used as the approach to data analysis, all studies provide a reliable foundation for distinguishing among mechanistic explanatory hypotheses - is incorrect and encourages faulty inferences. Sleep et al. (2007) overlook the critical distinction between inferences that result from studies designed a priori to discriminate among a set of candidate explanations versus inferences that result from exploring data post hoc from studies designed originally to meet pattern-based objectives. No approach to data analysis, including model selection, has the power to overcome fundamental limitations on inferences imposed by study design. The comments by Sleep et al. (2007) reinforce the need for scientists to understand clearly the inferential basis for their scientific claims, including the roles and limitations of data analysis.

KW - Candidate models

KW - Data analysis

KW - Hypothesis testing

KW - Information theory

KW - Model selection

KW - Null hypotheses

KW - Research hypotheses

UR - http://www.scopus.com/inward/record.url?scp=48249157312&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=48249157312&partnerID=8YFLogxK

U2 - 10.2193/2007-187

DO - 10.2193/2007-187

M3 - Article

VL - 71

SP - 2122

EP - 2124

JO - Journal of Wildlife Management

JF - Journal of Wildlife Management

SN - 0022-541X

IS - 7

ER -