Predictive Inference and Scientific Reproducibility

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Most statistical analyses use hypothesis tests or estimation about parameters to form inferential conclusions. I think this is noble, but misguided. The point of view expressed here is that observables are fundamental, and that the goal of statistical modeling should be to predict future observations, given the current data and other relevant information. Further, the prediction of future observables provides multiple advantages to practicing scientists, and to science in general. These include an interpretable numerical summary of a quantity of direct interest to current and future researchers, a calibrated prediction of what’s likely to happen in future experiments, a prediction that can be either “corroborated” or “refuted” through experimentation, and avoidance of inference about parameters; quantities that exists only as convenient indices of hypothetical distributions. Finally, the predictive probability of a future observable can be used as a standard for communicating the reliability of the current work, regardless of whether confirmatory experiments are conducted. Adoption of this paradigm would improve our rigor for scientific accuracy and reproducibility by shifting our focus from “finding differences” among hypothetical parameters to predicting observable events based on our current scientific understanding.

Original languageEnglish (US)
Pages (from-to)291-295
Number of pages5
JournalAmerican Statistician
Volume73
Issue numbersup1
DOIs
StatePublished - Mar 29 2019

Fingerprint

Predictive Inference
Reproducibility
Prediction
Statistical Modeling
Hypothesis Test
Experimentation
Experiment
Likely
Paradigm
Predict
Inference

Keywords

  • Observables
  • Prediction
  • Predictive distribution
  • Scientific inference

ASJC Scopus subject areas

  • Statistics and Probability
  • Mathematics(all)
  • Statistics, Probability and Uncertainty

Cite this

Predictive Inference and Scientific Reproducibility. / Billheimer, David D.

In: American Statistician, Vol. 73, No. sup1, 29.03.2019, p. 291-295.

Research output: Contribution to journalArticle

@article{a2a0ec1acdc54618aefc697faa482725,
title = "Predictive Inference and Scientific Reproducibility",
abstract = "Most statistical analyses use hypothesis tests or estimation about parameters to form inferential conclusions. I think this is noble, but misguided. The point of view expressed here is that observables are fundamental, and that the goal of statistical modeling should be to predict future observations, given the current data and other relevant information. Further, the prediction of future observables provides multiple advantages to practicing scientists, and to science in general. These include an interpretable numerical summary of a quantity of direct interest to current and future researchers, a calibrated prediction of what’s likely to happen in future experiments, a prediction that can be either “corroborated” or “refuted” through experimentation, and avoidance of inference about parameters; quantities that exists only as convenient indices of hypothetical distributions. Finally, the predictive probability of a future observable can be used as a standard for communicating the reliability of the current work, regardless of whether confirmatory experiments are conducted. Adoption of this paradigm would improve our rigor for scientific accuracy and reproducibility by shifting our focus from “finding differences” among hypothetical parameters to predicting observable events based on our current scientific understanding.",
keywords = "Observables, Prediction, Predictive distribution, Scientific inference",
author = "Billheimer, {David D}",
year = "2019",
month = "3",
day = "29",
doi = "10.1080/00031305.2018.1518270",
language = "English (US)",
volume = "73",
pages = "291--295",
journal = "American Statistician",
issn = "0003-1305",
publisher = "American Statistical Association",
number = "sup1",

}

TY - JOUR

T1 - Predictive Inference and Scientific Reproducibility

AU - Billheimer, David D

PY - 2019/3/29

Y1 - 2019/3/29

N2 - Most statistical analyses use hypothesis tests or estimation about parameters to form inferential conclusions. I think this is noble, but misguided. The point of view expressed here is that observables are fundamental, and that the goal of statistical modeling should be to predict future observations, given the current data and other relevant information. Further, the prediction of future observables provides multiple advantages to practicing scientists, and to science in general. These include an interpretable numerical summary of a quantity of direct interest to current and future researchers, a calibrated prediction of what’s likely to happen in future experiments, a prediction that can be either “corroborated” or “refuted” through experimentation, and avoidance of inference about parameters; quantities that exists only as convenient indices of hypothetical distributions. Finally, the predictive probability of a future observable can be used as a standard for communicating the reliability of the current work, regardless of whether confirmatory experiments are conducted. Adoption of this paradigm would improve our rigor for scientific accuracy and reproducibility by shifting our focus from “finding differences” among hypothetical parameters to predicting observable events based on our current scientific understanding.

AB - Most statistical analyses use hypothesis tests or estimation about parameters to form inferential conclusions. I think this is noble, but misguided. The point of view expressed here is that observables are fundamental, and that the goal of statistical modeling should be to predict future observations, given the current data and other relevant information. Further, the prediction of future observables provides multiple advantages to practicing scientists, and to science in general. These include an interpretable numerical summary of a quantity of direct interest to current and future researchers, a calibrated prediction of what’s likely to happen in future experiments, a prediction that can be either “corroborated” or “refuted” through experimentation, and avoidance of inference about parameters; quantities that exists only as convenient indices of hypothetical distributions. Finally, the predictive probability of a future observable can be used as a standard for communicating the reliability of the current work, regardless of whether confirmatory experiments are conducted. Adoption of this paradigm would improve our rigor for scientific accuracy and reproducibility by shifting our focus from “finding differences” among hypothetical parameters to predicting observable events based on our current scientific understanding.

KW - Observables

KW - Prediction

KW - Predictive distribution

KW - Scientific inference

UR - http://www.scopus.com/inward/record.url?scp=85063204638&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85063204638&partnerID=8YFLogxK

U2 - 10.1080/00031305.2018.1518270

DO - 10.1080/00031305.2018.1518270

M3 - Article

AN - SCOPUS:85063204638

VL - 73

SP - 291

EP - 295

JO - American Statistician

JF - American Statistician

SN - 0003-1305

IS - sup1

ER -