Summary results of the 2014-2015 DARPA Chikungunya challenge

Sara Y. Del Valle, Benjamin H. McMahon, Jason Asher, Richard Hatchett, Joceline C Lega, Heidi E Brown, Mark E. Leany, Yannis Pantazis, David J. Roberts, Sean Moore, A. Townsend Peterson, Luis E. Escobar, Huijie Qiao, Nicholas W. Hengartner, Harshini Mukundan

Research output: Contribution to journalArticle

3 Citations (Scopus)

Abstract

Background: Emerging pathogens such as Zika, chikungunya, Ebola, and dengue viruses are serious threats to national and global health security. Accurate forecasts of emerging epidemics and their severity are critical to minimizing subsequent mortality, morbidity, and economic loss. The recent introduction of chikungunya and Zika virus to the Americas underscores the need for better methods for disease surveillance and forecasting. Methods: To explore the suitability of current approaches to forecasting emerging diseases, the Defense Advanced Research Projects Agency (DARPA) launched the 2014-2015 DARPA Chikungunya Challenge to forecast the number of cases and spread of chikungunya disease in the Americas. Challenge participants (n=38 during final evaluation) provided predictions of chikungunya epidemics across the Americas for a six-month period, from September 1, 2014 to February 16, 2015, to be evaluated by comparison with incidence data reported to the Pan American Health Organization (PAHO). This manuscript presents an overview of the challenge and a summary of the approaches used by the winners. Results: Participant submissions were evaluated by a team of non-competing government subject matter experts based on numerical accuracy and methodology. Although this manuscript does not include in-depth analyses of the results, cursory analyses suggest that simpler models appear to outperform more complex approaches that included, for example, demographic information and transportation dynamics, due to the reporting biases, which can be implicitly captured in statistical models. Mosquito-dynamics, population specific information, and dengue-specific information correlated best with prediction accuracy. Conclusion: We conclude that with careful consideration and understanding of the relative advantages and disadvantages of particular methods, implementation of an effective prediction system is feasible. However, there is a need to improve the quality of the data in order to more accurately predict the course of epidemics.

Original languageEnglish (US)
Article number245
JournalBMC Infectious Diseases
Volume18
Issue number1
DOIs
StatePublished - May 30 2018

Fingerprint

Chikungunya virus
Pan American Health Organization
Research
Ebolavirus
Dengue Virus
Dengue
Population Dynamics
Statistical Models
Culicidae
Economics
Demography
Morbidity
Mortality
Incidence
Zika Virus

Keywords

  • Chikungunya
  • Forecasting
  • Mechanistic models
  • Morphological models

ASJC Scopus subject areas

  • Infectious Diseases

Cite this

Del Valle, S. Y., McMahon, B. H., Asher, J., Hatchett, R., Lega, J. C., Brown, H. E., ... Mukundan, H. (2018). Summary results of the 2014-2015 DARPA Chikungunya challenge. BMC Infectious Diseases, 18(1), [245]. https://doi.org/10.1186/s12879-018-3124-7

Summary results of the 2014-2015 DARPA Chikungunya challenge. / Del Valle, Sara Y.; McMahon, Benjamin H.; Asher, Jason; Hatchett, Richard; Lega, Joceline C; Brown, Heidi E; Leany, Mark E.; Pantazis, Yannis; Roberts, David J.; Moore, Sean; Peterson, A. Townsend; Escobar, Luis E.; Qiao, Huijie; Hengartner, Nicholas W.; Mukundan, Harshini.

In: BMC Infectious Diseases, Vol. 18, No. 1, 245, 30.05.2018.

Research output: Contribution to journalArticle

Del Valle, SY, McMahon, BH, Asher, J, Hatchett, R, Lega, JC, Brown, HE, Leany, ME, Pantazis, Y, Roberts, DJ, Moore, S, Peterson, AT, Escobar, LE, Qiao, H, Hengartner, NW & Mukundan, H 2018, 'Summary results of the 2014-2015 DARPA Chikungunya challenge', BMC Infectious Diseases, vol. 18, no. 1, 245. https://doi.org/10.1186/s12879-018-3124-7
Del Valle, Sara Y. ; McMahon, Benjamin H. ; Asher, Jason ; Hatchett, Richard ; Lega, Joceline C ; Brown, Heidi E ; Leany, Mark E. ; Pantazis, Yannis ; Roberts, David J. ; Moore, Sean ; Peterson, A. Townsend ; Escobar, Luis E. ; Qiao, Huijie ; Hengartner, Nicholas W. ; Mukundan, Harshini. / Summary results of the 2014-2015 DARPA Chikungunya challenge. In: BMC Infectious Diseases. 2018 ; Vol. 18, No. 1.
@article{db9bd568f5b54f08968813951dc56910,
title = "Summary results of the 2014-2015 DARPA Chikungunya challenge",
abstract = "Background: Emerging pathogens such as Zika, chikungunya, Ebola, and dengue viruses are serious threats to national and global health security. Accurate forecasts of emerging epidemics and their severity are critical to minimizing subsequent mortality, morbidity, and economic loss. The recent introduction of chikungunya and Zika virus to the Americas underscores the need for better methods for disease surveillance and forecasting. Methods: To explore the suitability of current approaches to forecasting emerging diseases, the Defense Advanced Research Projects Agency (DARPA) launched the 2014-2015 DARPA Chikungunya Challenge to forecast the number of cases and spread of chikungunya disease in the Americas. Challenge participants (n=38 during final evaluation) provided predictions of chikungunya epidemics across the Americas for a six-month period, from September 1, 2014 to February 16, 2015, to be evaluated by comparison with incidence data reported to the Pan American Health Organization (PAHO). This manuscript presents an overview of the challenge and a summary of the approaches used by the winners. Results: Participant submissions were evaluated by a team of non-competing government subject matter experts based on numerical accuracy and methodology. Although this manuscript does not include in-depth analyses of the results, cursory analyses suggest that simpler models appear to outperform more complex approaches that included, for example, demographic information and transportation dynamics, due to the reporting biases, which can be implicitly captured in statistical models. Mosquito-dynamics, population specific information, and dengue-specific information correlated best with prediction accuracy. Conclusion: We conclude that with careful consideration and understanding of the relative advantages and disadvantages of particular methods, implementation of an effective prediction system is feasible. However, there is a need to improve the quality of the data in order to more accurately predict the course of epidemics.",
keywords = "Chikungunya, Forecasting, Mechanistic models, Morphological models",
author = "{Del Valle}, {Sara Y.} and McMahon, {Benjamin H.} and Jason Asher and Richard Hatchett and Lega, {Joceline C} and Brown, {Heidi E} and Leany, {Mark E.} and Yannis Pantazis and Roberts, {David J.} and Sean Moore and Peterson, {A. Townsend} and Escobar, {Luis E.} and Huijie Qiao and Hengartner, {Nicholas W.} and Harshini Mukundan",
year = "2018",
month = "5",
day = "30",
doi = "10.1186/s12879-018-3124-7",
language = "English (US)",
volume = "18",
journal = "BMC Infectious Diseases",
issn = "1471-2334",
publisher = "BioMed Central",
number = "1",

}

TY - JOUR

T1 - Summary results of the 2014-2015 DARPA Chikungunya challenge

AU - Del Valle, Sara Y.

AU - McMahon, Benjamin H.

AU - Asher, Jason

AU - Hatchett, Richard

AU - Lega, Joceline C

AU - Brown, Heidi E

AU - Leany, Mark E.

AU - Pantazis, Yannis

AU - Roberts, David J.

AU - Moore, Sean

AU - Peterson, A. Townsend

AU - Escobar, Luis E.

AU - Qiao, Huijie

AU - Hengartner, Nicholas W.

AU - Mukundan, Harshini

PY - 2018/5/30

Y1 - 2018/5/30

N2 - Background: Emerging pathogens such as Zika, chikungunya, Ebola, and dengue viruses are serious threats to national and global health security. Accurate forecasts of emerging epidemics and their severity are critical to minimizing subsequent mortality, morbidity, and economic loss. The recent introduction of chikungunya and Zika virus to the Americas underscores the need for better methods for disease surveillance and forecasting. Methods: To explore the suitability of current approaches to forecasting emerging diseases, the Defense Advanced Research Projects Agency (DARPA) launched the 2014-2015 DARPA Chikungunya Challenge to forecast the number of cases and spread of chikungunya disease in the Americas. Challenge participants (n=38 during final evaluation) provided predictions of chikungunya epidemics across the Americas for a six-month period, from September 1, 2014 to February 16, 2015, to be evaluated by comparison with incidence data reported to the Pan American Health Organization (PAHO). This manuscript presents an overview of the challenge and a summary of the approaches used by the winners. Results: Participant submissions were evaluated by a team of non-competing government subject matter experts based on numerical accuracy and methodology. Although this manuscript does not include in-depth analyses of the results, cursory analyses suggest that simpler models appear to outperform more complex approaches that included, for example, demographic information and transportation dynamics, due to the reporting biases, which can be implicitly captured in statistical models. Mosquito-dynamics, population specific information, and dengue-specific information correlated best with prediction accuracy. Conclusion: We conclude that with careful consideration and understanding of the relative advantages and disadvantages of particular methods, implementation of an effective prediction system is feasible. However, there is a need to improve the quality of the data in order to more accurately predict the course of epidemics.

AB - Background: Emerging pathogens such as Zika, chikungunya, Ebola, and dengue viruses are serious threats to national and global health security. Accurate forecasts of emerging epidemics and their severity are critical to minimizing subsequent mortality, morbidity, and economic loss. The recent introduction of chikungunya and Zika virus to the Americas underscores the need for better methods for disease surveillance and forecasting. Methods: To explore the suitability of current approaches to forecasting emerging diseases, the Defense Advanced Research Projects Agency (DARPA) launched the 2014-2015 DARPA Chikungunya Challenge to forecast the number of cases and spread of chikungunya disease in the Americas. Challenge participants (n=38 during final evaluation) provided predictions of chikungunya epidemics across the Americas for a six-month period, from September 1, 2014 to February 16, 2015, to be evaluated by comparison with incidence data reported to the Pan American Health Organization (PAHO). This manuscript presents an overview of the challenge and a summary of the approaches used by the winners. Results: Participant submissions were evaluated by a team of non-competing government subject matter experts based on numerical accuracy and methodology. Although this manuscript does not include in-depth analyses of the results, cursory analyses suggest that simpler models appear to outperform more complex approaches that included, for example, demographic information and transportation dynamics, due to the reporting biases, which can be implicitly captured in statistical models. Mosquito-dynamics, population specific information, and dengue-specific information correlated best with prediction accuracy. Conclusion: We conclude that with careful consideration and understanding of the relative advantages and disadvantages of particular methods, implementation of an effective prediction system is feasible. However, there is a need to improve the quality of the data in order to more accurately predict the course of epidemics.

KW - Chikungunya

KW - Forecasting

KW - Mechanistic models

KW - Morphological models

UR - http://www.scopus.com/inward/record.url?scp=85047659227&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85047659227&partnerID=8YFLogxK

U2 - 10.1186/s12879-018-3124-7

DO - 10.1186/s12879-018-3124-7

M3 - Article

VL - 18

JO - BMC Infectious Diseases

JF - BMC Infectious Diseases

SN - 1471-2334

IS - 1

M1 - 245

ER -