The Case of Performance Variability on Dragonfly-based Systems

Abhinav Bhatele, Jayaraman J. Thiagarajan, Taylor Groves, Rushil Anirudh, Staci A. Smith, Brandon Cook, David K. Lowenthal

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Performance of a parallel code running on a large supercomputer can vary significantly from one run to another even when the executable and its input parameters are left unchanged. Such variability can occur due to perturbation of the computation and/or communication in the code. In this paper, we investigate the case of performance variability arising due to network effects on supercomputers that use a dragonfly topology-specifically, Cray XC systems equipped with the Aries interconnect. We perform post-mortem analysis of network hardware counters, profiling output, job queue logs, and placement information, all gathered from periodic representative application runs. We investigate the causes of performance variability using deviation prediction and recursive feature elimination. Additionally, using time-stepped performance data of individual applications, we train machine learning models that can forecast the execution time of future time steps.

Original languageEnglish (US)
Title of host publicationProceedings - 2020 IEEE 34th International Parallel and Distributed Processing Symposium, IPDPS 2020
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages896-905
Number of pages10
ISBN (Electronic)9781728168760
DOIs
StatePublished - May 2020
Event34th IEEE International Parallel and Distributed Processing Symposium, IPDPS 2020 - New Orleans, United States
Duration: May 18 2020May 22 2020

Publication series

NameProceedings - 2020 IEEE 34th International Parallel and Distributed Processing Symposium, IPDPS 2020

Conference

Conference34th IEEE International Parallel and Distributed Processing Symposium, IPDPS 2020
CountryUnited States
CityNew Orleans
Period5/18/205/22/20

Keywords

  • data analytics
  • dragonfly network
  • forecasting
  • machine learning
  • performance models
  • performance variability

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Hardware and Architecture
  • Safety, Risk, Reliability and Quality

Fingerprint Dive into the research topics of 'The Case of Performance Variability on Dragonfly-based Systems'. Together they form a unique fingerprint.

  • Cite this

    Bhatele, A., Thiagarajan, J. J., Groves, T., Anirudh, R., Smith, S. A., Cook, B., & Lowenthal, D. K. (2020). The Case of Performance Variability on Dragonfly-based Systems. In Proceedings - 2020 IEEE 34th International Parallel and Distributed Processing Symposium, IPDPS 2020 (pp. 896-905). [9139880] (Proceedings - 2020 IEEE 34th International Parallel and Distributed Processing Symposium, IPDPS 2020). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/IPDPS47924.2020.00096