A comparative study of high-performance computing on the cloud

Aniruddha Marathe, Rachel Harris, David K Lowenthal, Bronis R. De Supinski, Barry Rountree, Martin Schulz, Xin Yuan

Research output: Chapter in Book/Report/Conference proceedingConference contribution

52 Citations (Scopus)

Abstract

The popularity of Amazon's EC2 cloud platform has increased in recent years. However, many high-performance computing (HPC) users consider dedicated high-performance clusters, typically found in large compute centers such as those in national laboratories, to be far superior to EC2 because of significant communication overhead of the latter. Our view is that this is quite narrow and the proper metrics for comparing high-performance clusters to EC2 is turnaround time and cost. In this paper, we compare the top-of-the-line EC2 cluster to HPC clusters at Lawrence Livermore National Laboratory (LLNL) based on turnaround time and total cost of execution. When measuring turnaround time, we include expected queue wait time on HPC clusters. Our results show that although as expected, standard HPC clusters are superior in raw performance, EC2 clusters may produce better turnaround times. To estimate cost, we developed a pricing model - relative to EC2's node-hour prices - to set node-hour prices for (currently free) LLNL clusters. We observe that the cost-effectiveness of running an application on a cluster depends on raw performance and application scalability.

Original languageEnglish (US)
Title of host publicationHPDC 2013 - Proceedings of the 22nd ACM International Symposium on High-Performance Parallel and Distributed Computing
Pages239-250
Number of pages12
DOIs
StatePublished - 2013
Event22nd ACM International Symposium on High-Performance Parallel and Distributed Computing, HPDC 2013 - New York, NY, United States
Duration: Jun 17 2013Jun 21 2013

Other

Other22nd ACM International Symposium on High-Performance Parallel and Distributed Computing, HPDC 2013
CountryUnited States
CityNew York, NY
Period6/17/136/21/13

Fingerprint

Turnaround time
Cluster computing
Costs
Cost effectiveness
Scalability
Communication

Keywords

  • cloud
  • cost
  • high-performance computing
  • turnaround time

ASJC Scopus subject areas

  • Software

Cite this

Marathe, A., Harris, R., Lowenthal, D. K., De Supinski, B. R., Rountree, B., Schulz, M., & Yuan, X. (2013). A comparative study of high-performance computing on the cloud. In HPDC 2013 - Proceedings of the 22nd ACM International Symposium on High-Performance Parallel and Distributed Computing (pp. 239-250) https://doi.org/10.1145/2462902.2462919

A comparative study of high-performance computing on the cloud. / Marathe, Aniruddha; Harris, Rachel; Lowenthal, David K; De Supinski, Bronis R.; Rountree, Barry; Schulz, Martin; Yuan, Xin.

HPDC 2013 - Proceedings of the 22nd ACM International Symposium on High-Performance Parallel and Distributed Computing. 2013. p. 239-250.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Marathe, A, Harris, R, Lowenthal, DK, De Supinski, BR, Rountree, B, Schulz, M & Yuan, X 2013, A comparative study of high-performance computing on the cloud. in HPDC 2013 - Proceedings of the 22nd ACM International Symposium on High-Performance Parallel and Distributed Computing. pp. 239-250, 22nd ACM International Symposium on High-Performance Parallel and Distributed Computing, HPDC 2013, New York, NY, United States, 6/17/13. https://doi.org/10.1145/2462902.2462919
Marathe A, Harris R, Lowenthal DK, De Supinski BR, Rountree B, Schulz M et al. A comparative study of high-performance computing on the cloud. In HPDC 2013 - Proceedings of the 22nd ACM International Symposium on High-Performance Parallel and Distributed Computing. 2013. p. 239-250 https://doi.org/10.1145/2462902.2462919
Marathe, Aniruddha ; Harris, Rachel ; Lowenthal, David K ; De Supinski, Bronis R. ; Rountree, Barry ; Schulz, Martin ; Yuan, Xin. / A comparative study of high-performance computing on the cloud. HPDC 2013 - Proceedings of the 22nd ACM International Symposium on High-Performance Parallel and Distributed Computing. 2013. pp. 239-250
@inproceedings{c08abfe08ec74a5caef0df40a78a4556,
title = "A comparative study of high-performance computing on the cloud",
abstract = "The popularity of Amazon's EC2 cloud platform has increased in recent years. However, many high-performance computing (HPC) users consider dedicated high-performance clusters, typically found in large compute centers such as those in national laboratories, to be far superior to EC2 because of significant communication overhead of the latter. Our view is that this is quite narrow and the proper metrics for comparing high-performance clusters to EC2 is turnaround time and cost. In this paper, we compare the top-of-the-line EC2 cluster to HPC clusters at Lawrence Livermore National Laboratory (LLNL) based on turnaround time and total cost of execution. When measuring turnaround time, we include expected queue wait time on HPC clusters. Our results show that although as expected, standard HPC clusters are superior in raw performance, EC2 clusters may produce better turnaround times. To estimate cost, we developed a pricing model - relative to EC2's node-hour prices - to set node-hour prices for (currently free) LLNL clusters. We observe that the cost-effectiveness of running an application on a cluster depends on raw performance and application scalability.",
keywords = "cloud, cost, high-performance computing, turnaround time",
author = "Aniruddha Marathe and Rachel Harris and Lowenthal, {David K} and {De Supinski}, {Bronis R.} and Barry Rountree and Martin Schulz and Xin Yuan",
year = "2013",
doi = "10.1145/2462902.2462919",
language = "English (US)",
pages = "239--250",
booktitle = "HPDC 2013 - Proceedings of the 22nd ACM International Symposium on High-Performance Parallel and Distributed Computing",

}

TY - GEN

T1 - A comparative study of high-performance computing on the cloud

AU - Marathe, Aniruddha

AU - Harris, Rachel

AU - Lowenthal, David K

AU - De Supinski, Bronis R.

AU - Rountree, Barry

AU - Schulz, Martin

AU - Yuan, Xin

PY - 2013

Y1 - 2013

N2 - The popularity of Amazon's EC2 cloud platform has increased in recent years. However, many high-performance computing (HPC) users consider dedicated high-performance clusters, typically found in large compute centers such as those in national laboratories, to be far superior to EC2 because of significant communication overhead of the latter. Our view is that this is quite narrow and the proper metrics for comparing high-performance clusters to EC2 is turnaround time and cost. In this paper, we compare the top-of-the-line EC2 cluster to HPC clusters at Lawrence Livermore National Laboratory (LLNL) based on turnaround time and total cost of execution. When measuring turnaround time, we include expected queue wait time on HPC clusters. Our results show that although as expected, standard HPC clusters are superior in raw performance, EC2 clusters may produce better turnaround times. To estimate cost, we developed a pricing model - relative to EC2's node-hour prices - to set node-hour prices for (currently free) LLNL clusters. We observe that the cost-effectiveness of running an application on a cluster depends on raw performance and application scalability.

AB - The popularity of Amazon's EC2 cloud platform has increased in recent years. However, many high-performance computing (HPC) users consider dedicated high-performance clusters, typically found in large compute centers such as those in national laboratories, to be far superior to EC2 because of significant communication overhead of the latter. Our view is that this is quite narrow and the proper metrics for comparing high-performance clusters to EC2 is turnaround time and cost. In this paper, we compare the top-of-the-line EC2 cluster to HPC clusters at Lawrence Livermore National Laboratory (LLNL) based on turnaround time and total cost of execution. When measuring turnaround time, we include expected queue wait time on HPC clusters. Our results show that although as expected, standard HPC clusters are superior in raw performance, EC2 clusters may produce better turnaround times. To estimate cost, we developed a pricing model - relative to EC2's node-hour prices - to set node-hour prices for (currently free) LLNL clusters. We observe that the cost-effectiveness of running an application on a cluster depends on raw performance and application scalability.

KW - cloud

KW - cost

KW - high-performance computing

KW - turnaround time

UR - http://www.scopus.com/inward/record.url?scp=84880064788&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84880064788&partnerID=8YFLogxK

U2 - 10.1145/2462902.2462919

DO - 10.1145/2462902.2462919

M3 - Conference contribution

SP - 239

EP - 250

BT - HPDC 2013 - Proceedings of the 22nd ACM International Symposium on High-Performance Parallel and Distributed Computing

ER -