MAKER as a service

Moving HPC applications to jetstream cloud

Nicholas Hazekamp, Upendra Kumar Devisetty, Nirav C Merchant, Douglas Thain

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

As cloud resources become more available as an execution platform, the need to transition applications between HPC and the cloud becomes a necessity. However, because of the complex setup and system specific demands of these applications, transition is difficult and may not scale as desired. Jetstream is a NSF funded cloud service that is aiming to provide these services for users in a dynamical allocated nature. In this work we look at three key areas to focus on when transitioning between resources: providing a portable reproducible environment, scaling between local and remote resources, and using feedback to the user for informing configuration and runtime decisions. Building on the MAKER bioinformatic application, we have deployed WQ-MAKER on the Jetstream cloud platform, helping to annotate over 30 genomes and accelerating performance from days to hours and weeks to days.

Original languageEnglish (US)
Title of host publicationProceedings - 2018 IEEE International Conference on Cloud Engineering, IC2E 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages72-78
Number of pages7
ISBN (Electronic)9781538650080
DOIs
StatePublished - May 16 2018
Event2018 IEEE International Conference on Cloud Engineering, IC2E 2018 - Orlando, United States
Duration: Apr 17 2018Apr 20 2018

Other

Other2018 IEEE International Conference on Cloud Engineering, IC2E 2018
CountryUnited States
CityOrlando
Period4/17/184/20/18

Fingerprint

Bioinformatics
Genes
Feedback

Keywords

  • Bioinformatics
  • Scalability
  • Workflows

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Hardware and Architecture

Cite this

Hazekamp, N., Devisetty, U. K., Merchant, N. C., & Thain, D. (2018). MAKER as a service: Moving HPC applications to jetstream cloud. In Proceedings - 2018 IEEE International Conference on Cloud Engineering, IC2E 2018 (pp. 72-78). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/IC2E.2018.00029

MAKER as a service : Moving HPC applications to jetstream cloud. / Hazekamp, Nicholas; Devisetty, Upendra Kumar; Merchant, Nirav C; Thain, Douglas.

Proceedings - 2018 IEEE International Conference on Cloud Engineering, IC2E 2018. Institute of Electrical and Electronics Engineers Inc., 2018. p. 72-78.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Hazekamp, N, Devisetty, UK, Merchant, NC & Thain, D 2018, MAKER as a service: Moving HPC applications to jetstream cloud. in Proceedings - 2018 IEEE International Conference on Cloud Engineering, IC2E 2018. Institute of Electrical and Electronics Engineers Inc., pp. 72-78, 2018 IEEE International Conference on Cloud Engineering, IC2E 2018, Orlando, United States, 4/17/18. https://doi.org/10.1109/IC2E.2018.00029
Hazekamp N, Devisetty UK, Merchant NC, Thain D. MAKER as a service: Moving HPC applications to jetstream cloud. In Proceedings - 2018 IEEE International Conference on Cloud Engineering, IC2E 2018. Institute of Electrical and Electronics Engineers Inc. 2018. p. 72-78 https://doi.org/10.1109/IC2E.2018.00029
Hazekamp, Nicholas ; Devisetty, Upendra Kumar ; Merchant, Nirav C ; Thain, Douglas. / MAKER as a service : Moving HPC applications to jetstream cloud. Proceedings - 2018 IEEE International Conference on Cloud Engineering, IC2E 2018. Institute of Electrical and Electronics Engineers Inc., 2018. pp. 72-78
@inproceedings{2d50e60f4f124afa8cdab3d8d166d542,
title = "MAKER as a service: Moving HPC applications to jetstream cloud",
abstract = "As cloud resources become more available as an execution platform, the need to transition applications between HPC and the cloud becomes a necessity. However, because of the complex setup and system specific demands of these applications, transition is difficult and may not scale as desired. Jetstream is a NSF funded cloud service that is aiming to provide these services for users in a dynamical allocated nature. In this work we look at three key areas to focus on when transitioning between resources: providing a portable reproducible environment, scaling between local and remote resources, and using feedback to the user for informing configuration and runtime decisions. Building on the MAKER bioinformatic application, we have deployed WQ-MAKER on the Jetstream cloud platform, helping to annotate over 30 genomes and accelerating performance from days to hours and weeks to days.",
keywords = "Bioinformatics, Scalability, Workflows",
author = "Nicholas Hazekamp and Devisetty, {Upendra Kumar} and Merchant, {Nirav C} and Douglas Thain",
year = "2018",
month = "5",
day = "16",
doi = "10.1109/IC2E.2018.00029",
language = "English (US)",
pages = "72--78",
booktitle = "Proceedings - 2018 IEEE International Conference on Cloud Engineering, IC2E 2018",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - MAKER as a service

T2 - Moving HPC applications to jetstream cloud

AU - Hazekamp, Nicholas

AU - Devisetty, Upendra Kumar

AU - Merchant, Nirav C

AU - Thain, Douglas

PY - 2018/5/16

Y1 - 2018/5/16

N2 - As cloud resources become more available as an execution platform, the need to transition applications between HPC and the cloud becomes a necessity. However, because of the complex setup and system specific demands of these applications, transition is difficult and may not scale as desired. Jetstream is a NSF funded cloud service that is aiming to provide these services for users in a dynamical allocated nature. In this work we look at three key areas to focus on when transitioning between resources: providing a portable reproducible environment, scaling between local and remote resources, and using feedback to the user for informing configuration and runtime decisions. Building on the MAKER bioinformatic application, we have deployed WQ-MAKER on the Jetstream cloud platform, helping to annotate over 30 genomes and accelerating performance from days to hours and weeks to days.

AB - As cloud resources become more available as an execution platform, the need to transition applications between HPC and the cloud becomes a necessity. However, because of the complex setup and system specific demands of these applications, transition is difficult and may not scale as desired. Jetstream is a NSF funded cloud service that is aiming to provide these services for users in a dynamical allocated nature. In this work we look at three key areas to focus on when transitioning between resources: providing a portable reproducible environment, scaling between local and remote resources, and using feedback to the user for informing configuration and runtime decisions. Building on the MAKER bioinformatic application, we have deployed WQ-MAKER on the Jetstream cloud platform, helping to annotate over 30 genomes and accelerating performance from days to hours and weeks to days.

KW - Bioinformatics

KW - Scalability

KW - Workflows

UR - http://www.scopus.com/inward/record.url?scp=85048316274&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85048316274&partnerID=8YFLogxK

U2 - 10.1109/IC2E.2018.00029

DO - 10.1109/IC2E.2018.00029

M3 - Conference contribution

SP - 72

EP - 78

BT - Proceedings - 2018 IEEE International Conference on Cloud Engineering, IC2E 2018

PB - Institute of Electrical and Electronics Engineers Inc.

ER -