Balancing the learning ability and memory demand of a perceptron-based dynamically trainable neural network

Edward Richter, Spencer Valancius, Josiah McClanahan, John Mixter, Ali Akoglu

Research output: Contribution to journalArticle

Abstract

Artificial neural networks (ANNs) have become a popular means of solving complex problems in prediction-based applications such as image and natural language processing. Two challenges prominent in the neural network domain are the practicality of hardware implementation and dynamically training the network. In this study, we address these challenges with a development methodology that balances the hardware footprint and the quality of the ANN. We use the well-known perceptron-based branch prediction problem as a case study for demonstrating this methodology. This problem is perfect to analyze dynamic hardware implementations of ANNs because it exists in hardware and trains dynamically. Using our hierarchical configuration search space exploration, we show that we can decrease the memory footprint of a standard perceptron-based branch predictor by 2.3(Formula presented.) with only a 0.6% decrease in prediction accuracy.

Original languageEnglish (US)
Pages (from-to)1-25
Number of pages25
JournalJournal of Supercomputing
DOIs
StateAccepted/In press - Apr 16 2018

Fingerprint

Perceptron
Balancing
Artificial Neural Network
Hardware Implementation
Neural Networks
Neural networks
Data storage equipment
Branch Prediction
Hardware
Decrease
Methodology
Prediction
Configuration Space
Natural Language
Search Space
Predictors
Branch
Demand
Learning
Processing

Keywords

  • Artificial neural network
  • Branch prediction
  • Perceptron
  • SimpleScalar

ASJC Scopus subject areas

  • Software
  • Theoretical Computer Science
  • Information Systems
  • Hardware and Architecture

Cite this

Balancing the learning ability and memory demand of a perceptron-based dynamically trainable neural network. / Richter, Edward; Valancius, Spencer; McClanahan, Josiah; Mixter, John; Akoglu, Ali.

In: Journal of Supercomputing, 16.04.2018, p. 1-25.

Research output: Contribution to journalArticle

Richter, Edward ; Valancius, Spencer ; McClanahan, Josiah ; Mixter, John ; Akoglu, Ali. / Balancing the learning ability and memory demand of a perceptron-based dynamically trainable neural network. In: Journal of Supercomputing. 2018 ; pp. 1-25.
@article{d6e37b434a2c424d8c964ce84c0a5b9c,
title = "Balancing the learning ability and memory demand of a perceptron-based dynamically trainable neural network",
abstract = "Artificial neural networks (ANNs) have become a popular means of solving complex problems in prediction-based applications such as image and natural language processing. Two challenges prominent in the neural network domain are the practicality of hardware implementation and dynamically training the network. In this study, we address these challenges with a development methodology that balances the hardware footprint and the quality of the ANN. We use the well-known perceptron-based branch prediction problem as a case study for demonstrating this methodology. This problem is perfect to analyze dynamic hardware implementations of ANNs because it exists in hardware and trains dynamically. Using our hierarchical configuration search space exploration, we show that we can decrease the memory footprint of a standard perceptron-based branch predictor by 2.3(Formula presented.) with only a 0.6{\%} decrease in prediction accuracy.",
keywords = "Artificial neural network, Branch prediction, Perceptron, SimpleScalar",
author = "Edward Richter and Spencer Valancius and Josiah McClanahan and John Mixter and Ali Akoglu",
year = "2018",
month = "4",
day = "16",
doi = "10.1007/s11227-018-2374-x",
language = "English (US)",
pages = "1--25",
journal = "Journal of Supercomputing",
issn = "0920-8542",
publisher = "Springer Netherlands",

}

TY - JOUR

T1 - Balancing the learning ability and memory demand of a perceptron-based dynamically trainable neural network

AU - Richter, Edward

AU - Valancius, Spencer

AU - McClanahan, Josiah

AU - Mixter, John

AU - Akoglu, Ali

PY - 2018/4/16

Y1 - 2018/4/16

N2 - Artificial neural networks (ANNs) have become a popular means of solving complex problems in prediction-based applications such as image and natural language processing. Two challenges prominent in the neural network domain are the practicality of hardware implementation and dynamically training the network. In this study, we address these challenges with a development methodology that balances the hardware footprint and the quality of the ANN. We use the well-known perceptron-based branch prediction problem as a case study for demonstrating this methodology. This problem is perfect to analyze dynamic hardware implementations of ANNs because it exists in hardware and trains dynamically. Using our hierarchical configuration search space exploration, we show that we can decrease the memory footprint of a standard perceptron-based branch predictor by 2.3(Formula presented.) with only a 0.6% decrease in prediction accuracy.

AB - Artificial neural networks (ANNs) have become a popular means of solving complex problems in prediction-based applications such as image and natural language processing. Two challenges prominent in the neural network domain are the practicality of hardware implementation and dynamically training the network. In this study, we address these challenges with a development methodology that balances the hardware footprint and the quality of the ANN. We use the well-known perceptron-based branch prediction problem as a case study for demonstrating this methodology. This problem is perfect to analyze dynamic hardware implementations of ANNs because it exists in hardware and trains dynamically. Using our hierarchical configuration search space exploration, we show that we can decrease the memory footprint of a standard perceptron-based branch predictor by 2.3(Formula presented.) with only a 0.6% decrease in prediction accuracy.

KW - Artificial neural network

KW - Branch prediction

KW - Perceptron

KW - SimpleScalar

UR - http://www.scopus.com/inward/record.url?scp=85045458499&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85045458499&partnerID=8YFLogxK

U2 - 10.1007/s11227-018-2374-x

DO - 10.1007/s11227-018-2374-x

M3 - Article

SP - 1

EP - 25

JO - Journal of Supercomputing

JF - Journal of Supercomputing

SN - 0920-8542

ER -