The impact of encoding–decoding schemes and weight normalization in spiking neural networks

Zhengzhong Liang, David Schwartz, Gregory Ditzler, Onur Ozan Koyluoglu

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

Spike-timing Dependent Plasticity (STDP) is a learning mechanism that can capture causal relationships between events. STDP is considered a foundational element of memory and learning in biological neural networks. Previous research efforts endeavored to understand the functionality of STDP's learning window in spiking neural networks (SNNs). In this study, we investigate the interaction among different encoding/decoding schemes, STDP learning windows and normalization rules for the SNN classifier, trained and tested on MNIST, NIST and ETH80-Contour datasets. The results show that when no normalization rules are applied, classical STDP typically achieves the best performance. Additionally, first-spike decoding classifiers require much less decoding time than a spike count decoding classifier. Thirdly, when no normalization rule is applied, the classifier accuracy decreases as the encoding duration increases from 10ms to 34ms using count decoding scheme. Finally, normalization of output weights is shown to improve the performance of a first-spike decoding classifier, which reveals the importance of weight normalization to SNN.

Original languageEnglish (US)
Pages (from-to)365-378
Number of pages14
JournalNeural Networks
Volume108
DOIs
StatePublished - Dec 1 2018
Externally publishedYes

Fingerprint

Decoding
Learning
Neural networks
Classifiers
Weights and Measures
Plasticity
Research
Data storage equipment

Keywords

  • Decoding
  • Encoding
  • Learning window
  • Normalization
  • Spike-timing dependent plasticity
  • Spiking neural network

ASJC Scopus subject areas

  • Cognitive Neuroscience
  • Artificial Intelligence

Cite this

The impact of encoding–decoding schemes and weight normalization in spiking neural networks. / Liang, Zhengzhong; Schwartz, David; Ditzler, Gregory; Koyluoglu, Onur Ozan.

In: Neural Networks, Vol. 108, 01.12.2018, p. 365-378.

Research output: Contribution to journalArticle

Liang, Zhengzhong ; Schwartz, David ; Ditzler, Gregory ; Koyluoglu, Onur Ozan. / The impact of encoding–decoding schemes and weight normalization in spiking neural networks. In: Neural Networks. 2018 ; Vol. 108. pp. 365-378.
@article{e47a313971824ad78f5070e6d3d8d6c8,
title = "The impact of encoding–decoding schemes and weight normalization in spiking neural networks",
abstract = "Spike-timing Dependent Plasticity (STDP) is a learning mechanism that can capture causal relationships between events. STDP is considered a foundational element of memory and learning in biological neural networks. Previous research efforts endeavored to understand the functionality of STDP's learning window in spiking neural networks (SNNs). In this study, we investigate the interaction among different encoding/decoding schemes, STDP learning windows and normalization rules for the SNN classifier, trained and tested on MNIST, NIST and ETH80-Contour datasets. The results show that when no normalization rules are applied, classical STDP typically achieves the best performance. Additionally, first-spike decoding classifiers require much less decoding time than a spike count decoding classifier. Thirdly, when no normalization rule is applied, the classifier accuracy decreases as the encoding duration increases from 10ms to 34ms using count decoding scheme. Finally, normalization of output weights is shown to improve the performance of a first-spike decoding classifier, which reveals the importance of weight normalization to SNN.",
keywords = "Decoding, Encoding, Learning window, Normalization, Spike-timing dependent plasticity, Spiking neural network",
author = "Zhengzhong Liang and David Schwartz and Gregory Ditzler and Koyluoglu, {Onur Ozan}",
year = "2018",
month = "12",
day = "1",
doi = "10.1016/j.neunet.2018.08.024",
language = "English (US)",
volume = "108",
pages = "365--378",
journal = "Neural Networks",
issn = "0893-6080",
publisher = "Elsevier Limited",

}

TY - JOUR

T1 - The impact of encoding–decoding schemes and weight normalization in spiking neural networks

AU - Liang, Zhengzhong

AU - Schwartz, David

AU - Ditzler, Gregory

AU - Koyluoglu, Onur Ozan

PY - 2018/12/1

Y1 - 2018/12/1

N2 - Spike-timing Dependent Plasticity (STDP) is a learning mechanism that can capture causal relationships between events. STDP is considered a foundational element of memory and learning in biological neural networks. Previous research efforts endeavored to understand the functionality of STDP's learning window in spiking neural networks (SNNs). In this study, we investigate the interaction among different encoding/decoding schemes, STDP learning windows and normalization rules for the SNN classifier, trained and tested on MNIST, NIST and ETH80-Contour datasets. The results show that when no normalization rules are applied, classical STDP typically achieves the best performance. Additionally, first-spike decoding classifiers require much less decoding time than a spike count decoding classifier. Thirdly, when no normalization rule is applied, the classifier accuracy decreases as the encoding duration increases from 10ms to 34ms using count decoding scheme. Finally, normalization of output weights is shown to improve the performance of a first-spike decoding classifier, which reveals the importance of weight normalization to SNN.

AB - Spike-timing Dependent Plasticity (STDP) is a learning mechanism that can capture causal relationships between events. STDP is considered a foundational element of memory and learning in biological neural networks. Previous research efforts endeavored to understand the functionality of STDP's learning window in spiking neural networks (SNNs). In this study, we investigate the interaction among different encoding/decoding schemes, STDP learning windows and normalization rules for the SNN classifier, trained and tested on MNIST, NIST and ETH80-Contour datasets. The results show that when no normalization rules are applied, classical STDP typically achieves the best performance. Additionally, first-spike decoding classifiers require much less decoding time than a spike count decoding classifier. Thirdly, when no normalization rule is applied, the classifier accuracy decreases as the encoding duration increases from 10ms to 34ms using count decoding scheme. Finally, normalization of output weights is shown to improve the performance of a first-spike decoding classifier, which reveals the importance of weight normalization to SNN.

KW - Decoding

KW - Encoding

KW - Learning window

KW - Normalization

KW - Spike-timing dependent plasticity

KW - Spiking neural network

UR - http://www.scopus.com/inward/record.url?scp=85053753438&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85053753438&partnerID=8YFLogxK

U2 - 10.1016/j.neunet.2018.08.024

DO - 10.1016/j.neunet.2018.08.024

M3 - Article

C2 - 30261415

AN - SCOPUS:85053753438

VL - 108

SP - 365

EP - 378

JO - Neural Networks

JF - Neural Networks

SN - 0893-6080

ER -