EXODUS: Stable and efficient training of spiking neural networks

ABSTRACT:

Introduction

Spiking Neural Networks (SNNs) are gaining significant traction in machine learning tasks where energy-efficiency is of utmost importance. Training such networks using the state-of-the-art back-propagation through time (BPTT) is, however, very time-consuming. Previous work employs an efficient GPU-accelerated backpropagation algorithm called SLAYER, which speeds up training considerably. SLAYER, however, does not take into account the neuron reset mechanism while computing the gradients, which we argue to be the source of numerical instability. To counteract this, SLAYER introduces a gradient scale hyper parameter across layers, which needs manual tuning.

Methods

In this paper, we modify SLAYER and design an algorithm called EXODUS, that accounts for the neur

SUBMITTER:  

PROVIDER: S-EPMC9945199 | biostudies-literature | 2023 Jan

REPOSITORIES: biostudies-literature