Dataset Information

Implementation of Dropout Neuronal Units Based on Stochastic Memristive Devices in Neural Networks with High Classification Accuracy.

ABSTRACT: Neural networks based on memristive devices have achieved great progress recently. However, memristive synapses with nonlinearity and asymmetry seriously limit the classification accuracy. Moreover, insufficient number of training samples in many cases also have negative effect on the classification accuracy of neural networks due to overfitting. In this work, dropout neuronal units are developed based on stochastic volatile memristive devices of Ag/Ta₂O₅:Ag/Pt. The memristive neural network using the dropout neuronal units effectively solves the problem of overfitting and mitigates the negative effects of the nonideality of memristive synapses, eventually achieves a classification accuracy comparable to the theoretical limit. The stochastic and volatile switching performances of the Ag/Ta₂O₅:Ag/Pt device are attributed to the stochastical rupture of the Ag filament under high electrical stress in the Ta₂O₅ layer, according to the TEM observation and the kinetic Monte Carlo simulation.

SUBMITTER: Huang HM

PROVIDER: S-EPMC7509653 | biostudies-literature | 2020 Sep

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Implementation of Dropout Neuronal Units Based on Stochastic Memristive Devices in Neural Networks with High Classification Accuracy.

Huang He-Ming HM Xiao Yu Y Yang Rui R Yu Ye-Tian YT He Hui-Kai HK Wang Zhe Z Guo Xin X

Advanced science (Weinheim, Baden-Wurttemberg, Germany) 20200726 18

Neural networks based on memristive devices have achieved great progress recently. However, memristive synapses with nonlinearity and asymmetry seriously limit the classification accuracy. Moreover, insufficient number of training samples in many cases also have negative effect on the classification accuracy of neural networks due to overfitting. In this work, dropout neuronal units are developed based on stochastic volatile memristive devices of Ag/Ta<sub>2</sub>O<sub>5</sub>:Ag/Pt. The memrist ...[more]

PMID: 32999852

Similar Datasets

Project description:While there is an abundance of research on neural networks that are “inspired” by the brain, few mimic the critical temporal compute features that allow the brain to efficiently perform complex computations. Even fewer methods emulate the heterogeneity of learning produced by biological neurons. Memory devices, such as memristors, are also investigated for their potential to implement neuronal functions in electronic hardware. However, memristors in computing architectures typically operate as non-volatile memories, either as storage or as the weights in a multiply-and-accumulate function that requires direct access to manipulate memristance via a costly learning algorithm. Hence, the integration of memristors into architectures as time-dependent computational units is studied, starting with the development of a compact and versatile mathematical model that is capable of emulating flux-linkage controlled analog (FLCA) memristors and their unique temporal characteristics. The proposed model, which is validated against experimental FLCA LixNbO2 intercalation devices, is used to create memristive circuits that mimic neuronal behavior such as desensitization, paired-pulse facilitation, and spike-timing-dependent plasticity. The model is used to demonstrate building blocks of biomimetic learning via dynamical memristive circuits that implement biomimetic learning rules in a self-training neural network, with dynamical memristive weights that are capable of associative lifelong learning. Successful training of the dynamical memristive neural network to perform image classification of handwritten digits is shown, including lifelong learning by having the dynamical memristive network relearn different characters in succession. An analog computing architecture that learns to associate input-to-input correlations is also introduced, with examples demonstrating image classification and pattern recognition without convolution. The biomimetic functions shown in this paper result from fully ion-driven memristive circuits devoid of integrating capacitors and thus are instructive for exploiting the immense potential of memristive technology for neuromorphic computation in hardware and allowing a common architecture to be applied to a wide range of learning rules, including STDP, magnitude, frequency, and pulse shape among others, to enable an inorganic implementation of the complex heterogeneity of biological neural systems.

Dataset Information

Implementation of Dropout Neuronal Units Based on Stochastic Memristive Devices in Neural Networks with High Classification Accuracy.

Publications

Implementation of Dropout Neuronal Units Based on Stochastic Memristive Devices in Neural Networks with High Classification Accuracy.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets