Noise Helps Optimization Escape From Saddle Points in the Synaptic Plasticity.

ABSTRACT: Numerous experimental studies suggest that noise is inherent in the human brain. However, the functional importance of noise remains unknown. n particular, from a computational perspective, such stochasticity is potentially harmful to brain function. In machine learning, a large number of saddle points are surrounded by high error plateaus and give the illusion of the existence of local minimum. As a result, being trapped in the saddle points can dramatically impair learning and adding noise will attack such saddle point problems in high-dimensional optimization, especially under the strict saddle condition. Motivated by these arguments, we propose one biologically plausible noise structure and demonstrate that noise can efficiently improve the optimization performance of spiking neural ne

SUBMITTER: Fang Y 

PROVIDER: S-EPMC7201302 | biostudies-literature | 2020

REPOSITORIES: biostudies-literature