Convergence analysis and application for high-order neural networks based on gradient descent learning algorithm via smooth regularization.

ABSTRACT: The pi-sigma network (PSN), as a high-order network, has demonstrated its capacity for rapid learning and strong nonlinear processing. This paper proposes a type algorithm, for PSNs using a batch gradient method based on [Formula: see text] regularization. Direct application of [Formula: see text] regularization during network training presents two main drawbacks. There are numerical oscillations and theoretical challenges in computing the gradients at the origin. We then introduced smoothing functions by approximating the [Formula: see text] regularization to overcome these obstacles, resulting in a new gradient descent method based on smoothing [Formula: see text] regularization (GDS[Formula: see text]). Numerical results for the 4-dimensional parity problem and the nonlinear Gabor funct

SUBMITTER: Mohamed KS 

PROVIDER: S-EPMC12770431 | biostudies-literature | 2025 Dec

REPOSITORIES: biostudies-literature