Learning Sparse Deep Neural Networks with a Spike-and-Slab Prior.

ABSTRACT: Deep learning has achieved great successes in many machine learning tasks. However, the deep neural networks (DNNs) are often severely over-parameterized, making them computationally expensive, memory intensive, less interpretable and mis-calibrated. We study sparse DNNs under the Bayesian framework: we establish posterior consistency and structure selection consistency for Bayesian DNNs with a spike-and-slab prior, and illustrate their performance using examples on high-dimensional nonlinear variable selection, large network compression and model calibration. Our numerical results indicate that sparsity is essential for improving the prediction accuracy and calibration of the DNN.

SUBMITTER: Sun Y 

PROVIDER: S-EPMC8570537 | biostudies-literature | 2022 Jan

REPOSITORIES: biostudies-literature