Linear Recursive Feature Machines provably recover low-rank matrices.

ABSTRACT: A fundamental problem in machine learning is to understand how neural networks make accurate predictions, while seemingly bypassing the curse of dimensionality. A possible explanation is that common training algorithms for neural networks implicitly perform dimensionality reduction-a process called feature learning. Recent work [A. Radhakrishnan, D. Beaglehole, P. Pandit, M. Belkin, Science 383, 1461-1467 (2024).] posited that the effects of feature learning can be elicited from a classical statistical estimator called the average gradient outer product (AGOP). The authors proposed Recursive Feature Machines (RFMs) as an algorithm that explicitly performs feature learning by alternating between 1) reweighting the feature vectors by the AGOP and 2) learning the prediction func

SUBMITTER: Radhakrishnan A 

PROVIDER: S-EPMC12002225 | biostudies-literature | 2025 Apr

REPOSITORIES: biostudies-literature