Fast Probabilistic Whitening Transformation for Ultra-High Dimensional Genetic Data.

ABSTRACT: Statistical methods often make assumptions about independence between the samples or features of a dataset. Yet correlation structure is ubiquitous in real data, so these assumptions are often not met in practice. Whitening transformations are widely applied to remove this correlation structure. Existing approaches to whitening are based on standard linear algebra, rather than a probabilistic model, and application to high dimensional datasets with n samples and p features is problematic as p approaches or exceeds n . Moreover, the computational time becomes prohibitive since the naive transform is cubic in p . Here we propose a probabilistic model for data whitening and examine its properties based on first principles as p increases. We demonstrat

SUBMITTER: Hoffman GE 

PROVIDER: S-EPMC12424976 | biostudies-literature | 2025 Sep

REPOSITORIES: biostudies-literature