Estimating the intrinsic dimension of datasets by a minimal neighborhood information.

ABSTRACT: Analyzing large volumes of high-dimensional data is an issue of fundamental importance in data science, molecular simulations and beyond. Several approaches work on the assumption that the important content of a dataset belongs to a manifold whose Intrinsic Dimension (ID) is much lower than the crude large number of coordinates. Such manifold is generally twisted and curved; in addition points on it will be non-uniformly distributed: two factors that make the identification of the ID and its exploitation really hard. Here we propose a new ID estimator using only the distance of the first and the second nearest neighbor of each point in the sample. This extreme minimality enables us to reduce the effects of curvature, of density variation, and the resulting computational cost. The ID estima

SUBMITTER: Facco E 

PROVIDER: S-EPMC5610237 | biostudies-literature | 2017 Sep

REPOSITORIES: biostudies-literature