TY - JOUR
T1 - Dimension estimation using random connection models
AU - Serra, P.
AU - Mandjes, M.R.H.
PY - 2017/11
Y1 - 2017/11
N2 - Information about intrinsic dimension is crucial to perform dimensionality reduction, compress information, design efficient algorithms, and do statistical adaptation. In this paper we propose an estimator for the intrinsic dimension of a data set. The estimator is based on binary neighbourhood information about the observations in the form of two adjacency matrices, and does not require any explicit distance information. The underlying graph is modelled according to a subset of a specific random connection model, sometimes referred to as the Poisson blob model. Computationally the estimator scales like n log n, and we specify its asymptotic distribution and rate of convergence. A simulation study on both real and simulated data shows that our approach compares favourably with some competing methods from the literature, including approaches that rely on distance information.
AB - Information about intrinsic dimension is crucial to perform dimensionality reduction, compress information, design efficient algorithms, and do statistical adaptation. In this paper we propose an estimator for the intrinsic dimension of a data set. The estimator is based on binary neighbourhood information about the observations in the form of two adjacency matrices, and does not require any explicit distance information. The underlying graph is modelled according to a subset of a specific random connection model, sometimes referred to as the Poisson blob model. Computationally the estimator scales like n log n, and we specify its asymptotic distribution and rate of convergence. A simulation study on both real and simulated data shows that our approach compares favourably with some competing methods from the literature, including approaches that rely on distance information.
KW - Adaptation
KW - Dimensionality reduction
KW - Intrinsic dimension
KW - Random connection model
KW - Random graph
UR - http://www.scopus.com/inward/record.url?scp=85040725812&partnerID=8YFLogxK
M3 - Article
VL - 18
JO - Journal of Machine Learning Research
JF - Journal of Machine Learning Research
SN - 1532-4435
M1 - 138
ER -