Information rates of nonparametric Gaussian process methods

A.W. Vaart, van der, J.H. Zanten, van

Research output: Contribution to journalArticleAcademicpeer-review

51 Citations (Scopus)
39 Downloads (Pure)

Abstract

We consider the quality of learning a response function by a nonparametric Bayesian approach using a Gaussian process (GP) prior on the response function. We upper bound the quadratic risk of the learning procedure, which in turn is an upper bound on the Kullback-Leibler information between the predictive and true data distribution. The upper bound is expressed in small ball probabilities and concentration measures of the GP prior. We illustrate the computation of the upper bound for the Matérn and squared exponential kernels. For these priors the risk, and hence the information criterion, tends to zero for all continuous response functions. However, the rate at which this happens depends on the combination of true response function and Gaussian prior, and is expressible in a certain concentration function. In particular, the results show that for good performance, the regularity of the GP prior should match the regularity of the unknown response function.
Original languageEnglish
Pages (from-to)2095-2119
JournalJournal of Machine Learning Research
Volume12
Publication statusPublished - 2011

Fingerprint Dive into the research topics of 'Information rates of nonparametric Gaussian process methods'. Together they form a unique fingerprint.

  • Cite this

    Vaart, van der, A. W., & Zanten, van, J. H. (2011). Information rates of nonparametric Gaussian process methods. Journal of Machine Learning Research, 12, 2095-2119.