Stochastic Approximation of Score Functions for Gaussian Processes

Gaussian processes are the cornerstone of statistical analysis in many application areas. However, most of the applications are limited by their need to use the Cholesky factorization in the computation of the likelihood. We present a matrix-free approach for computing the solution of the maximum likelihood problem via a stochastic approximation of the score functions using the Hutchinson estimator. Under certain conditions, including bounded condition number of the covariance matrix, the approach achieves O(n) storage and nearly O(n) computational effort per optimization step, where n is the number of data sites. Furthermore, we discuss a modification of the approximation in which design elements of the stochastic terms mimic patterns from a 2^n factorial design. We prove these designs are always at least as good as the unstructured design, and we demonstrate through simulation that they can produce a substantial improvement over random designs. Our findings are validated by several numerical examples and an environmental application for data sets with up to 1 billion observations.