Uniformly Convergent Expansions of Positive Definite Functions

Uniformly Convergent Expansions of Positive Definite Functions by Stephen Crowley <stephencrowley214@gmail.com> October 22, 2024

Theorem 1. The covariance function \(K (t)\) of a stationary Gaussian process has a uniformly convergent expansion in terms of functions from the orthogonal complement of the null space of the inner product defined by \(K\). This uniform convergence holds initially on the real line and extends to the entire complex plane.

Proof. Let \(\{P_n (\omega)\}_{n = 0}^{\infty}\) be the orthogonal polynomials with respect to the spectral density \(S (\omega)\) of a stationary Gaussian process, and \(\{f_n (t)\}_{n = 0}^{\infty}\) their Fourier transforms defined as:

\(\displaystyle f_n (t) = \int P_n (\omega) e^{i \omega t} d \omega\)

Let \(K (t)\) be the covariance function of the Gaussian process.

1) First, the orthogonality of the polynomials \(P_n (\omega)\) is established:

a) By definition of orthogonal polynomials, for \(m \neq n\):

\(\displaystyle \int P_m (\omega) P_n (\omega) S (\omega) d \omega = 0\)

b) The spectral density and covariance function form a Fourier transform pair:

\(\displaystyle K (t) = \int S (\omega) e^{i \omega t} d \omega\)

2) The null space property of \(\{f_n (t)\}_{n = 1}^{\infty}\) is proven:

a) Consider the inner product \(\langle f_n, K \rangle\) for \(n \geq 1\):

\(\displaystyle \langle f_n, K \rangle = \int f_n (t) K (t) dt = \int f_n (t) \left( \int S (\omega) e^{i \omega t} d \omega \right) dt\)

b) Applying Fubini's theorem:

\(\displaystyle \langle f_n, K \rangle = \int S (\omega) \left( \int f_n (t) e^{i \omega t} dt \right) d \omega = \int S (\omega) P_n (\omega) d \omega = 0\)

Thus, \(\{f_n (t)\}_{n = 1}^{\infty}\) are in the null space of the inner product defined by \(K\).

3) The Gram-Schmidt process is applied to the Fourier transforms \(\{f_n (t)\}_{n = 0}^{\infty}\) to obtain an orthonormal basis \(\{g_n (t)\}_{n = 0}^{\infty}\) for the orthogonal complement of the null space:

\(\displaystyle \tilde{g}_0 (t) = f_0 (t)\)

\(\displaystyle g_0 (t) = \frac{\tilde{g}_0 (t)}{\| \tilde{g}_0 (t)\|}\)

For \(n \geq 1\):

\(\displaystyle \tilde{g}_n (t) = f_n (t) - \sum_{k = 0}^{n - 1} \langle f_n, g_k \rangle g_k (t)\)

\(\displaystyle g_n (t) = \frac{\tilde{g}_n (t)}{\| \tilde{g}_n (t)\|}\)

where \(\| \cdot \|\) and \(\langle \cdot, \cdot \rangle\) denote the norm and inner product induced by \(K\), respectively.

4) \(K (t)\) can be expressed in terms of this basis:

\(\displaystyle K (t) = \sum_{n = 0}^{\infty} \alpha_n g_n (t)\)

where \(\alpha_n = \langle K, g_n \rangle\) are the projections of \(K\) onto \(g_n (t)\).

5) The partial sum is defined as:

\(\displaystyle S_N (t) = \sum_{n = 0}^N \alpha_n g_n (t)\)

6) The sequence of partial sums \(S_N (t)\) converges uniformly to \(K (t)\) in the canonical metric induced by the kernel as \(N \to \infty\).

7) To realize this, recall that the canonical metric is defined as:

\(\displaystyle d (f, g) = \sqrt{\int \int (f (t) - g (t)) (f (s) - g (s)) K (t - s) dtds}\)

8) The error in this metric is considered:

\(\displaystyle d (K, S_N)^2 = \int \int (K (t) - S_N (t)) (K (s) - S_N (s)) K (t - s) dtds\)

9) As the kernel operator is compact in this metric:

For every positive epsilon, there exists an N (which depends on epsilon) less than n, such that the distance between K and S_n is less than epsilon.

\(\displaystyle \exists N (\epsilon) < n : d (K, S_n) < \epsilon \quad \forall \epsilon > 0\)

10) Extension to the Complex Plane:

a) The covariance function \(K (t)\) of a stationary Gaussian process is positive definite and therefore analytic in the complex plane.

b) The partial sum \(S_N (t)\) is a finite sum of analytic functions (as \(g_n (t)\) are analytic), and is thus analytic in the complex plane.

c) The convergence of \(S_N (t)\) to \(K (t)\) on the real line is uniform, as shown in steps 1-9.

d) Consider any open disk D in the complex plane that intersects the real line. The intersection of D with the real line contains an accumulation point.

e) By the Identity Theorem for analytic functions, since \(K (t)\) and \(S_N (t)\) agree on a set with an accumulation point within D (namely, the intersection of D with the real line), they must agree on the entire disk D.

f) As this holds for any disk intersecting the real line, and such disks cover the entire complex plane, the uniform convergence of \(S_N (t)\) to \(K (t)\) extends to the entire complex plane.

Thus, it has been shown that the covariance function \(K (t)\) has a uniformly convergent expansion in terms of functions from the orthogonal complement of the null space of the inner product defined by \(K\). This uniform convergence holds initially on the real line and extends to the entire complex plane.\(\Box\)