| Uniformly Convergent Expansions of Positive
Definite Functions |
|
by Stephen Crowley
<stephencrowley214@gmail.com>
|
|
|
Theorem 1. The covariance function
\(K (t)\) of a stationary Gaussian process has a uniformly convergent
expansion in terms of functions from the orthogonal complement of the null
space of the inner product defined by \(K\). This uniform convergence holds
initially on the real line and extends to the entire complex plane.
Proof. Let \(\{P_n (\omega)\}_{n = 0}^{\infty}\) be the
orthogonal polynomials with respect to the spectral density \(S (\omega)\)
of a stationary Gaussian process, and \(\{f_n (t)\}_{n = 0}^{\infty}\) their
Fourier transforms defined as:
| \(\displaystyle f_n (t) = \int P_n (\omega) e^{i \omega
t} d \omega\) |
Let \(K (t)\) be the covariance function of the Gaussian process.
1) First, the orthogonality of the polynomials \(P_n (\omega)\) is
established:
a) By definition of orthogonal polynomials, for \(m \neq n\):
| \(\displaystyle \int P_m (\omega) P_n (\omega) S
(\omega) d \omega = 0\) |
b) The spectral density and covariance function form a Fourier transform
pair:
| \(\displaystyle K (t) = \int S (\omega) e^{i \omega t} d
\omega\) |
2) The null space property of \(\{f_n (t)\}_{n = 1}^{\infty}\) is proven:
a) Consider the inner product \(\langle f_n, K \rangle\) for \(n \geq 1\):
| \(\displaystyle \langle f_n, K \rangle = \int f_n (t) K
(t) dt = \int f_n (t) \left( \int S
(\omega) e^{i \omega t} d \omega
\right) dt\) |
b) Applying Fubini's theorem:
| \(\displaystyle \langle f_n, K \rangle = \int S (\omega)
\left( \int f_n (t) e^{i \omega t} dt
\right) d \omega = \int S (\omega)
P_n (\omega) d \omega = 0\) |
Thus, \(\{f_n (t)\}_{n = 1}^{\infty}\) are in the null space of the inner
product defined by \(K\).
3) The Gram-Schmidt process is applied to the Fourier transforms \(\{f_n
(t)\}_{n = 0}^{\infty}\) to obtain an orthonormal basis \(\{g_n (t)\}_{n =
0}^{\infty}\) for the orthogonal complement of the null space:
| \(\displaystyle \tilde{g}_0 (t) = f_0 (t)\) |
| \(\displaystyle g_0 (t) = \frac{\tilde{g}_0 (t)}{\|
\tilde{g}_0 (t)\|}\) |
For \(n \geq 1\):
| \(\displaystyle \tilde{g}_n (t) = f_n (t) - \sum_{k =
0}^{n - 1} \langle f_n, g_k \rangle g_k
(t)\) |
| \(\displaystyle g_n (t) = \frac{\tilde{g}_n (t)}{\|
\tilde{g}_n (t)\|}\) |
where \(\| \cdot \|\) and \(\langle \cdot, \cdot \rangle\) denote the norm
and inner product induced by \(K\), respectively.
4) \(K (t)\) can be expressed in terms of this basis:
| \(\displaystyle K (t) = \sum_{n = 0}^{\infty} \alpha_n
g_n (t)\) |
where \(\alpha_n = \langle K, g_n \rangle\) are the projections of \(K\)
onto \(g_n (t)\).
5) The partial sum is defined as:
| \(\displaystyle S_N (t) = \sum_{n = 0}^N \alpha_n g_n
(t)\) |
6) The sequence of partial sums \(S_N (t)\) converges uniformly to \(K (t)\)
in the canonical metric induced by the kernel as \(N \to \infty\).
7) To realize this, recall that the canonical metric is defined as:
| \(\displaystyle d (f, g) = \sqrt{\int \int (f (t) - g
(t)) (f (s) - g (s)) K (t - s) dtds}\) |
8) The error in this metric is considered:
| \(\displaystyle d (K, S_N)^2 = \int \int (K (t) - S_N
(t)) (K (s) - S_N (s)) K (t - s) dtds\) |
9) As the kernel operator is compact in this metric:
For every positive epsilon, there exists an N (which depends on epsilon)
less than n, such that the distance between K and Sn is less than
epsilon.
| \(\displaystyle \exists N (\epsilon) < n : d (K, S_n) <
\epsilon \quad \forall \epsilon > 0\) |
10) Extension to the Complex Plane:
a) The covariance function \(K (t)\) of a stationary Gaussian process is
positive definite and therefore analytic in the complex plane.
b) The partial sum \(S_N (t)\) is a finite sum of analytic functions (as
\(g_n (t)\) are analytic), and is thus analytic in the complex plane.
c) The convergence of \(S_N (t)\) to \(K (t)\) on the real line is uniform,
as shown in steps 1-9.
d) Consider any open disk D in the complex plane that intersects the real
line. The intersection of D with the real line contains an accumulation
point.
e) By the Identity Theorem for analytic functions, since \(K (t)\) and \(S_N
(t)\) agree on a set with an accumulation point within D (namely, the
intersection of D with the real line), they must agree on the entire disk D.
f) As this holds for any disk intersecting the real line, and such disks
cover the entire complex plane, the uniform convergence of \(S_N (t)\) to
\(K (t)\) extends to the entire complex plane.
Thus, it has been shown that the covariance function \(K (t)\) has a
uniformly convergent expansion in terms of functions from the orthogonal
complement of the null space of the inner product defined by \(K\). This
uniform convergence holds initially on the real line and extends to the
entire complex plane.\(\Box\)
