A Recipe For Finding The Eigenfunctions of Stationary Gaussian Process Integral Covariance Operators

Theorem. The Eigenfunctions of Integral Covariance Operators Corresponding To Stationary Gaussian Processes Are Given By The Orthogonal Complement of the Null Space Of the Inner Product Operator where the Null Space Is Identified To Be The Fourier Transforms of the Orthogonal Polynomials Whose Orthogonality Measure Is The Spectral Density Of The Gaussian Process To Which It Corresponds

Proof. Let $C(x)$ be the covariance function of a stationary Gaussian process on $\mathbb{R}$. The covariance operator $T$ is defined by:

$$(Tf)(x) = \int_{-\infty}^\infty C(x-y) f(y) \, dy.$$

The spectral density $S(\omega)$ of the process is related to $C(x)$ by Bochner's theorem:

$$C(x) = \int_{-\infty}^{\infty} e^{i\omega x} S(\omega) \, d\omega.$$

We consider polynomials $\{p_n(\omega)\}$ orthogonal with respect to $S(\omega)$ over its domain:

$$\int_{-\infty}^{\infty} p_n(\omega) p_m(\omega) S(\omega) \, d\omega = \delta_{nm}.$$

The inverse Fourier transforms of these polynomials are:

$$r_n(x) = \frac{1}{\sqrt{2\pi}} \int_{-\infty}^{\infty} p_n(\omega) e^{i\omega x} \, d\omega.$$
 

Null Space Property

We prove that $r_n(x)$ form the null space of the kernel inner product:

$$\int_0^\infty C(x)r_n(x)dx = 0$$

\begin{align*}
\int_0^\infty C(x)r_n(x)dx &= \int_0^\infty C(x) \frac{1}{\sqrt{2\pi}} \int_{-\infty}^{\infty} p_n(\omega) e^{i\omega x} \, d\omega \, dx \\
&= \int_0^\infty \int_{-\infty}^{\infty} e^{i\omega' x} S(\omega') \, d\omega' \frac{1}{\sqrt{2\pi}} \int_{-\infty}^{\infty} p_n(\omega) e^{i\omega x} \, d\omega \, dx \\
&= \frac{1}{\sqrt{2\pi}} \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} S(\omega') p_n(\omega) \int_0^\infty e^{i(\omega'+\omega) x} \, dx \, d\omega \, d\omega' \\
&= \frac{1}{\sqrt{2\pi}} \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} S(\omega') p_n(\omega) \pi \delta(\omega'+\omega) \, d\omega \, d\omega' \\
&= \frac{\pi}{\sqrt{2\pi}} \int_{-\infty}^{\infty} S(\omega') p_n(-\omega') \, d\omega' = 0
\end{align*}

Eigenfunctions from Orthogonalized Null Space

By orthogonalizing the null space $\{r_n(x)\}$, we obtain the eigenfunctions $\{\psi_n(x)\}$ of the covariance operator $T$. The orthogonalization process gives:

$$\psi_n(x) = \sum_{k=0}^n a_{nk} r_k(x)$$

where the coefficients $a_{nk}$ are given by:

$$a_{nk} = \begin{cases}
1 & \text{if } k = n \\
-\sum_{j=k}^{n-1} a_{nj} \langle r_n, \psi_j \rangle & \text{if } k < n \\
0 & \text{if } k > n
\end{cases}$$

We prove that these are indeed eigenfunctions:

Let $\psi_n(x) = \sum_k a_{nk} r_k(x)$. Then:

\begin{align*}
\int_{-\infty}^\infty C(x-y) \psi_n(y) \, dy &= \int_{-\infty}^\infty C(x-y) \sum_k a_{nk} r_k(y) \, dy \\
&= \sum_k a_{nk} \int_{-\infty}^\infty C(x-y) r_k(y) \, dy \\
&= \sum_k a_{nk} \int_{-\infty}^x C(x-y) r_k(y) \, dy \\
&= \sum_k a_{nk} \left[r_k(x) \int_0^\infty C(z) \psi_n(z) \, dz - \int_0^\infty C(z) \int_0^z r_k'(x-t) \, dt \, dz\right] \\
&= \lambda_n \sum_k a_{nk} r_k(x) = \lambda_n \psi_n(x)
\end{align*}

where the eigenvalue $\lambda_n$ is given by:

$$\lambda_n = \int_0^\infty C(z) \psi_n(z)  dz$$


Thus, it is shown that the orthogonalized null space functions are eigenfunctions of the covariance operator, providing a direct method to construct eigenfunctions for stationary operators. The eigenvalues are determined by the inner product of the covariance function with the corresponding eigenfunction. QED.

Note: this process does not depend upon the integral operator being trace class,  the kernel function itself being square integrable, or being restricted to a compact domain.

--Stephen Crowley

The Code of Existence

 The rain fell steadily on the dense canopy of trees, each droplet slipping from one branch to another, winding through the intricate web of limbs like water on a well-worn itinerary. Beneath this veil of droplets, a lone figure marched with unwavering intent, cloaked in layers that concealed both identity and intent. His path was one of peregrination, though it was no pilgrimage of peace. He moved with purpose, each step facilitated by a lifetime of intermediation between light and dark, between what was known and what was sought.

He had become something else in this journey—no longer simply a man, but an operator of forces beyond the tangible. His mind, once consumed by the mundane, now gnawed at greater truths, spurred on by a gnosis that reached far beyond typical human comprehension. The trees around him, ancient and constant, bore witness to his passage, as their roots clung deep into the substrate of the earth, an embodiment of silent perseverance.

In his hand, he carried a tome bound in brocade, its pages filled with arcane knowledge. Not just a text, but a statement of purpose, each word a step in the complex dance of conjuration, a delicate balance between the manifest and the intangible. The pages were dog-eared, worn from repeated reference, a testament to the frequency with which he had sought to decode the codomain of reality and weave his own thread through it. In truth, the knowledge within was more than any single mind could encompass, a fascicle of endless possibilities bound to an infinite series of outcomes.

The path twisted, leading him into a clearing where a massive syzygy of celestial bodies aligned overhead, their faint glow cast down upon the forest floor. It was here that he would execute his plan. He pulled forth a piece of chalk and began to scribe a great circular symbol upon the ground—a diagram of percolation that, when complete, would allow him to issue forth the energy he sought to channel. The chalk scratched across the earth with an almost poetic rhythm, a sound so perfectly aligned with the natural hum of the universe that it seemed to come from the depths of the earth itself.

As he finished, the wind picked up, a distant rumble of thunder heralding the imminent arrival of a storm. It was no coincidence that this moment had been chosen. The storm was both a symbol and a conduit, its energy a mechanism through which his plan could operate. He stepped back, hands trembling slightly with anticipation, for he was about to cross into the realm of the incredible.

The symbol on the ground began to glow faintly, at first nothing more than a whisper of light, but it quickly expanded. In that moment, he felt the depth of his own inquiry into the cosmos compress into a singularity—a focal point of all he had pondered. The air around him grew dense, filled with a strange, palpable energy, and in the distance, the trees groaned under the weight of this unearthly pressure.

He had done it. He had cracked the code of existence itself, the scalar complexities of life and the universe now operating under his will. But there was a cost. His mind, once sharp and full of calculated reason, now felt the extent of the expansion. He realized that he was no longer just pondering abstract concepts, but had become intimately, irrevocably connected to them. He was subsumed by the very forces he sought to control, each thought, each notion stretching him beyond his limits.

In that moment of revelation, he understood the truth: the journey had never been one of mastery over knowledge, but rather a lesson in acquiescence to forces far greater than any one mind could grasp. His quest, initially conceived as a battle against the limits of human understanding, was nothing more than a quest for alignment with the cosmic will.

He exhaled deeply, feeling his spirit resonate with the deep, unbridled hum of the universe. The circle beneath him, once glowing with raw power, began to fade, leaving behind only the faintest traces of chalk and energy, as if the events that transpired had been no more than a dream.