The Spectral Representation of Stationary Processes: Bridging Gelfand-Vilenkin and Wiener-Khinchin

Introduction

At the heart of stochastic process theory lies a profound connection between time and frequency domains, elegantly captured by two fundamental theorems: the Gelfand-Vilenkin Spectral Representation Theorem and the Wiener-Khinchin Theorem. These results, while often presented separately, are intimately linked, offering complementary insights into the nature of stationary processes.

Gelfand-Vilenkin Theorem

The Gelfand-Vilenkin theorem provides a general, measure-theoretic framework for representing wide-sense stationary processes. Consider a stochastic process $\{X(t) : t \in \mathbb{R}\}$ on a probability space $(\Omega, \mathcal{F}, P)$. The theorem states that we can represent $X(t)$ as:

$$X(t) = \int_{\mathbb{R}} e^{i\omega t} dZ(\omega)$$

Here, $Z(\omega)$ is a complex-valued process with orthogonal increments, and the integral is taken over the real line. This representation expresses the process as a superposition of complex exponentials, each contributing to the overall behavior of $X(t)$ at different frequencies.

The key to understanding this representation lies in the spectral measure $\mu$, which is defined by $E[|Z(A)|^2] = \mu(A)$ for Borel sets $A$. This measure encapsulates the distribution of "energy" across different frequencies in the process.

Wiener-Khinchin Theorem

The Wiener-Khinchin theorem, in its classical form, states that for a wide-sense stationary process, the power spectral density $S(\omega)$ is the Fourier transform of the autocorrelation function:

$$S(\omega) = \int_{\mathbb{R}} R(\tau) e^{-i\omega\tau} d\tau$$

Bridging the Theorems

The connection becomes clear when we recognize that the spectral measure $\mu$ from Gelfand-Vilenkin is related to the power spectral density $S(\omega)$ from Wiener-Khinchin by:

$$d\mu(\omega) = \frac{1}{2\pi} S(\omega) d\omega$$

This relationship holds when $S(\omega)$ exists as a well-defined function. However, the beauty of the Gelfand-Vilenkin approach is that it allows for spectral measures that may not have a density, accommodating processes with more complex spectral structures.

Spectral Density Example

To illustrate the connection between spectral properties and sample path behavior, consider a process with a spectral density of the form:

$$S(\omega) = \frac{1}{\sqrt{1 - \omega^2}}, \quad |\omega| < 1$$

This density has singularities at $\omega = \pm 1$, which profoundly influence the behavior of the process in the time domain:

- The sample paths will be continuous and infinitely differentiable.

- The paths will exhibit rapid oscillations, reflecting the strong presence of frequencies near $\pm 1$.

- The process will show a mix of components with different periods, with those corresponding to $|\omega|$ near 1 having larger amplitudes on average.

- The autocorrelation function is $R(\tau) = J_0(\tau)$, where $J_0$ is the Bessel function of the first kind of order zero.

Frequency Interpretation

In our spectral density $S(\omega) = 1 / \sqrt{1 - \omega^2}$ with $|\omega| < 1$:

- $\omega$ represents angular frequency, with $|\omega|$ closer to 0 corresponding to longer-period components in the process.

- $|\omega|$ closer to 1 corresponds to shorter-period components.

- As $|\omega|$ approaches 1, $S(\omega)$ increases sharply, approaching infinity.

- This means components with $|\omega|$ near 1 contribute more strongly to the process variance.

Dirac Delta Example

Consider a spectral measure that is a Dirac delta function at $\omega = 0.25$:

$$S(\omega) = \delta(\omega - 0.25) + \delta(\omega + 0.25)$$

In this case:

- The process can be written as: $X(t) = A \cos(0.25t) + B \sin(0.25t)$

- The covariance function is $R(\tau) = \cos(0.25\tau)$

- The period of the covariance function is $2\pi/0.25 = 8\pi \approx 25.13$

- This illustrates that a frequency of 0.25 in the spectral domain corresponds to a period of $8\pi$ in the time domain

This example demonstrates the crucial relationship: for any peak or concentration of spectral mass at a frequency $\omega_0$, we'll see corresponding oscillations in the covariance function with period $2\pi/\omega_0$.