Harmonizable Stochastic Processes

M.M. Rao, along with other notable researchers, have made significant contributions to the theory of harmonizable processes. Some of the fundamental theorems and results one might find in a comprehensive textbook on this topic are:

1. Loève's Harmonizability Theorem:

A complex-valued stochastic process {X(t), t ∈ R} is harmonizable if and only if its covariance function C(s,t) can be represented as:

C(s,t) = ∫∫ exp(iλs - iμt) dF(λ,μ)

where F is a complex measure of bounded variation on R² (called the spectral measure).

2. Characterization of Harmonizable Processes:

A process X(t) is harmonizable if and only if it admits a representation:

X(t) = ∫ exp(iλt) dZ(λ)

where Z(λ) is a process with orthogonal increments.

3. Cramér's Representation Theorem for Harmonizable Processes:

For any harmonizable process X(t), there exists a unique (up to equivalence) complex-valued orthogonal random measure Z(λ) such that:

X(t) = ∫ exp(iλt) dZ(λ)

4. Karhunen-Loève Theorem for Harmonizable Processes:

A harmonizable process X(t) has the representation:

X(t) = ∑ₖ √λₖ ξₖ φₖ(t)

where λₖ and φₖ(t) are eigenvalues and eigenfunctions of the integral operator associated with the covariance function, and ξₖ are uncorrelated random variables.

5. Rao's Decomposition Theorem:

Any harmonizable process can be uniquely decomposed into the sum of a purely harmonizable process and a process harmonizable in the wide sense.

6. Spectral Representation of Harmonizable Processes:

The spectral density f(λ,μ) of a harmonizable process, when it exists, is related to the spectral measure F by:

dF(λ,μ) = f(λ,μ) dλdμ

7. Continuity and Differentiability Theorem:

A harmonizable process X(t) is mean-square continuous if and only if its spectral measure F is continuous in each variable separately. It is mean-square differentiable if and only if ∫∫ (λ² + μ²) dF(λ,μ) < ∞.

8. Prediction Theory for Harmonizable Processes:

The best linear predictor of a harmonizable process X(t) given its past {X(s), s ≤ t} can be expressed in terms of the spectral measure F.

9. Sampling Theorem for Harmonizable Processes:

If a harmonizable process X(t) has a spectral measure F supported on a bounded set, then X(t) can be reconstructed from its samples at a sufficiently high rate.

10. Rao's Theorem on Equivalent Harmonizable Processes:

Two harmonizable processes are equivalent if and only if their spectral measures are equivalent.

11. Stationarity Conditions:

A harmonizable process is (wide-sense) stationary if and only if its spectral measure is concentrated on the diagonal λ = μ.

12. Gladyshev's Theorem:

A process X(t) is harmonizable if and only if for any finite set of times {t₁, ..., tₙ}, the characteristic function of (X(t₁), ..., X(tₙ)) has a certain specific form involving the spectral measure.

These theorems form the core of the theory of harmonizable processes, providing a rich framework for analyzing a wide class of non-stationary processes. M.M. Rao's contributions, particularly in the areas of decomposition and characterization of harmonizable processes, have been instrumental in developing this field.

 

 

 

Not many features left before arb4j is feature-complete enough for a 'release'

The arb4j library is coming along well... check it out on github

 

The Spectral Representation of Stationary Processes: Bridging Gelfand-Vilenkin and Wiener-Khinchin

Introduction

At the heart of stochastic process theory lies a profound connection between time and frequency domains, elegantly captured by two fundamental theorems: the Gelfand-Vilenkin Spectral Representation Theorem and the Wiener-Khinchin Theorem. These results, while often presented separately, are intimately linked, offering complementary insights into the nature of stationary processes.

Gelfand-Vilenkin Theorem

The Gelfand-Vilenkin theorem provides a general, measure-theoretic framework for representing wide-sense stationary processes. Consider a stochastic process $\{X(t) : t \in \mathbb{R}\}$ on a probability space $(\Omega, \mathcal{F}, P)$. The theorem states that we can represent $X(t)$ as:

$$X(t) = \int_{\mathbb{R}} e^{i\omega t} dZ(\omega)$$

Here, $Z(\omega)$ is a complex-valued process with orthogonal increments, and the integral is taken over the real line. This representation expresses the process as a superposition of complex exponentials, each contributing to the overall behavior of $X(t)$ at different frequencies.

The key to understanding this representation lies in the spectral measure $\mu$, which is defined by $E[|Z(A)|^2] = \mu(A)$ for Borel sets $A$. This measure encapsulates the distribution of "energy" across different frequencies in the process.

Wiener-Khinchin Theorem

The Wiener-Khinchin theorem, in its classical form, states that for a wide-sense stationary process, the power spectral density $S(\omega)$ is the Fourier transform of the autocorrelation function:

$$S(\omega) = \int_{\mathbb{R}} R(\tau) e^{-i\omega\tau} d\tau$$

Bridging the Theorems

The connection becomes clear when we recognize that the spectral measure $\mu$ from Gelfand-Vilenkin is related to the power spectral density $S(\omega)$ from Wiener-Khinchin by:

$$d\mu(\omega) = \frac{1}{2\pi} S(\omega) d\omega$$

This relationship holds when $S(\omega)$ exists as a well-defined function. However, the beauty of the Gelfand-Vilenkin approach is that it allows for spectral measures that may not have a density, accommodating processes with more complex spectral structures.

Spectral Density Example

To illustrate the connection between spectral properties and sample path behavior, consider a process with a spectral density of the form:

$$S(\omega) = \frac{1}{\sqrt{1 - \omega^2}}, \quad |\omega| < 1$$

This density has singularities at $\omega = \pm 1$, which profoundly influence the behavior of the process in the time domain:

- The sample paths will be continuous and infinitely differentiable.

- The paths will exhibit rapid oscillations, reflecting the strong presence of frequencies near $\pm 1$.

- The process will show a mix of components with different periods, with those corresponding to $|\omega|$ near 1 having larger amplitudes on average.

- The autocorrelation function is $R(\tau) = J_0(\tau)$, where $J_0$ is the Bessel function of the first kind of order zero.

Frequency Interpretation

In our spectral density $S(\omega) = 1 / \sqrt{1 - \omega^2}$ with $|\omega| < 1$:

- $\omega$ represents angular frequency, with $|\omega|$ closer to 0 corresponding to longer-period components in the process.

- $|\omega|$ closer to 1 corresponds to shorter-period components.

- As $|\omega|$ approaches 1, $S(\omega)$ increases sharply, approaching infinity.

- This means components with $|\omega|$ near 1 contribute more strongly to the process variance.

Dirac Delta Example

Consider a spectral measure that is a Dirac delta function at $\omega = 0.25$:

$$S(\omega) = \delta(\omega - 0.25) + \delta(\omega + 0.25)$$

In this case:

- The process can be written as: $X(t) = A \cos(0.25t) + B \sin(0.25t)$

- The covariance function is $R(\tau) = \cos(0.25\tau)$

- The period of the covariance function is $2\pi/0.25 = 8\pi \approx 25.13$

- This illustrates that a frequency of 0.25 in the spectral domain corresponds to a period of $8\pi$ in the time domain

This example demonstrates the crucial relationship: for any peak or concentration of spectral mass at a frequency $\omega_0$, we'll see corresponding oscillations in the covariance function with period $2\pi/\omega_0$.