The representation of the process,
Xt
X
t
, is the sequence of random variables
X
i
X
i
. The choice basis of
φ
i
t
φ
i
t
is unrestricted. Of particular interest is to
restrict the basis functions to those which make the
X
i
X
i
uncorrelated random variables.
When this requirement is satisfied, the resulting representation
of
Xt
X
t
is termed the Karhunen-Loève
expansion. Mathematically, we require
∀i,j,i≠j:E
X
i
X
j
=E
X
i
E
X
j
i
j
i
j
X
i
X
j
X
i
X
j
. This requirement can be expressed in terms of the
correlation function of
Xt
X
t
.
E
X
i
X
j
=E∫0TXα
φ
i
αdα∫0TXβ
φ
j
βdβ=∫0T∫0T
φ
i
α
φ
j
β
R
X
αβdαdβ
X
i
X
j
α
0
T
X
α
φ
i
α
β
0
T
X
β
φ
j
β
β
0
T
α
0
T
φ
i
α
φ
j
β
R
X
α
β
As
E
X
i
X
i
is given by
E
X
i
=∫0T
m
X
α
φ
i
αdα
X
i
α
0
T
m
X
α
φ
i
α
our requirement becomes
∀i,j,i≠j:∫0T∫0T
φ
i
α
φ
j
β
R
X
αβdαdβ=∫0T
m
X
α
φ
i
αdα∫0T
m
X
β
φ
j
βdβ
i
j
i
j
β
0
T
α
0
T
φ
i
α
φ
j
β
R
X
α
β
α
0
T
m
X
α
φ
i
α
β
0
T
m
X
β
φ
j
β
(1)
Simple manipulations result in the expression
∀i,j,i≠j:∫0T
φ
i
α∫0T
K
X
αβ
φ
j
βdβdα=0
i
j
i
j
α
0
T
φ
i
α
β
0
T
K
X
α
β
φ
j
β
0
(2)
When
i=j
i
j
, the quantity
E
X
i
2−E
X
i
2
X
i
2
X
i
2
is just the variance of
X
i
X
i
. Our requirement is obtained by satisfying
∫0T
φ
i
α∫0T
K
X
αβ
φ
j
βdβdα=
λ
i
δ
i
j
α
0
T
φ
i
α
β
0
T
K
X
α
β
φ
j
β
λ
i
δ
i
j
or
∀i,j,i≠j:∫0T
φ
i
α
g
j
αdα=0
i
j
i
j
α
0
T
φ
i
α
g
j
α
0
(3)
where
g
j
α=∫0T
K
X
αβ
φ
j
βdβ
g
j
α
β
0
T
K
X
α
β
φ
j
β
Furthermore, this requirement must hold for each
jj which differs from the choice of
ii. A choice of a function
g
j
α
g
j
α
satisfying this requirement is a function which is
proportional to
φ
j
α
φ
j
α
:
g
j
α=
λ
j
φ
j
α
g
j
α
λ
j
φ
j
α
. Therefore,
∫0T
K
X
αβ
φ
j
βdβ=
λ
j
φ
j
α
β
0
T
K
X
α
β
φ
j
β
λ
j
φ
j
α
(4)
The
φ
i
φ
i
which allow the representation of
Xt
X
t
to be a sequence of uncorrelated random variables must
satisfy this integral equation. This type of equation occurs
often in applied mathematics; it is termed the
eigenequation. The sequences
φ
i
φ
i
and
λ
i
λ
i
are the eigenfunctions and eigenvalues of
K
X
αβ
K
X
α
β
, the covariance function of
Xt
X
t
. It is easily verified that:
K
X
tu=∑i=1∞
λ
i
φ
i
t
φ
i
u
K
X
t
u
i
1
λ
i
φ
i
t
φ
i
u
This result is termed
Mercer's Theorem.
The approach to solving for the eigenfunction and eigenvalues of
K
X
tu
K
X
t
u
is to convert the integral equation into an ordinary
differential equation which can be solved. This approach is
best illustrated by an example.
K
X
tu=σ2min{tu}
K
X
t
u
σ
2
t
u
. The eigenequation can be written in this case as
σ2∫0tuφudu+t∫tTφudu=λφt
σ
2
u
0
t
u
φ
u
t
u
t
T
φ
u
λ
φ
t
Evaluating the first derivative of this expression,
σ2tφt+σ2∫tTφudu−σ2tφt=λddtφt
σ
2
t
φ
t
σ
2
u
t
T
φ
u
σ
2
t
φ
t
λ
t
φ
t
or
σ2∫tTφudu=λddtφ
σ
2
u
t
T
φ
u
λ
t
φ
Evaluating the derivative of the last expression yields the
simple equation
-σ2φt=λd2dt2φ
σ
2
φ
t
λ
2
t
φ
This equation has a general solution of the form
φt=Asinσλt+Bcosσλt
φ
t
A
σ
λ
t
B
σ
λ
t
. It is easily seen that
BB must be zero. The amplitude
AA is found by requiring
∥φ∥=1
φ
1
. To find λλ, one
must return to the original integral equation. Substituting, we
have
σ2A∫0tusinσλudu+σ2tA∫tTsinσλudu=λAsinσλt
σ
2
A
u
0
t
u
σ
λ
u
σ
2
t
A
u
t
T
σ
λ
u
λ
A
σ
λ
t
After some manipulation, we find that
∀t,t∈0T:Aλsinσλt−AσtλcosσλT=λAsinσλt
t
t
0
T
A
λ
σ
λ
t
A
σ
t
λ
σ
λ
T
λ
A
σ
λ
t
or
∀t,t∈0T:AσtλcosσλT=0
t
t
0
T
A
σ
t
λ
σ
λ
T
0
Therefore,
∀n,n=12…:σλT=n−1/2π
n
n
1
2
…
σ
λ
T
n
1/2
and we have
λ
n
=σ2T2n+1/22π2
λ
n
σ
2
T
2
n
1/2
2
2
φ
n
t=2T1/2sinn+1/2πtT
φ
n
t
2
T
1/2
n
1/2
t
T
The Karhunen-Loève expansion has several important properties.
- The eigenfunctions of a positive-definite covariance
function constitute a complete set. One can easily show that
these eigenfunctions are also mutually orthogonal with respect
to both the usual inner product and with respect to the inner
product derived from the covariance function.
- If
Xt
X
t
Gaussian, X
i X
i are Gaussian
random variables. As the random variables
X
i
X
i
are uncorrelated and Gaussian, the
X
i
X
i
comprise a sequence of statistically independent
random variables.
- Assume
K
X
tu=
N
0
2δt−u
K
X
t
u
N
0
2
δ
t
u
: the stochastic process
Xt
X
t
is white. Then
∫
N
0
2δt−uφudu=λφt
u
N
0
2
δ
t
u
φ
u
λ
φ
t
for all
φt
φ
t
. Consequently, if
λ
i
=
N
0
2
λ
i
N
0
2
, this constraint equation is satisfied no matter what choice is made for the orthonormal set
φ
i
t
φ
i
t
. Therefore, the representation of white, Gaussian
processes consists of a sequence of statistically
independent, identically-distributed (mean zero and variance
N
0
2
N
0
2
) Gaussian random variables. This example
constitutes the simplest case of the Karhunen-Loève
expansion.
