One indication that things are simpler when using the
spectral representation is
B100=∑j=1h
λ
j
100
P
j
B
100
j
1
h
λ
j
100
P
j
(4)
As this holds for all powers it even holds for power
series. As a result,
ⅇB=∑j=1hⅇ
λ
j
P
j
B
j
1
h
λ
j
P
j
It is also extremely useful in attempting to solve
Bx=b
B
x
b
for
x
x.
Replacing
B
B
by its spectral representation and
b
b
by
Ib
I
b
or, more to the point by
∑j
P
j
b
j
j
P
j
b
we find
∑j=1h
λ
j
P
j
x=∑j=1h
P
j
b
j
1
h
λ
j
P
j
x
j
1
h
P
j
b
Multiplying through by
P
1
P
1
gives
λ
1
P
1
x=
P
1
b
λ
1
P
1
x
P
1
b
or
P
1
x=
P
1
b
λ
1
P
1
x
P
1
b
λ
1
.
Multiplying through by the subsequent
P
j
P
j
's
gives
P
j
x=
P
j
b
λ
j
P
j
x
P
j
b
λ
j
.
Hence,
x=∑j=1h
P
j
x=∑j=1h1
λ
j
P
j
b
x
j
1
h
P
j
x
j
1
h
1
λ
j
P
j
b
(5)
We clearly run in to trouble when one of the eigenvalues
vanishes. This, of course, is to be expected. For a zero
eigenvalue indicates a nontrivial null space which
signifies dependencies in the columns of
B
B
and hence the lack of a unique solution to
Bx=b
B
x
b
.
Another way in which Equation 5
may be viewed is to note that, when
B
B
is symmetric,
this
previous equation
takes the form
zI−B-1=∑j=1h1z−
λ
j
P
j
z
I
B
j
1
h
1
z
λ
j
P
j
Now if
0
0
is not an eigenvalue we may set
z=0
z
0
in the above and arrive at
B-1=∑j=1h1
λ
j
P
j
B
j
1
h
1
λ
j
P
j
(6)
Hence, the solution to
Bx=b
B
x
b
is
x=B-1b=∑j=1h1
λ
j
P
j
b
x
B
b
j
1
h
1
λ
j
P
j
b
as in
Equation 5.
With
Equation 6
we have finally reached a point where we can begin to
define an inverse even for matrices with dependent
columns, i.e., with a zero eigenvalue. We simply exclude
the offending term in
Equation 6.
Supposing that
λ
h
=0
λ
h
0
we define the
pseudo-inverse of
B
B
to be
B
+
≡∑j=1h−11
λ
j
P
j
B
+
j
1
h
1
1
λ
j
P
j
Let us now see whether it is deserving of its name. More
precisely, when
b∈ℛB
b
ℛ
B
we would expect that
x=
B
+
b
x
B
+
b
indeed satisfies
Bx=b
B
x
b
.
Well
B
B
+
b=B∑j=1h−11
λ
j
P
j
b=∑j=1h−11
λ
j
B
P
j
b=∑j=1h−11
λ
j
λ
j
P
j
b=∑j=1h−1
P
j
b
B
B
+
b
B
j
1
h
1
1
λ
j
P
j
b
j
1
h
1
1
λ
j
B
P
j
b
j
1
h
1
1
λ
j
λ
j
P
j
b
j
1
h
1
P
j
b
It remains to argue that the latter sum really is
b
b.
We know that
∀b,b∈ℜB:b=∑j=1h
P
j
b
b
b
B
b
j
1
h
P
j
b
The latter informs us that
b⊥NBT
⊥
b
N
B
.
As
B=BT
B
B
,
we have, in fact, that
b⊥NB
⊥
b
N
B
.
As
P
h
P
h
is nothing but orthogonal projection onto
NB
N
B
it follows that
P
h
b=0
P
h
b
0
and so
B
B
+
b=b
B
B
+
b
b
,
that is,
x=
B
+
b
x
B
+
b
is a solution to
Bx=b
B
x
b
.
The representation
Equation 4 is
unarguably terse and in fact is often written out in terms
of individual eigenvectors. Let us see how this is
done. Note that if
x∈ℜ
P
1
x
P
1
then
x=
P
1
x
x
P
1
x
and so,
Bx=B
P
1
x=∑j=1h
λ
j
P
j
P
1
x=
λ
1
P
1
x=
λ
1
x
B
x
B
P
1
x
j
1
h
λ
j
P
j
P
1
x
λ
1
P
1
x
λ
1
x
i.e.,
x
x
is an eigenvector of
B
B
associated with
λ
1
λ
1
.
Similarly, every (nonzero) vector in
ℜ
P
j
P
j
is an eigenvector of
B
B
associated with
λ
j
λ
j
.
Next let us demonstrate that each element of
ℜ
P
j
P
j
is orthogonal to each element of
ℜ
P
k
P
k
when
j≠k
j
k
.
If
x∈ℜ
P
j
x
P
j
and
y∈ℜ
P
k
y
P
k
then
xTy=
P
j
xT
P
k
y=xT
P
j
P
k
y=0
x
y
P
j
x
P
k
y
x
P
j
P
k
y
0
With this we note that if
x
j
,
1
x
j
,
2
…
x
j
,
n
j
x
j
,
1
x
j
,
2
…
x
j
,
n
j
constitutes a basis for
ℜ
P
j
P
j
then in fact the union of such bases,
{
x
j
,
p
|1≤j≤h∧1≤p≤
n
j
}
x
j
,
p
1
j
h
1
p
n
j
forms a linearly independent set. Notice now that this
set has
∑j=1h
n
j
j
1
h
n
j
elements. That these dimensions indeed sum to the ambient
dimension,
n
n,
follows directly from the fact that the underlying
P
j
P
j
sum to the
n
n-by-
n
n
identity matrix. We have just proven
