From our understanding of eigenvalues and eigenvectors we have
discovered several things about our operator matrix,
AA. We know that if the
eigenvectors of AA span
ℂn
n
and we know how to express any vector
xx in terms of
v1v2…vn
v
1
v
2
…
v
n
, then we have the operator
AA all figured out. If we have
AA acting on xx, then this is equal to
AA acting on the combinations of
eigenvectors. Which we know proves to be fairly easy!
We are still left with two questions that need to be
addressed:
-
When do the eigenvectors
v1v2…vn
v
1
v
2
…
v
n
of AA span
ℂn
n
(assuming
v1v2…vn
v
1
v
2
…
v
n
are linearly independent)?
-
How do we express a given vector xx in terms of
v1v2…vn
v
1
v
2
…
v
n
?
When do the eigenvectors
v1v2…vn
v
1
v
2
…
v
n
of AA span
ℂn
n
?
If
AA has
nn distinct
eigenvalues
∀i,i≠j:
λ
i
≠
λ
j
i
i
j
λ
i
λ
j
where
ii and
jj are integers, then
AA has
nn linearly independent
eigenvectors
v1v2…vn
v
1
v
2
…
v
n
which then span
ℂn
n
.
The proof of this statement is not very hard, but is not
really interesting enough to include here. If you wish to
research this idea further, read Strang, G., "Linear Algebra
and its Application" for the proof.
Furthermore,
nn distinct
eigenvalues means
detA−λI=
c
n
λn+
c
n
−
1
λn−1+…+
c
1
λ+
c
0
=0
A
λ
I
c
n
λ
n
c
n
−
1
λ
n
1
…
c
1
λ
c
0
0
has
nn distinct roots.
How do we express a given vector xx in terms of
v1v2…vn
v
1
v
2
…
v
n
?
We want to find
α
1
α
2
…
α
n
∈ℂ
α
1
α
2
…
α
n
such that
x=
α
1
v1+
α
2
v2+…+
α
n
vn
x
α
1
v
1
α
2
v
2
…
α
n
v
n
(1)
In order to find this set of variables, we will begin by
collecting the vectors
v1v2…vn
v
1
v
2
…
v
n
as columns in a n×n matrix
V
V.
V=
⋮⋮ ⋮v1v2…vn⋮⋮ ⋮
V
⋮
⋮
⋮
v
1
v
2
…
v
n
⋮
⋮
⋮
Now
Equation 1 becomes
x=
⋮⋮ ⋮v1v2…vn⋮⋮ ⋮
α
1
⋮
α
n
x
⋮
⋮
⋮
v
1
v
2
…
v
n
⋮
⋮
⋮
α
1
⋮
α
n
or
x=Vα
x
V
α
which gives us an easy form to solve for our variables in
question,
αα:
α=V-1x
α
V
-1
x
Note that
VV is invertible since
it has
nn linearly independent
columns.
Let us recall our knowledge of functions and there basis and
examine the role of VV.
x=Vα
x
V
α
x
1
⋮
x
n
=V
α
1
⋮
α
n
x
1
⋮
x
n
V
α
1
⋮
α
n
where αα is
just xx expressed
in a different basis:
x=
x
1
10⋮0+
x
2
01⋮0+…+
x
n
00⋮1
x
x
1
1
0
⋮
0
x
2
0
1
⋮
0
…
x
n
0
0
⋮
1
x=
α
1
⋮v1⋮+
α
2
⋮v2⋮+…+
α
n
⋮vn⋮
x
α
1
⋮
v
1
⋮
α
2
⋮
v
2
⋮
…
α
n
⋮
v
n
⋮
VV transforms
xx from the
standard basis to the basis
v1v2…vn
v
1
v
2
…
v
n
We can also use the vectors
v1v2…vn
v
1
v
2
…
v
n
to represent the output, b
b, of a system:
b=Ax=A
α
1
v1+
α
2
v2+…+
α
n
vn
b
A
x
A
α
1
v
1
α
2
v
2
…
α
n
v
n
Ax=
α
1
λ
1
v1+
α
2
λ
2
v2+…+
α
n
λ
n
vn=b
A
x
α
1
λ
1
v
1
α
2
λ
2
v
2
…
α
n
λ
n
v
n
b
Ax=
⋮⋮ ⋮v1v2…vn⋮⋮ ⋮
λ
1
α
1
⋮
λ
1
α
n
A
x
⋮
⋮
⋮
v
1
v
2
…
v
n
⋮
⋮
⋮
λ
1
α
1
⋮
λ
1
α
n
Ax=VΛα
A
x
V
Λ
α
Ax=VΛV-1x
A
x
V
Λ
V
-1
x
where ΛΛ is the matrix
with the eigenvalues down the diagonal:
Λ=
λ
1
0…00
λ
2
…0⋮⋮⋱⋮00…
λ
n
Λ
λ
1
0
…
0
0
λ
2
…
0
⋮
⋮
⋱
⋮
0
0
…
λ
n
Finally, we can cancel out the x
x and are left with a final equation for
AA:
A=VΛV-1
A
V
Λ
V
-1
For our interpretation, recall our key formulas:
α=V-1x
α
V
-1
x
b=∑i
α
i
λ
i
vi
b
i
α
i
λ
i
v
i
We can interpret operating on xx with
AA as:
x
1
⋮
x
n
→
α
1
⋮
α
n
→
λ
1
α
1
⋮
λ
1
α
n
→
b
1
⋮
b
n
x
1
⋮
x
n
α
1
⋮
α
n
λ
1
α
1
⋮
λ
1
α
n
b
1
⋮
b
n
where the three steps (arrows) in the above illustration represent
the following three operations:
-
Transform x x
using
V-1
V
-1
, which yields
αα
-
Multiplication by Λ
Λ
-
Inverse transform using V
V, which gives us bb
"My introduction to signal processing course at Rice University."