In this section, our linear systems will be n×n matrices
of complex numbers. For a little background into some of the
concepts that this module is based on, refer to the basics of
linear algebra.
Let
A
A be an n×n matrix, where
A
A is a linear operator on vectors in
ℂn
n
.
Ax=b
A
x
b
(1)
where
x
x and
b
b
are n×1 vectors (
Figure 1).
- Definition 1:
eigenvector
An eigenvector of
A
A is a vector
v∈ℂn
v
n
such that
Av=λv
A
v
λ
v
(2)
where
λ λ is called the
corresponding
eigenvalue.
A
A only changes the
length of
v
v, not its direction.
Through Figure 2 and Figure 3,
let us look at the difference between Equation 1 and Equation 2.
If
v
v is an eigenvector of
A A, then only its length changes.
See Figure 3 and notice how our vector's
length is simply scaled by our variable,
λλ, called the
eigenvalue:
When dealing with a matrix
A A, eigenvectors are the
simplest possible vectors to operate
on.
From inspection and understanding of eigenvectors, find
the two eigenvectors,
v1
v
1
and
v2
v
2
,
of
A=
300-1
A
3
0
0
-1
Also, what are the corresponding eigenvalues,
λ
1
λ
1
and
λ
2
λ
2
? Do not worry if you are having problems
seeing these values from the information given so far,
we will look at more rigorous ways to find these values
soon.
The eigenvectors you found should be:
v1=10
v
1
1
0
v2=01
v
2
0
1
And the corresponding eigenvalues are
λ
1
=3
λ
1
3
λ
2
=-1
λ
2
-1
Show that these two vectors,
v1=11
v
1
1
1
v2=1-1
v
2
1
-1
are eigenvectors of
A
A, where
A=3-1-13
A
3
-1
-1
3
. Also, find the corresponding eigenvalues.
In order to prove that these two vectors are
eigenvectors, we will show that these statements meet
the requirements stated in the definition.
Av1=
3-1-13
11
=
22
A
v
1
3
-1
-1
3
1
1
2
2
Av2=
3-1-13
1-1
=
4-4
A
v
2
3
-1
-1
3
1
-1
4
-4
These results show us that
A
A
only scales the two vectors (i.e.
changes their length) and thus it proves that Equation 2 holds true for the following
two eigenvalues that you were asked to find:
λ
1
=2
λ
1
2
λ
2
=4
λ
2
4
If you need more convincing, then one could also easily
graph the vectors and their corresponding product with
AA to see that the results
are merely scaled versions of our original vectors,
v1
v
1
and
v2
v
2
.
In the above examples, we relied on your understanding of the
definition and on some basic observations to find and prove the
values of the eigenvectors and eigenvalues. However, as you
can probably tell, finding these values will not always be
that easy. Below, we walk through a rigorous and mathematical
approach at calculating the eigenvalues and eigenvectors of a
matrix.
Find
λ∈ℂ
λ
such that
v≠0
v
0
,
where
0
0 is the "zero vector." We will start with Equation 2, and then work our way down
until we find a way to explicitly calculate
λ
λ.
Av=λv
A
v
λ
v
Av-λv=0
A
v
λ
v
0
A-λIv=0
A
λ
I
v
0
In the previous step, we used the fact that
λv=λIv
λ
v
λ
I
v
where II is the identity
matrix.
I=
10…001…000⋱⋮0……1
I
1
0
…
0
0
1
…
0
0
0
⋱
⋮
0
…
…
1
So,
A-λI
A
λ
I
is just a new matrix.
Given the following matrix,
AA, then we can find our new
matrix,
A-λI
A
λ
I
.
A=
a11a12a21a22
A
a
1
1
a
1
2
a
2
1
a
2
2
A-λI=
a11-λa12a21a22-λ
A
λ
I
a
1
1
λ
a
1
2
a
2
1
a
2
2
λ
If
A-λIv=0
A
λ
I
v
0
for some
v≠0
v
0
, then
A-λI
A
λ
I
is not invertible. This
means:
detA-λI=0
A
λ
I
0
This determinant (shown directly above) turns out to be a
polynomial expression (of order
nn). Look at the examples
below to see what this means.
Starting with matrix AA
(shown below), we will find the polynomial expression,
where our eigenvalues will be the dependent variable.
A=
3-1-13
A
3
-1
-1
3
A-λI=
3-λ-1-13-λ
A
λ
I
3
λ
-1
-1
3
λ
detA-λI=3-λ2-
-1
2=λ2-6λ+8
A
λ
I
3
λ
2
-1
2
λ
2
6
λ
8
λ=
24
λ
2
4
Starting with matrix AA
(shown below), we will find the polynomial expression,
where our eigenvalues will be the dependent variable.
A=
a11a12a21a22
A
a
1
1
a
1
2
a
2
1
a
2
2
A-λI=
a11-λa12a21a22-λ
A
λ
I
a
1
1
λ
a
1
2
a
2
1
a
2
2
λ
detA-λI=λ2-a11+a22λ-a21a12+a11a22
A
λ
I
λ
2
a
1
1
a
2
2
λ
a
2
1
a
1
2
a
1
1
a
2
2
If you have not already noticed it, calculating the
eigenvalues is equivalent to calculating the roots of
detA-λI=
c
n
λn+
c
n
−
1
λn-1+…+
c
1
λ+
c
0
=0
A
λ
I
c
n
λ
n
c
n
−
1
λ
n
1
…
c
1
λ
c
0
0
Therefore, by simply using calculus to solve for the roots
of our polynomial we can easily find the eigenvalues of our
matrix.
Given an eigenvalue,
λ
i
λ
i
, the associated eigenvectors are given by
Av=
λ
i
v
A
v
λ
i
v
Av1⋮vn=
λ
1
v1⋮
λ
n
vn
A
v
1
⋮
v
n
λ
1
v
1
⋮
λ
n
v
n
set of nn equations with
nn unknowns. Simply
solve the nn
equations to find the eigenvectors.
Say the eigenvectors of AA,
v1v2…vn
v
1
v
2
…
v
n
,
span
ℂn
n
, meaning
v1v2…vn
v
1
v
2
…
v
n
are linearly independent and we can write any
x∈ℂn
x
n
as
x=
α
1
v1+
α
2
v2+…+
α
n
vn
x
α
1
v
1
α
2
v
2
…
α
n
v
n
(3)
where
α1α2…αn
∈ℂ
α
1
α
2
…
α
n
. All that we are doing is rewriting
xx in terms of eigenvectors of
AA. Then,
Ax=A
α
1
v1+
α
2
v2+…+
α
n
vn
A
x
A
α
1
v
1
α
2
v
2
…
α
n
v
n
Ax=
α
1
Av1+
α
2
Av2+…+
α
n
Avn
A
x
α
1
A
v
1
α
2
A
v
2
…
α
n
A
v
n
Ax=
α
1
λ
1
v1+
α
2
λ
2
v2+…+
α
n
λ
n
vn=b
A
x
α
1
λ
1
v
1
α
2
λ
2
v
2
…
α
n
λ
n
v
n
b
Therefore we can write,
x=∑i
α
i
vi
x
i
α
i
v
i
and this leads us to the following depicted system:
where in Figure 4 we have,
b=∑i
α
i
λ
i
vi
b
i
α
i
λ
i
v
i
By breaking up a vector, xx, into a combination of
eigenvectors, the calculation of
Ax
A
x
is broken into "easy to swallow" pieces.
For the following matrix, AA and
vector, xx, solve
for their product. Try solving it using two different
methods: directly and using eigenvectors.
A=
3-1-13
A
3
-1
-1
3
x=
53
x
5
3
Direct Method (use basic matrix
multiplication)
Ax=
3-1-13
53
=
124
A
x
3
-1
-1
3
5
3
12
4
Eigenvectors (use the eigenvectors
and eigenvalues we found earlier for this same matrix)
v1=11
v
1
1
1
v2=1-1
v
2
1
-1
λ
1
=2
λ
1
2
λ
2
=4
λ
2
4
As shown in Equation 3, we want to
represent xx as
a sum of its scaled eigenvectors. For this case, we have:
x=4v1+v2
x
4
v
1
v
2
x=
53
=4
11
+
1-1
x
5
3
4
1
1
1
-1
Ax=A4v1+v2=
λ
i
4v1+v2
A
x
A
4
v
1
v
2
λ
i
4
v
1
v
2
Therefore, we have
Ax=4×2
11
+4
1-1
=
124
A
x
4
2
1
1
4
1
-1
12
4
Notice that this method using eigenvectors required
no matrix multiplication. This may
have seemed more complicated here, but just imagine
AA being really big, or even
just a few dimensions larger!
"My introduction to signal processing course at Rice University."