This brief tutorial on some key terms in linear algebra is not
meant to replace or be very helpful to those of you trying to
gain a deep insight into linear algebra. Rather, this brief
introduction to some of the terms and ideas of linear algebra is
meant to provide a little background to those trying to get a
better understanding or learn about eigenvectors and
eigenfunctions, which play a big role in deriving a few
important ideas on Signals and Systems. The goal of these
concepts will be to provide a background for signal
decomposition and to lead up to the derivation of the Fourier Series.
A set of vectors
∀
x
,
x
i
∈Cn:
x
1
x
2
…
x
k
x
x
i
n
x
1
x
2
…
x
k
are linearly independent if none of
them can be written as a linear combination of the others.
- Definition 1: Linearly Independent
For a given set of vectors,
x
1
x
2
…
x
n
x
1
x
2
…
x
n
, they are linearly independent if
c
1
x
1
+
c
2
x
2
+…+
c
n
x
n
=0
c
1
x
1
c
2
x
2
…
c
n
x
n
0
only when
c
1
=
c
2
=…=
c
n
=0
c
1
c
2
…
c
n
0
We are given the following two vectors:
x
1
=32
x
1
3
2
x
2
=-6-4
x
2
-6
-4
These are not linearly independent as
proven by the following statement, which, by inspection,
can be seen to not adhere to the definition of linear
independence stated above.
(
x
2
=-2
x
1
)⇒(2
x
1
+
x
2
=0)
x
2
-2
x
1
2
x
1
x
2
0
Another approach to reveal a vectors independence is by
graphing the vectors. Looking at these two vectors
geometrically (as in Figure 1), one can again
prove that these vectors are not
linearly independent.
We are given the following two vectors:
x
1
=32
x
1
3
2
x
2
=12
x
2
1
2
These are linearly independent since
c
1
x
1
=−(
c
2
x
2
)
c
1
x
1
c
2
x
2
only if
c
1
=
c
2
=0
c
1
c
2
0
. Based on the definition, this proof shows that
these vectors are indeed linearly independent. Again, we
could also graph these two vectors (see Figure 2) to check for linear independence.
Are
x
1
x
2
x
3
x
1
x
2
x
3
linearly independent?
x
1
=32
x
1
3
2
x
2
=12
x
2
1
2
x
3
=-10
x
3
-1
0
By playing around with the vectors and doing a little
trial and error, we will discover the following
relationship:
x
1
−
x
2
+2
x
3
=0
x
1
x
2
2
x
3
0
Thus we have found a linear combination of these three
vectors that equals zero without setting the coefficients
equal to zero. Therefore, these vectors are not
linearly independent!
As we have seen in the two above examples, often times the
independence of vectors can be easily seen through a graph.
However this may not be as easy when we are given three or
more vectors. Can you easily tell whether or not these
vectors are independent from Figure 3. Probably not,
which is why the method used in the above solution becomes
important.
A set of mm vectors in
Cn
n
cannot be linearly independent if
m>n
m
n
.
- Definition 2: Span
The
span of a set of vectors
x
1
x
2
…
x
k
x
1
x
2
…
x
k
is the set of vectors that can be written as a linear
combination of
x
1
x
2
…
x
k
x
1
x
2
…
x
k
span
x
1
…
x
k
=∀
α
,
α
i
∈Cn:
α
1
x
1
+
α
2
x
2
+…+
α
k
x
k
span
x
1
…
x
k
α
α
i
n
α
1
x
1
α
2
x
2
…
α
k
x
k
Given the vector
x
1
=32
x
1
3
2
the span of
x
1
x
1
is a line.
Given the vectors
x
1
=32
x
1
3
2
x
2
=12
x
2
1
2
the span of these vectors is
C2
2
.
- Definition 3: Basis
A basis for
Cn
n
is a set of vectors that: (1) spans
Cn
n
and (2) is linearly independent.
Clearly, any set of
nn linearly
independent vectors is a
basis for
Cn
n
.
We are given the following vector
e
i
=0⋮010⋮0
e
i
0
⋮
0
1
0
⋮
0
where the 11 is always in the
iith place and the remaining
values are zero. Then the basis for
Cn
n
is
∀
i
,i=12…n:
e
i
i
i
1
2
…
n
e
i
∀
i
,i=12…n:
e
i
i
i
1
2
…
n
e
i
is called the standard basis.
h
1
=11
h
1
1
1
h
2
=1-1
h
2
1
-1
h
1
h
2
h
1
h
2
is a basis for
C2
2
.
If
b
1
…
b
2
b
1
…
b
2
is a basis for
Cn
n
, then we can express any
x∈Cn
x
n
as a linear combination of the
b
i
b
i
's:
∀
α
,
α
i
∈C:x=
α
1
b
1
+
α
2
b
2
+…+
α
n
b
n
α
α
i
x
α
1
b
1
α
2
b
2
…
α
n
b
n
Given the following vector,
x=12
x
1
2
writing
x
x in terms of
e
1
e
2
e
1
e
2
gives us
x=
e
1
+2
e
2
x
e
1
2
e
2
Try and write
x
x
in terms of
h
1
h
2
h
1
h
2
(defined in the previous example).
x=32
h
1
+-12
h
2
x
3
2
h
1
-1
2
h
2
In the two basis examples above,
x
x is the same vector in both cases, but we can express
it in many different ways (we give only two out of many, many
possibilities). You can take this even further by extending
this idea of a basis to function spaces.
As mentioned in the introduction, these concepts of linear
algebra will help prepare you to understand the
Fourier Series, which
tells us that we can express periodic functions,
ft
f
t
,
in terms of their basis functions,
ei
ω
0
nt
ω
0
n
t
.
