We begin with the simple geometric interpretation of
matrix-vector multiplication. Namely, the multiplication of
the n-by-1 vector xx by the
m-by-n matrix
A
A produces a linear combination of the columns of
A
A. More precisely, if
a
j
a
j
denotes the
j
jth column of
A
A, then
Ax=a1a2…anx1x2…xn=x1a1+x2a2+…+xnan
A
x
a1
a2
…
an
x1
x2
…
xn
x1
a1
x2
a2
…
xn
an
(1)
The picture that I wish to place in your mind's eye is that
Ax
A
x
lies in the
subspace
spanned
by the columns of
A
A. This subspace occurs so frequently that
we find it useful to distinguish it with a definition.
- Definition 1:
Column Space
The column space of the m-by-n matrix S is
simply the span of the its columns,
i.e.
RaS≡
{Sx|x∈
R
n
}
Ra
S
S
x
x
R
n
.
This is a
subspace
of
ℜ
m
ℜ
m
.
The notation
Ra
Ra stands for range in this context.
Let us examine the matrix:
A=0100-10100001
A
0100
-1010
0001
(2)
The column space of this matrix is:
RaA={
x
1
0-10+
x
2
100+
x
3
010+
x
4
001|x∈
ℝ
4
}
Ra
A
x
ℝ
4
x
1
0
-1
0
x
2
1
0
0
x
3
0
1
0
x
4
0
0
1
(3)
As the third column is simply a multiple of the first,
we may write:
RaA={
x
1
010+
x
2
100+
x
3
001|x∈
ℝ
3
}
Ra
A
x
ℝ
3
x
1
0
1
0
x
2
1
0
0
x
3
0
0
1
(4)
As the three remaining columns are linearly independent we may
go no further. In this case,
RaA
Ra
A
comprises all of
ℝ
3
ℝ
3
.
To determine the basis for
RaA
Ra
A
(where
A A is an arbitrary matrix) we must
find a way to discard its dependent columns. In the example
above, it was easy to see that columns 1 and 3 were
colinear. We seek, of course, a more systematic means of
uncovering these, and perhaps other less obvious,
dependencies. Such dependencies are more easily discerned from
the row reduced
form. In the reduction of the above problem, we come
very easily to the matrix
A
red
=-101001000001
A
red
-1010
0100
0001
(5)
Once we have done this, we can recognize that the
pivot column are
the linearly
independent columns of
A red
A red
. One now asks how this might
help us distinguish the independent columns of
A
A. For, although the rows of
A
red
A
red
are linear combinations of the rows of
A A, no such thing is true with respect
to the columns. The answer is:
pay attention to the
indices of the pivot columns. In our example,
columns {1, 2, 4} are the pivot columns of
A
red
A
red
and hence the first, second, and fourth columns of
A
A,
i.e.,
0-10100001
0
-1
0
1
0
0
0
0
1
(6)
comprise a basis for
RaA
Ra
A
. In general:
- Definition 2:
A Basis for the Column Space
Suppose
A
A is m-by-n. If columns
{
c
j
|j=
1
,
...
,
r
}
j
1
,
...
,
r
c
j
are the pivot columns of
A
red
A
red
then columns
{
c
j
|j=
1
,
...
,
r
}
j
1
,
...
,
r
c
j
of
A
A constitute a basis for
RaA
Ra
A
.
