This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.
We are now equipped to set up systems of linear equations as matrix-
vector equations so that they can be solved in a standard way on a computer.
Suppose we want to solve the equations from Example 1 from "Linear Algebra: Introduction" for
x
1
x
1
and
x
2
x
2
using
a computer. Recall that Equations 1 and 2 from Linear Algebra: Introduction are
x
1
+
x
2
=
85
x
1
1
.
2
=
x
2
1
.
5
-
1
.
2
x
1
+
x
2
=
85
x
1
1
.
2
=
x
2
1
.
5
-
1
.
2
(1)The first step is to arrange each equation with all references to
x
1
x
1
in the
first column, all references to
x
2
x
2
in the second column, etc., and all constants
on the right-hand side:
x
1
+
x
2
=
85
0
.
3
x
1
-
1
.
2
x
2
=
0
.
x
1
+
x
2
=
85
0
.
3
x
1
-
1
.
2
x
2
=
0
.
(2)Then the equations can be converted to a single matrix-vector equation. The coefficients form a matrix, keeping their same relative positions, and the variables and constants each form a vector:
1
1
0
.
3
-
1
.
2
x
1
x
2
=
85
0
.
1
1
0
.
3
-
1
.
2
x
1
x
2
=
85
0
.
(3)
Verify by the rules of matrix multiplication that the system of
equations in 2 in equivalent to the matrix equation in 3.
Equation 3 is of the general form
where in this case
A=110.3-1.2 , x=xlx2,b=850.A=110.3-1.2 , x=xlx2,b=850.
(5)Given any
A ∈Rn×nA∈Rn×n and b∈Rnb∈Rn, MATLAB can solve Equation 4 for
x
x (as long as a solution exists). Key ideas in the solution process are the
identity matrix and the inverse of a matrix.
When the matrix A is the 1×11×1 matrix
a
a, the vector
x
x is the l-vector
x
x,
and the vector
b
b is the l-vector
b
b, then the matrix equation
A
x
=bAx=b becomes
the scalar equation
The scalar a-1a-1 is the inverse of the scalar
a
a, so we may multiply on both sides
of Equation 6 to obtain the result
a
-
1
(
a
x
)
=
a
-
1
b
1
x
=
a
-
1
b
.
a
-
1
(
a
x
)
=
a
-
1
b
1
x
=
a
-
1
b
.
(7)We would like to generalize this simple idea to the matrix equation Ax=bAx=b
so that we can find an inverse of the matrix
A
A, called A-1A-1, and write
A-1(
A
x
)
=A-1b
I
x
=
A
-
1
b
.
A-1(
A
x
)
=A-1b
I
x
=
A
-
1
b
.
(8)In this equation the matrix
I
I is the identity matrix
I=
1
0
0
...
0
0
1
0
...
0
0
0
1
...
0
⋮
⋮
⋮
⋱
⋮
0
0
0
...
1
.
I=
1
0
0
...
0
0
1
0
...
0
0
0
1
...
0
⋮
⋮
⋮
⋱
⋮
0
0
0
...
1
.
(9)It is clear that the identity matrix
I
I and the inverse of a matrix, A-1A-1, play a
key role in the solution of linear equations. Let's study them in more detail.
The Matrix Identity. When we multiply a scalar by 1, we get back
that same scalar. For this reason, the number 1 is called the multiplicativeidentity element. This may seem trivial, but the generalization to matrices is
more interesting. The question is, is there a matrix that, when it multiplies
another matrix, does not change the other matrix? The answer is yes. The
matrix is called the identity matrix and is denoted by I
I. The identity matrix is
always square, with whatever size is appropriate for the matrix multiplication.
The identity matrix has l's on its main diagonal and 0's0's everywhere else. For
example,
I
5
=
[
1
0
0
0
0
0
1
0
0
0
0
0
1
0
0
0
0
0
1
0
0
0
0
0
1
]
.
I
5
=[
1
0
0
0
0
0
1
0
0
0
0
0
1
0
0
0
0
0
1
0
0
0
0
0
1
].
(10)The subscript 5 indicates the size. In terms of the unit coordinate vectors
e
i
e
i
,
we can also write the n×nn×n identity matrix as
I
n
=
|
|
|
e
1
e
2
...
e
n
|
|
|
.
I
n
=
|
|
|
e
1
e
2
...
e
n
|
|
|
.
(11)For any matrix A∈Rn×pA∈Rn×p, we have
For p=1p=1, we obtain the following special form for any vector x∈Rnx∈Rn :
This last equation can be written as the sum
x
=
∑
i
=
1
n
x
i
e
i
.
x
=
∑
i
=
1
n
x
i
e
i
.
(14)This is a special case of one of the results you proved in Exercise 3 from "Linear Algebra: Matrices". Figure 1 illustrates Equation 14 and shows how the vector
x
x can be broken into component vectors xieixiei, each of which lies in the direction of one coordinate axis. The values of the xi'sxi's are the coordinates of
xx, and their magnitudes are also the lengths of the component vectors.
Use Equation 13 and the rules for matrix multiplication to
show that x∈Rnx∈Rn may also be written as
x
=
∑
i
=
1
n
(
x
,
e
i
)
e
i
.
x
=
∑
i
=
1
n
(
x
,
e
i
)
e
i
.
(15) This verifies Equation 11 from "Linear Algebra: Direction Cosines".
When the product of two numbers is 1 (the
identity element), we say that they are inverses of each other, like 2 and 0.5.
Likewise, we say that two square matrices are inverses of each other if their
product is the identity matrix:
A
B
=
I
⇔
B
=
A
-
1
.
A
B
=
I
⇔
B
=
A
-
1
.
(16)An interesting and useful result is implied by this definition. Take the first
form of Equation 16 and multiply by
B
B from the left:
A
B
=
I
⇒
B
(
A
B
)
=
B
I
⇒
(
B
A
)
B
=
B
⇒
B
A
=
I
.
A
B
=
I
⇒
B
(
A
B
)
=
B
I
⇒
(
B
A
)
B
=
B
⇒
B
A
=
I
.
(17)We emphasize that the inference made in the last step here is only valid when
B
B and
A
A are square matrices. The result means that, even though matrix
multiplication is not commutative in general, we have a special case where it
is commutative. If
A
A and
B
B are inverses of each other, then
A
B
=
B
A
=
I
.
A
B
=
B
A
=
I
.
(18)
Prove that the inverse of the 2 ×2×2 rotation matrix R(θ)R(θ) is
RT(θ)RT(θ).
Matrices that are not square do not have inverses. In fact, not all
square matrices have inverses. So it becomes an important issue to determine
which matrices do have inverses. If a matrix has an inverse, the inverse is
unique. If a square matrix has no inverse, it is called a singular matrix. The
determinant of a square matrix is a scalar computed from the numbers in the
matrix. It tells us whether a matrix will have an inverse or not. A matrix
is singular if and only if its determinant is zero. In MATLAB, the notation
det(A) is used to compute the determinant. Whenever the matrix A in Equation 4 is singular, it means one of two things about the system of equations represented: either the equations are inconsistent and there is no solution, or the equations are dependent and there are infinitely many solutions.
Solving
A
x
=
b
A
x
=
b. Let's now study ways to solve the matrix equation
A
x=bAx=b. We will assume that a unique solution for
x
x exists. Thus a unique
matrix A-1A-1 exists with the property A-1A=IA-1A=I. The trick is to find it. Here
is one procedure.
For convenience, rewrite the matrix equation Ax =b=b as
[Ab]x-1=0.[Ab]x-1=0.
(19)The matrix
[
Ab
]∈Rnx(n+1)[Ab]∈Rnx(n+1) is called the augmented matrix for the system
of equations. The augmented matrix may be viewed as a systematic way of
writing all the information necessary to solve the equations.
The advantage of Equation 19 is that we may premultiply both sides
by any matrix
C
1
C
1
(of compatible size), and the right-hand side remains zero
(although this is equivalent to multiplying on both sides of Equation 4,
which some may prefer). We can repeat this operation as often as we like
with matrices C2,C3C2,C3, etc. The general result is
[
C
m
⋯
C
2
C
1
A
C
m
⋯
C
2
C
1
b
]
x
-
1
=
0
.
[
C
m
⋯
C
2
C
1
A
C
m
⋯
C
2
C
1
b
]
x
-
1
=
0
.
(20)Now suppose that we have found a sequence of matrices C1,...C1,... ,
C
m
C
m
that transforms the matrix
A
A to the identity matrix:
Cm⋯C2C1A=I.Cm⋯C2C1A=I.
The augmented matrix equation in 20 simplifies to
[
I
Cm⋯C2C1b] x-1=0,[ICm⋯C2C1b]x-1=0,
(21)which can be multiplied out to reveal that the solution for x is
x
=
C
m
⋯
C
2
C
1
b
.
x
=
C
m
⋯
C
2
C
1
b
.
(22)The method may be summarized as follows:
- form the augmented matrix
[
A
b
]
[Ab] for the system;
- premultiply
[
A
b
]
[Ab] by a sequence of selected matrices
C
i
C
i
, designed to take
A
A to
I
I; and
- when
A
A is transformed to
I
I,
b
b will be transformed to
x
x, so the solution will appear as the last column of the transformed augmented matrix.
We may also conclude that the product of the matrices Ci must be the inverse
of A since A-1A-1 is the unique matrix for which A-1A=IA-1A=I. In solving for x
by this method, we have found A-1A-1 implicitly as the product Cm⋯C2C1Cm⋯C2C1.
Consider the equation
3
1
2
4
x
1
x
2
=
6
5
3
1
2
4
x
1
x
2
=
6
5
(23)The augmented matrix for this equation is
[
A
b
]
=
3
1
6
2
4
5
.
[
A
b
]
=
3
1
6
2
4
5
.
(25)Now if we could add -2/3-2/3 times the first row to the second row, we would
get 0 in the lower left corner. This is the first step in transforming A to the
identity I. We can accomplish this row operation with the matrix
C
1
=
1
0
-
2
/
3
1
C
1
=
1
0
-
2
/
3
1
(26)
C
1
[
A
b
]
=
3
1
6
0
10
/
3
1
.
C
1
[
A
b
]
=
3
1
6
0
10
/
3
1
.
(27)Now adding -3/10-3/10 times the new second row to the first row will introduce
0 in the (1,2)(1,2) position, bringing us closer still to the identity. Thus
C
2
=
1
-
3
/
10
0
1
C
2
=
1
-
3
/
10
0
1
(28)
C
2
C
1
[
A
b
]
=
3
0
57
/
10
0
10
/
3
1
.
C
2
C
1
[
A
b
]
=
3
0
57
/
10
0
10
/
3
1
.
(29)We now complete the transformation to identity by normalizing each row to
get the needed l's:
C
3
=
1
/
3
0
0
3
/
10
C
3
=
1
/
3
0
0
3
/
10
(30)
C
3
C
2
C
1
[
A
b
]
=
1
0
19
/
10
0
1
3
/
10
.
C
3
C
2
C
1
[
A
b
]
=
1
0
19
/
10
0
1
3
/
10
.
(31)According to the last column, the solution is
x
=
19
/
10
3
/
10
.
x
=
19
/
10
3
/
10
.
(32)We note in passing that the inverse of
A
A is the product of the C'sC's, so
A
-
1
=
C
3
C
2
C
1
=
1
/
3
0
0
3
/
10
1
-
3
/
10
0
1
1
0
-
2
/
3
1
=
1
/
3
-
1
/
10
0
3
/
l
0
1
0
-
2
/
3
1
=
2
/
5
-
1
/
10
-
1
/
5
3
/
10
·
A
-
1
=
C
3
C
2
C
1
=
1
/
3
0
0
3
/
10
1
-
3
/
10
0
1
1
0
-
2
/
3
1
=
1
/
3
-
1
/
10
0
3
/
l
0
1
0
-
2
/
3
1
=
2
/
5
-
1
/
10
-
1
/
5
3
/
10
·
The method we have just used, combined with a particular way of
choosing the
C
i
C
i
matrices, is called Gauss elimination. Gauss elimination
requires less computation than finding the inverse of A because only the
effect of A-1A-1 on the specific vector
b
b is computed. MATLAB can solve
for
x
x by either method, as shown in Demo 4.2. For hand computations, we
suggest choosing the
C
i
C
i
matrices so that
C
1
C
1
produces O'sO's everywhere below
the diagonal in the first column,
C
2
C
2
produces O'sO's below the diagonal in the
second column, and so on up to Cn-1Cn-1. Then
C
n
C
n
produces O'sO's above the
diagonal in the nthnth column, Cn+1Cn+1 produces O'sO's above the diagonal in column
n-1n-1, etc. The last one, C2n-1C2n-1, normalizes the diagonal elements to l's. We
have assumed for simplicity that no O'sO's on the diagonal will be encountered
in hand computations.
Demo 1 (MATLAB). From the command level of MATLAB, solve the
matrix equation of Example 1 from "Linear Algebra: Introduction" by typing
>> A = [1 1;0.3 -1.2]
>> b = [85;0]
You have entered the matrices A and b, which describe the problem. You
can now solve for x by finding the inverse of A and multiplying b:
In this example the inverse is computed quickly because A is a small matrix.
For a large (say, 30×3030×30) matrix, the answer would take longer to compute,
and the more efficient method of Gauss elimination would reduce waiting time.
You can use Gauss elimination in MATLAB by typing
You should get the same answer. Now type the MATLAB code required to compute AxAx and to show Ax-b=0.Ax-b=0.
(MATLAB) Write the following system of equations as Ax=bAx=b
and solve using MATLAB:
3
(
x
1
-
x
3
)
+
2
(
x
2
-
1
)
-
6
=
x
3
3
(
x
1
-
x
3
)
+
2
(
x
2
-
1
)
-
6
=
x
3
(33)
4
x
3
=
7
x
2
-
5
4
x
3
=
7
x
2
-
5
(34)
6
(
x
1
+
x
2
+
2
)
=
x
3
+
1
.
6
(
x
1
+
x
2
+
2
)
=
x
3
+
1
.
(35)
(What does "Endorsed by" mean?)
"Reviewer's Comments: 'I recommend this book as a "required primary textbook." This text attempts to lower the barriers for students that take courses such as Principles of Electrical Engineering, […]"