# Connexions

You are here: Home » Content » A First Course in Electrical and Computer Engineering » Linear Algebra: Matrices

### Lenses

What is a lens?

#### Definition of a lens

##### Lenses

A lens is a custom view of the content in the repository. You can think of it as a fancy kind of list that will let you see content through the eyes of organizations and people you trust.

##### What is in a lens?

Lens makers point to materials (modules and collections), creating a guide that includes their own comments and descriptive tags about the content.

##### Who can create a lens?

Any individual member, a community, or a respected organization.

##### What are tags?

Tags are descriptors added by lens makers to help label content, attaching a vocabulary that is meaningful in the context of the lens.

#### Endorsed by (What does "Endorsed by" mean?)

This content has been endorsed by the organizations listed. Click each link for a list of all content endorsed by the organization.
• IEEE-SPS

This collection is included inLens: IEEE Signal Processing Society Lens
By: IEEE Signal Processing Society

Click the "IEEE-SPS" link to see all content they endorse.

• College Open Textbooks

This collection is included inLens: Community College Open Textbook Collaborative
By: CC Open Textbook Collaborative

"Reviewer's Comments: 'I recommend this book as a "required primary textbook." This text attempts to lower the barriers for students that take courses such as Principles of Electrical Engineering, […]"

Click the "College Open Textbooks" link to see all content they endorse.

Click the tag icon to display tags associated with this content.

#### Affiliated with (What does "Affiliated with" mean?)

This content is either by members of the organizations listed or about topics related to the organizations listed. Click each link to see a list of all content affiliated with the organization.
• Bookshare

This collection is included inLens: Bookshare's Lens
By: Bookshare - A Benetech Initiative

"Accessible versions of this collection are available at Bookshare. DAISY and BRF provided."

Click the "Bookshare" link to see all content affiliated with them.

• NSF Partnership

This collection is included inLens: NSF Partnership in Signal Processing
By: Sidney Burrus

Click the "NSF Partnership" link to see all content affiliated with them.

Click the tag icon to display tags associated with this content.

• Featured Content

This collection is included inLens: Connexions Featured Content
By: Connexions

"A First Course in Electrical and Computer Engineering provides readers with a comprehensive, introductory look at the world of electrical engineering. It was originally written by Louis Scharf […]"

Click the "Featured Content" link to see all content affiliated with them.

Click the tag icon to display tags associated with this content.

#### Also in these lenses

• UniqU content

This collection is included inLens: UniqU's lens
By: UniqU, LLC

Click the "UniqU content" link to see all content selected in this lens.

• Evowl

This collection is included inLens: Rice LMS's Lens
By: Rice LMS

"Language: en"

Click the "Evowl" link to see all content selected in this lens.

• Lens for Engineering

This module and collection are included inLens: Lens for Engineering
By: Sidney Burrus

Click the "Lens for Engineering" link to see all content selected in this lens.

### Recently Viewed

This feature requires Javascript to be enabled.

### Tags

(What is a tag?)

These tags come from the endorsement, affiliation, and other lenses that include this content.

Inside Collection:

Collection by: Louis Scharf. E-mail the author

# Linear Algebra: Matrices

Module by: Louis Scharf. E-mail the author

## Note:

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

The word matrix dates at least to the thirteenth century, when it was used to describe the rectangular copper tray, or matrix, that held individual leaden letters that were packed into the matrix to form a page of composed text. Each letter in the matrix, call it aijaij, occupied a unique position (ij)(ij) in the matrix. In modern day mathematical parlance, a matrix is a collection of numbers arranged in a two-dimensional array (a rectangle). We indicate a matrix with a boldfaced capital letter and the constituent elements with double subscripts for the row and column:

In this equation A A is an mxnmxn matrix, meaning that A A has m m horizontal rows and n n vertical columns. As an extension of the previously used notation, we write A R m x n A R m x n to show that A A is a matrix of size m x n m x n with a i j R a i j R . The scalar element a i j a i j is located in the matrix at the i t h i t h row and the j t h j t h column. For example, a 23 a 23 is located in the second row and the third column as illustrated in Figure 1.

The main diagonal of any matrix consists of the elements aiiaii. (The two subscripts are equal.) The main diagonal runs southeast from the top left corner of the matrix, but it does not end in the lower right corner unless the matrix is square (Rm×m)(Rm×m).

The transpose of a matrix A Rm×nARm×n is another matrix B B whose element in row j j and column i i is bji=aijbji=aij for 1im1im and 1jn1jn. We write B=ATB=AT to indicate that B B is the transpose of A A. In MATLAB, transpose is denoted by A'. A more intuitive way of describing the transpose operation is to say that it flips the matrix about its main diagonal so that rows become columns and columns become rows.

## Exercise 1

If ARm×nARm×n, then AT?_AT?_. Find the transpose of the matrix A=215479.A=215479.

Matrix Addition and Scalar Multiplication. Two matrices of the same size (in both dimensions) may be added or subtracted in the same way as vectors, by adding or subtracting the corresponding elements. The equation C=A±BC=A±B means that for each i and j,cij=aij±bijj,cij=aij±bij. Scalar multiplication of a matrix multiplies each element of the matrix by the scalar:

a X = a x 11 a x 12 a x 13 ... a x 1 n a x 21 a x 22 a x 23 ... a x 2 n a x 31 a x 32 a x 33 ... a x 3 n a x m l a x m 2 a x m 3 ... a x m n . a X = a x 11 a x 12 a x 13 ... a x 1 n a x 21 a x 22 a x 23 ... a x 2 n a x 31 a x 32 a x 33 ... a x 3 n a x m l a x m 2 a x m 3 ... a x m n .
(1)

Matrix Multiplication. A vector can be considered a matrix with only one column. Thus we intentionally blur the distinction between Rn×1Rn×1 and Rn. Also a matrix can be viewed as a collection of vectors, each column of the matrix being a vector:

A = | | | a l a 2 ... a n | | | = [ a 11 a 21 | a m 1 a l 2 a 22 | a m 2 a 1 n a 2 n | a m n ] . A = | | | a l a 2 ... a n | | | = [ a 11 a 21 | a m 1 a l 2 a 22 | a m 2 a 1 n a 2 n | a m n ] .
(2)

In the transpose operation, columns become rows and vice versa. The transpose of an n×1n×1 matrix, a column vector, is a 1 ×n×n matrix, a row vector:

x=x1x2|xn;xT=[x1x2xn].x=x1x2|xn;xT=[x1x2xn].
(3)

Now we can define matrix-matrix multiplication in terms of inner products of vectors. Let's begin with matrices ARm×nARm×n and BRn×pBRn×p. To find the product AB, first divide each matrix into column vectors and row vectors as follows:

Thus a i a i is the ithith column of A and αjTαjT is the jthjth row of A. For matrix multiplication to be defined, the width of the first matrix must match the length of the second one so that all rows αiTαiT and columns b i b i have the same number of elements n n. The matrix product, C=C= AB, is an m×pm×p matrix defined by its elements as cij=(αi,bj)cij=(αi,bj). In words, each element of the product matrix, cijcij, is the inner product of the ithith row of the first matrix and the jthjth column of the second matrix.

For n-vectors x x and y y, the matrix product xTyxTy takes on a special significance. The product is, of course, a 1×11×1 matrix (a scalar). The special significance is that xTyxTy is the inner product of x x and y y:

Thus the notation xTyxTy is often used in place of (x,y)(x,y). Recall from Demo 1 from "Linear Algebra: Other Norms" that MATLAB uses x'*y to denote inner product.

Another special case of matrix multiplication is the outer product. Like the inner product, it involves two vectors, but this time the result is a matrix:

In the outer product, the inner products that define its elements are between one-dimensional row vectors of x x and one-dimensional column vectors of y T y T , meaning the (i,j)(i,j) element of A A is xiyjxiyj.

## Exercise 2

Find C=ABC=AB where A A and B B are given by

1. A= 1 - 1 2 3 0 5 , B = 0 - 2 1 - 3 4 2 2 0 2 - 2 3 1 ; A= 1 - 1 2 3 0 5 , B = 0 - 2 1 - 3 4 2 2 0 2 - 2 3 1 ;
2. A=1001,B=12345678A=1001,B=12345678 ;
3. A=1-1-11-11111,B=036147258A=1-1-11-11111,B=036147258.

There are several other equivalent ways to define matrix multiplication, and a careful study of the following discussion should improve your understanding of matrix multiplication. Consider A Rm×n,BRn×pRm×n,BRn×p, and C=ABC=AB so that CRm×pCRm×p. In pictures, we have

m [ C p ] = m [ A n ] p B n . m [ C p ] = m [ A n ] p B n .
(4)

In our definition, we represent C C on an entry-by-entry basis as

c i j = ( α i , b j ) = Σ k = 1 n a i k b k j . c i j = ( α i , b j ) = Σ k = 1 n a i k b k j .
(5)

In pictures,

You will prove in Exercise 3 that we can also represent C C on a column basis:

c j = Σ k = 1 n a k b k j . c j = Σ k = 1 n a k b k j .
(6)

Finally, C C can be represented as a sum of matrices, each matrix being an outer product:

C = Σ i = 1 n a i β i T C = Σ i = 1 n a i β i T
(7)

A numerical example should help clarify these three methods.

## Example 1

Let

A = 1 2 1 3 2 1 2 4 3 3 2 1 , B = 1 2 1 2 2 1 1 3 2 2 1 1 . A = 1 2 1 3 2 1 2 4 3 3 2 1 , B = 1 2 1 2 2 1 1 3 2 2 1 1 .
(8)

Using the first method of matrix multiplication, on an entry-by-entry basis, we have

c i j = Σ k = 1 4 a i k b k j c i j = Σ k = 1 4 a i k b k j
(9)

or

C = [ ( 1 1 + 2 2 + 1 1 + 3 2 ) ( 2 1 + 1 2 + 2 1 + 4 2 ) ( 3 1 + 3 2 + 2 1 + 1 2 ) ( 1 2 + 2 2 + 1 3 + 3 1 ) ( 2 2 + 1 2 + 2 3 + 4 1 ) ( 3 2 + 3 2 + 2 3 + 1 1 ) ( 1 1 + 2 1 + 1 2 + 3 1 ) ( 2 1 + 1 1 + 2 2 + 4 . 1 ) ( 3 1 + 3 1 + 2 2 + 1 1 ) ] C = [ ( 1 1 + 2 2 + 1 1 + 3 2 ) ( 2 1 + 1 2 + 2 1 + 4 2 ) ( 3 1 + 3 2 + 2 1 + 1 2 ) ( 1 2 + 2 2 + 1 3 + 3 1 ) ( 2 2 + 1 2 + 2 3 + 4 1 ) ( 3 2 + 3 2 + 2 3 + 1 1 ) ( 1 1 + 2 1 + 1 2 + 3 1 ) ( 2 1 + 1 1 + 2 2 + 4 . 1 ) ( 3 1 + 3 1 + 2 2 + 1 1 ) ]
(10)

or

C = 12 12 8 14 16 11 13 l 9 l 1 . C = 12 12 8 14 16 11 13 l 9 l 1 .
(11)

On a column basis,

c j = Σ k = 1 4 a k b k j c j = Σ k = 1 4 a k b k j
(12)

c1=1231+2132+1221+3412;c2=1232+2132+1223+3411; c 3 = 1 2 3 1 + 2 1 3 1 + 1 2 2 2 + 3 4 1 1 . c1=1231+2132+1221+3412;c2=1232+2132+1223+3411; c 3 = 1 2 3 1 + 2 1 3 1 + 1 2 2 2 + 3 4 1 1 .

Collecting terms together, we have

C = [ c 1 c 2 c 3 ] C = [ ( 1 1 + 2 2 + 1 1 + 3 2 ) ( 2 1 + 1 2 + 2 1 + 4 2 ) ( 3 1 + 3 2 + 2 1 + 1 2 ) ( 1 2 + 2 2 + 1 3 + 3 1 ) ( 2 2 + 1 2 + 2 3 + 4 1 ) ( 3 2 + 3 2 + 2 3 + 1 1 ) ( 1 1 + 2 1 + 1 2 + 3 1 ) ( 2 1 + 1 1 + 2 2 + 4 . 1 ) ( 3 1 + 3 1 + 2 2 + 1 1 ) ] C = [ c 1 c 2 c 3 ] C = [ ( 1 1 + 2 2 + 1 1 + 3 2 ) ( 2 1 + 1 2 + 2 1 + 4 2 ) ( 3 1 + 3 2 + 2 1 + 1 2 ) ( 1 2 + 2 2 + 1 3 + 3 1 ) ( 2 2 + 1 2 + 2 3 + 4 1 ) ( 3 2 + 3 2 + 2 3 + 1 1 ) ( 1 1 + 2 1 + 1 2 + 3 1 ) ( 2 1 + 1 1 + 2 2 + 4 . 1 ) ( 3 1 + 3 1 + 2 2 + 1 1 ) ]
(13)

On a matrix-by-matrix basis,

C = Σ i = 1 4 a i β i T C = Σ i = 1 4 a i β i T
(14)
C = [ 1 2 3 ] [ 1 2 1 ] + [ 2 1 3 ] [ 2 2 1 ] + [ 1 3 2 ] + [ 1 2 2 ] [ 1 3 2 ] + [ 3 4 1 ] [ 1 1 1 ] C=[ 1 2 3 ][ 1 2 1 ]+[ 2 1 3 ][ 2 2 1 ]+[ 1 3 2 ]+[ 1 2 2 ][ 1 3 2 ]+[ 3 4 1 ][ 1 1 1 ]
(15)
= 1 1 1 2 1 1 2 1 2 2 2 1 3 1 3 2 3 1 + 2 2 2 2 2 1 1 2 1 2 1 1 3 2 3 2 3 1 + 1 1 1 3 1 2 2 1 2 3 2 2 2 1 2 3 2 2 + 3 2 3 1 3 1 4 2 4 1 4 1 1 2 1 1 1 1 = 1 1 1 2 1 1 2 1 2 2 2 1 3 1 3 2 3 1 + 2 2 2 2 2 1 1 2 1 2 1 1 3 2 3 2 3 1 + 1 1 1 3 1 2 2 1 2 3 2 2 2 1 2 3 2 2 + 3 2 3 1 3 1 4 2 4 1 4 1 1 2 1 1 1 1
(16)
C = [ ( 1 1 + 2 2 + 1 1 + 3 2 ) ( 2 1 + 1 2 + 2 1 + 4 2 ) ( 3 1 + 3 2 + 2 1 + 1 2 ) ( 1 2 + 2 2 + 1 3 + 3 1 ) ( 2 2 + 1 2 + 2 3 + 4 1 ) ( 3 2 + 3 2 + 2 3 + 1 1 ) ( 1 1 + 2 1 + 1 2 + 3 1 ) ( 2 1 + 1 1 + 2 2 + 4 . 1 ) ( 3 1 + 3 1 + 2 2 + 1 1 ) ] C = [ ( 1 1 + 2 2 + 1 1 + 3 2 ) ( 2 1 + 1 2 + 2 1 + 4 2 ) ( 3 1 + 3 2 + 2 1 + 1 2 ) ( 1 2 + 2 2 + 1 3 + 3 1 ) ( 2 2 + 1 2 + 2 3 + 4 1 ) ( 3 2 + 3 2 + 2 3 + 1 1 ) ( 1 1 + 2 1 + 1 2 + 3 1 ) ( 2 1 + 1 1 + 2 2 + 4 . 1 ) ( 3 1 + 3 1 + 2 2 + 1 1 ) ]
(17)

as we had in each of the other cases. Thus we see that the methods are equivalent-simply different ways of organizing the same computation!

## Exercise 3

Prove that Equations 9, 11, and 13 are equivalent definitions of matrix multiplication. That is, if C=ABC=AB where ARm×nARm×n and BRn×pBRn×p, show that the matrix-matrix product can also be defined by

c i j = k = 1 n a i k b k j , c i j = k = 1 n a i k b k j ,
(18)

and, if c k c k is the kthkth column of C C and a k a k is the kthkth column of A A, then

c j = k = 1 n a k b k j . c j = k = 1 n a k b k j .
(19)

Show that the matrix CC may also be written as the ‚ "sum of outer products"

C = k = 1 n a k β k T . C = k = 1 n a k β k T .
(20)

Write out the elements in a typical outer product akβkTakβkT.

## Exercise 4

Given ARm×n,BRp×qARm×n,BRp×q, and CRr×sCRr×s, for each of the following postulates, either prove that it is true or give a counterexample showing that it is false:

1. (AT)T=A(AT)T=A.
2. AB=BAAB=BA when n=pn=p and m=qm=q. Is matrix multiplication commutative?
3. A(B+C)=AB+ACA(B+C)=AB+AC when n=p=rn=p=r and q=sq=s. Is matrix multiplication distributive over addition?
4. ((AB )T=BTAT)T=BTAT when n=pn=p.
5. ( A B ) C = A ( B C ) (AB)C=A(BC) when n=pn=p and q=rq=r. Is matrix multiplication associative?

## Example 2: Rotation

We know from the chapter on complex numbers that a complex number z1=x1+jy1z1=x1+jy1 may be rotated by angle θ θ in the complex plane by forming the product

z 2 = e j θ z 1 . z 2 = e j θ z 1 .
(21)

When written out, the real and imaginary parts of z 2 z 2 are

z2 = ( cos θ + j sin θ ) (x1+jy1) = ( cos θ ) x 1 - ( sin θ ) y 1 + j [ ( sin θ ) x 1 + ( cos θ ) y 1 ] . z2 = ( cos θ + j sin θ ) (x1+jy1) = ( cos θ ) x 1 - ( sin θ ) y 1 + j [ ( sin θ ) x 1 + ( cos θ ) y 1 ] .
(22)

If the real and imaginary parts of z 1 z 1 and z 2 z 2 are organized into vectors z 1 z 1 and z 2 z 2 as in the chapter on complex numbers, then rotation may be carried out with the matrix-vector multiply

z 2 = x 2 y 2 = c o s θ - s i n θ s i n θ c o s θ x 1 y 1 . z 2 = x 2 y 2 = c o s θ - s i n θ s i n θ c o s θ x 1 y 1 .
(23)

We call the matrix R(θ)=cosθ-sinθsinθcosθR(θ)=cosθ-sinθsinθcosθ a rotation matrix.

## Exercise 5

Let R(θ)R(θ) denote a 2×22×2 rotation matrix. Prove and interpret the following two properties:

1. RT(θ)=R(-θ)RT(θ)=R(-θ);
2. RT(θ)R(θ)=R(θ)RT(θ)=1001.RT(θ)R(θ)=R(θ)RT(θ)=1001.

## Content actions

PDF | EPUB (?)

### What is an EPUB file?

EPUB is an electronic book format that can be read on a variety of mobile devices.

PDF | EPUB (?)

### What is an EPUB file?

EPUB is an electronic book format that can be read on a variety of mobile devices.

#### Collection to:

My Favorites (?)

'My Favorites' is a special kind of lens which you can use to bookmark modules and collections. 'My Favorites' can only be seen by you, and collections saved in 'My Favorites' can remember the last module you were on. You need an account to use 'My Favorites'.

| A lens I own (?)

#### Definition of a lens

##### Lenses

A lens is a custom view of the content in the repository. You can think of it as a fancy kind of list that will let you see content through the eyes of organizations and people you trust.

##### What is in a lens?

Lens makers point to materials (modules and collections), creating a guide that includes their own comments and descriptive tags about the content.

##### Who can create a lens?

Any individual member, a community, or a respected organization.

##### What are tags?

Tags are descriptors added by lens makers to help label content, attaching a vocabulary that is meaningful in the context of the lens.

| External bookmarks

#### Module to:

My Favorites (?)

'My Favorites' is a special kind of lens which you can use to bookmark modules and collections. 'My Favorites' can only be seen by you, and collections saved in 'My Favorites' can remember the last module you were on. You need an account to use 'My Favorites'.

| A lens I own (?)

#### Definition of a lens

##### Lenses

A lens is a custom view of the content in the repository. You can think of it as a fancy kind of list that will let you see content through the eyes of organizations and people you trust.

##### What is in a lens?

Lens makers point to materials (modules and collections), creating a guide that includes their own comments and descriptive tags about the content.

##### Who can create a lens?

Any individual member, a community, or a respected organization.

##### What are tags?

Tags are descriptors added by lens makers to help label content, attaching a vocabulary that is meaningful in the context of the lens.

| External bookmarks