Connexions

You are here: Home » Content » Linear Algebra: The Basics
Content Actions
Lenses

What is a lens?

Lenses

A lens is a custom view of Connexions content. You can think of it as a fancy kind of list that will let you see Connexions through the eyes of organizations and people you trust.

What is in a lens?

Lens makers point to Connexions materials (modules and collections), creating a guide that includes their own comments and descriptive tags about the content.

Who can create a lens?

Any individual Connexions member, a community, or a respected organization.

This content is ...
Affiliated with (?)
This content is either by members of the organizations listed or about topics related to the organizations listed. Click each link to see a list of all content affiliated with the organization.

  • This module is included inLens: Rice University OpenCourseWare
    By: OpenCourseWare ConsortiumAs a part of collection:"Signals and Systems"

    Click the "Rice University OCW" link to see all content affiliated with them.

    Rice University OCW
Also in these lenses

  • This module is included inLens: richb's DSP resources
    By: Richard BaraniukAs a part of collection:"Signals and Systems"

    Comments:

    "My introduction to signal processing course at Rice University."

    Click the "richb's DSP" link to see all content selected in this lens.

    richb's DSP
Tags

(?)

These tags come from the endorsement, affiliation, and other lenses that include this content.

Linear Algebra: The Basics

Module by: Michael Haag, Justin Romberg

Summary: This module will give a very brief tutorial on some of the basic terms and ideas of linear algebra. These will include linear independence, span, and basis.

This brief tutorial on some key terms in linear algebra is not meant to replace or be very helpful to those of you trying to gain a deep insight into linear algebra. Rather, this brief introduction to some of the terms and ideas of linear algebra is meant to provide a little background to those trying to get a better understanding or learn about eigenvectors and eigenfunctions, which play a big role in deriving a few important ideas on Signals and Systems. The goal of these concepts will be to provide a background for signal decomposition and to lead up to the derivation of the Fourier Series.

Linear Independence

A set of vectors ∀x,xi∈ℂn: x1x2…xk x x i n x 1 x 2 … x k are linearly independent if none of them can be written as a linear combination of the others.
Definition 1: Linearly Independent
For a given set of vectors, x1x2…xn x 1 x 2 … x n , they are linearly independent if c 1 x1+ c 2 x2+…+ c n xn=0 c 1 x 1 c 2 x 2 … c n x n 0 only when c 1 = c 2 =…= c n =0 c 1 c 2 … c n 0
Example 
We are given the following two vectors: x1= 32 x 1 3 2 x2= -6-4 x 2 -6 -4 These are not linearly independent as proven by the following statement, which, by inspection, can be seen to not adhere to the definition of linear independence stated above. x2=-2x1⇒2x1+x2=0 x 2 -2 x 1 2 x 1 x 2 0 Another approach to reveal a vectors independence is by graphing the vectors. Looking at these two vectors geometrically (as in Figure 1), one can again prove that these vectors are not linearly independent.
vec_f1.png
Figure 1: Graphical representation of two vectors that are not linearly independent.
Example 1 
We are given the following two vectors: x1= 32 x 1 3 2 x2= 12 x 2 1 2 These are linearly independent since c 1 x1=- c 2 x2 c 1 x 1 c 2 x 2 only if c 1 = c 2 =0 c 1 c 2 0 . Based on the definition, this proof shows that these vectors are indeed linearly independent. Again, we could also graph these two vectors (see Figure 2) to check for linear independence.
vec_f2.png
Figure 2: Graphical representation of two vectors that are linearly independent.
Problem 1
Are x1x2x3 x 1 x 2 x 3 linearly independent? x1= 32 x 1 3 2 x2= 12 x 2 1 2 x3= -10 x 3 -1 0
[ Click for Solution 1 ]
Solution 1
By playing around with the vectors and doing a little trial and error, we will discover the following relationship: x1-x2+2x3=0 x 1 x 2 2 x 3 0 Thus we have found a linear combination of these three vectors that equals zero without setting the coefficients equal to zero. Therefore, these vectors are not linearly independent!
[ Hide Solution 1 ]
As we have seen in the two above examples, often times the independence of vectors can be easily seen through a graph. However this may not be as easy when we are given three or more vectors. Can you easily tell whether or not these vectors are independent from Figure 3. Probably not, which is why the method used in the above solution becomes important.
vec_f3.png
Figure 3: Plot of the three vectors. Can be shown that a linear combination exists among the three, and therefore they are not linear independent.
hint: A set of mm vectors in ℂn n cannot be linearly independent if m>n m n .

Span

Definition 2: Span
The span of a set of vectors x1x2…xk x 1 x 2 … x k is the set of vectors that can be written as a linear combination of x1x2…xk x 1 x 2 … x k span x1…xk = ∀α, α i ∈ℂn: α 1 x1+ α 2 x2+…+ α k xk span x 1 … x k α α i n α 1 x 1 α 2 x 2 … α k x k
Example 
Given the vector x1= 32 x 1 3 2 the span of x1 x 1 is a line.
Example 
Given the vectors x1= 32 x 1 3 2 x2= 12 x 2 1 2 the span of these vectors is ℂ2 2 .

Basis

Definition 3: Basis
A basis for ℂn n is a set of vectors that: (1) spans ℂn n and (2) is linearly independent.
Clearly, any set of nn linearly independent vectors is a basis for ℂn n .
Example 2 
We are given the following vector ei= 0⋮010⋮0 e i 0 ⋮ 0 1 0 ⋮ 0 where the 11 is always in the iith place and the remaining values are zero. Then the basis for ℂn n is ∀i,i= 12…n :ei i i 1 2 … n e i
note: ∀i,i= 12…n :ei i i 1 2 … n e i is called the standard basis.
Example 3 
h1= 11 h 1 1 1 h2= 1-1 h 2 1 -1 h1h2 h 1 h 2 is a basis for ℂ2 2 .
vec_f4.png
Figure 4: Plot of basis for ℂ2 2
If b1…b2 b 1 … b 2 is a basis for ℂn n , then we can express any x∈ℂn x n as a linear combination of the b i b i 's: ∀α, α i ∈ℂ:x= α 1 b1+ α 2 b2+…+ α n bn α α i x α 1 b 1 α 2 b 2 … α n b n
Example 4 
Given the following vector, x= 12 x 1 2 writing x x in terms of e1e2 e 1 e 2 gives us x=e1+2e2 x e 1 2 e 2
Problem 2
Try and write x x in terms of h1h2 h 1 h 2 (defined in the previous example).
[ Click for Solution 2 ]
Solution 2
x=32h1+-12h2 x 3 2 h 1 -1 2 h 2
[ Hide Solution 2 ]
In the two basis examples above, x x is the same vector in both cases, but we can express it in many different ways (we give only two out of many, many possibilities). You can take this even further by extending this idea of a basis to function spaces.
note: As mentioned in the introduction, these concepts of linear algebra will help prepare you to understand the Fourier Series, which tells us that we can express periodic functions, ft f t , in terms of their basis functions, ⅇⅈ ω 0 nt ω 0 n t .

Download LabVIEW Source

Download LabVIEW Source

Comments, questions, feedback, criticisms?

Send feedback