It is important to analyze the LMS algorithm to determine
under what conditions it is stable, whether or not it converges
to the Wiener solution, to determine how quickly it converges,
how much degredation is suffered due to the noisy gradient,
etc. In particular, we need to know how to choose the parameter

**Mean of W**

does

**Patently False Assumption**

With the independence assumption,

Now

Putting this back into our equation

*if*

If

So the LMS algorithm, *if* it
converges, gives filter coefficients which on average are
the Wiener coefficients! This is, of course, a desirable
result.

**First-order stability**

But does

Let's rewrite the analysis in term of

**Linear Algebra Fact**

Since

Using this fact,

Since

For a correlation matrix,

**Rate of convergence**

Each of the modes decays as

#### Good news:

*initial*rate of convergence is dominated by the fastest mode

#### Bad news:

*final*rate of convergence is dominated by the slowest mode

Comments:"A good introduction in adaptive filters, a major DSP application."