Approximation and Projections in Hilbert Space 2.7 2002/07/30 2007/07/20 11:27:57.773 GMT-5 Justin Romberg jrom@ece.gatech.edu Prashant Singh prash@ece.rice.edu Mariyah Poonawala mariyah@rice.edu Richard G. Baraniuk richb@rice.edu approximation Hilbert space projection This module introduces approximation and projections in Hilbert space.

Introduction Given a line 'l' and a point 'p' in the plane, what's the closest point 'm' to 'p' on 'l'?

Figure of point 'p' and line 'l' mentioned above.

Same problem: Let x and v be vectors in 2 . Say v 1 . For what value of α is x α v 2 minimized? (what point in span{v} best approximates x?)

The condition is that x α ^ v and α v are orthogonal.

Calculating α How to calculate α ^ ? We know that ( x α ^ v ) is perpendicular to every vector in span{v}, so β ∀ β x α ^ v β v 0 β x v α ^ β v v 0 because v v 1 , so x v α ^ 0 α ^ x v Closest vector in span{v} = x v v , where x v v is the projection of x onto v. We can do the same thing in higher dimensions. Let V H be a subspace of a Hilbert space H. Let x H be given. Find the y V that best approximates x. i.e., x y is minimized. Find an orthonormal basis b1 … bk for V Project x onto V using y i 1 k x bi bi then y is the closest point in V to x and (x-y) ⊥ V ( v ∀ v V x y v 0 x 3 , V span 1 0 0 0 1 0 , x a b c . So, y i 1 2 x bi bi a 1 0 0 b 0 1 0 a b 0 V = {space of periodic signals with frequency no greater than 3 w0 }. Given periodic f(t), what is the signal in V that best approximates f? { 1 T w0 k t , k = -3, -2, ..., 2, 3} is an ONB for V g t 1 T k -3 3 f t w0 k t w0 k t is the closest signal in V to f(t) ⇒ reconstruct f(t) using only 7 terms of its Fourier series. Let V = {functions piecewise constant between the integers} ONB for V. bi 1 i 1 t i 0 where {bi} is an ONB. Best piecewise constant approximation? g t i f bi bi f bi t f t bi t t i 1 i f t This demonstration explores approximation using a Fourier basis and a Haar Wavelet basis.See here for instructions on how to use the demo.