We will now revisit “The Fundamental Theorem of Approximation” for the
extremely important case where our set AA is a subspace. Specifically, suppose
that HH is a Hilbert space, and let AA be a (closed) subspace of HH. From
before, we have that for any x∈Hx∈H there is a unique x^∈Ax^∈A such that x^x^
is the closest point in AA to xx. When AA is also a subspace, we also have:
x^∈Ax^∈A is the minimizer of x-x^x-x^ if any only if x^-x⊥Ax^-x⊥A i.e., x^-x,y=0x^-x,y=0 for all y∈Ay∈A.
- (a) Suppose that x^-x⊥Ax^-x⊥A. Then for any y∈Ay∈A with y≠x^,y≠x^,y-x2=y-x^+x^-x2y-x2=y-x^+x^-x2.
Note that y-x^∈Ay-x^∈A, but x^-x⊥Ax^-x⊥A, so that y-x^,x^-x=0y-x^,x^-x=0, and we
can apply Pythagoras to obtain y-x2=y-x^2+x^-xy-x2=y-x^2+x^-x. Since y≠x^y≠x^, we thus
have that y-x2>x^-x2y-x2>x^-x2.
Thus x^x^ must be the closest point in AA to xx.
- (b) Suppose that x^x^ minimizes x-x^x-x^. Suppose for the
sake of a contradiction that ∃y∈A∃y∈A such that y=1y=1 and x-x^,y=δ≠0x-x^,y=δ≠0.
Let z=x^+δyz=x^+δy.
x
-
z
2
=
x
-
x
^
-
δ
y
2
=
x
-
x
^
,
x
-
x
^
-
x
-
x
^
,
δ
y
-
δ
y
,
x
-
x
^
+
δ
y
,
δ
y
=
x
-
x
^
2
-
δ
¯
δ
-
δ
δ
¯
+
δ
δ
¯
=
x
-
x
^
2
-
|
δ
|
2
.
x
-
z
2
=
x
-
x
^
-
δ
y
2
=
x
-
x
^
,
x
-
x
^
-
x
-
x
^
,
δ
y
-
δ
y
,
x
-
x
^
+
δ
y
,
δ
y
=
x
-
x
^
2
-
δ
¯
δ
-
δ
δ
¯
+
δ
δ
¯
=
x
-
x
^
2
-
|
δ
|
2
.
(1)
Thus x-z≤x-x^x-z≤x-x^,
contradicting the assumption that x^x^ minimizes x-x^.x-x^.
This result suggests a that a possible method for finding the best
approximation to a signal xx from a vector space VV is to simply look for a
vector x^x^ such that x^-x⊥Vx^-x⊥V. In the coming lectures we will show how to do this, but it will require a
brief review of some concepts from linear algebra.
