-
The orthogonal projection of y onto S, where
SS is a closed subspace
of VV, is
y
̂
=∑i<xi,y>xi
y
̂
i
i
x
i
y
x
i
s.t.
xi
x
i
is an ON basis for SS. Orthogonal projection yields the
best approximation of yy in SS:
y
̂
=argminx∈S∥y-x∥
y
̂
argmin
x
S
y
x
The approximation error
e=y-
y
̂
e
y
y
̂
obeys to orthogonality principle:
e⊥S
⊥
e
S
We illustrate this concept using
V=ℝ3
V
3
below but stress that the same geometrical
interpretation applies to any Hilbert space.
A proof of the orthogonality principle is:
e⊥S⇔∀i:<e,xi>=0
⇔
⊥
e
S
i
e
x
i
0
<y-
y
̂
,xi>=0
y
y
̂
x
i
0
<y,xi>=<
y
̂
,xi>
y
x
i
y
̂
x
i
<y,xi>=<∑j<xj,y>xj,xi>
y
x
i
j
j
x
j
y
x
j
x
i
<y,xi>=∑j<xj,y>¯<xj,xi>
y
x
i
j
j
x
j
y
x
j
x
i
<y,xi>=∑j<y,xj>
d
i
−
j
y
x
i
j
j
y
x
j
d
i
−
j
<y,xi>=<y,xi>
y
x
i
y
x
i