-
The orthogonal projection of y onto S, where
SS is a closed subspace
of VV, is
y
̂
=∑
i
i〈(
x
i
,y)〉
x
i
y
̂
i
i
x
i
y
x
i
s.t.
x
i
x
i
is an ON basis for SS. Orthogonal projection yields the
best approximation of yy in SS:
y
̂
=argmin
x∈S
∥y−x∥
y
̂
argmin
x
S
y
x
The approximation error
e=y−
y
̂
e
y
y
̂
obeys to orthogonality principle:
e⊥S
⊥
e
S
We illustrate this concept using
V=R3
V
3
below but stress that the same geometrical
interpretation applies to any Hilbert space.
A proof of the orthogonality principle is:
e⊥S⇔∀
i
:〈e,
x
i
〉=0
⇔
⊥
e
S
i
e
x
i
0
〈y−
y
̂
,
x
i
〉=0
y
y
̂
x
i
0
〈y,
x
i
〉=〈
y
̂
,
x
i
〉
y
x
i
y
̂
x
i
〈y,
x
i
〉=〈∑
j
j〈(
x
j
,y)〉
x
j
,
x
i
〉
y
x
i
j
j
x
j
y
x
j
x
i
〈y,
x
i
〉=∑
j
j〈
x
j
,y〉¯〈(
x
j
,
x
i
)〉
y
x
i
j
j
x
j
y
x
j
x
i
〈y,
x
i
〉=∑
j
j〈(y,
x
j
)〉
d
i
−
j
y
x
i
j
j
y
x
j
d
i
−
j
〈y,
x
i
〉=〈y,
x
i
〉
y
x
i
y
x
i
