Recall that if PP is an orthogonal projection onto a subspace AA, we can write
any xx as
x
=
P
x
+
(
I
-
P
)
x
x
=
P
x
+
(
I
-
P
)
x
(1)where Px∈APx∈A and (I-P)x⊥A(I-P)x⊥A. We now turn to how to actually find PP.
We begin with the finite-dimensional case, assuming that {v1,...,vN}{v1,...,vN} is a basis for AA. If (I-P)x⊥A(I-P)x⊥A then we have that for any xx
(
I
-
P
)
x
,
v
j
=
0
for
j
=
1
,
.
.
.
,
N
(
I
-
P
)
x
,
v
j
=
0
for
j
=
1
,
.
.
.
,
N
(2)We also note that since Px∈APx∈A, we can write Px=∑k=1NckvkPx=∑k=1Nckvk. Thus we obtain
x
-
∑
k
=
1
N
c
k
v
k
,
v
j
=
0
for
j
=
1
,
.
.
.
,
N
x
-
∑
k
=
1
N
c
k
v
k
,
v
j
=
0
for
j
=
1
,
.
.
.
,
N
(3)from which we obtain
x
,
v
j
=
∑
k
=
1
N
c
k
v
k
,
v
j
for
j
=
1
,
.
.
.
,
N
x
,
v
j
=
∑
k
=
1
N
c
k
v
k
,
v
j
for
j
=
1
,
.
.
.
,
N
(4)We know xx and v1,...,vNv1,...,vN. Our goal is to find c1,...,cNc1,...,cN. Note that a procedure for calculating c1,...,ckc1,...,ck for any given xx is
equivalent to one that computes PxPx.
To find c1,...,cNc1,...,cN, observe that Equation 4 represents a set of NN equations with NN unknowns.
〈
v
1
,
v
1
〉
〈
v
2
,
v
1
〉
⋯
〈
v
N
,
v
1
〉
〈
v
1
,
v
2
〉
〈
v
2
,
v
2
〉
〈
v
N
,
v
2
〉
⋮
⋱
⋮
〈
v
1
,
v
N
〉
〈
v
2
,
v
N
〉
⋯
〈
v
N
,
v
N
〉
c
1
c
2
⋮
c
N
=
〈
x
,
v
1
〉
〈
x
,
v
2
〉
⋮
〈
x
,
v
N
〉
〈
v
1
,
v
1
〉
〈
v
2
,
v
1
〉
⋯
〈
v
N
,
v
1
〉
〈
v
1
,
v
2
〉
〈
v
2
,
v
2
〉
〈
v
N
,
v
2
〉
⋮
⋱
⋮
〈
v
1
,
v
N
〉
〈
v
2
,
v
N
〉
⋯
〈
v
N
,
v
N
〉
c
1
c
2
⋮
c
N
=
〈
x
,
v
1
〉
〈
x
,
v
2
〉
⋮
〈
x
,
v
N
〉
(5)More compactly, we want to find a vector c∈CNc∈CN such that Gc=bGc=b where
b
=
〈
x
,
v
1
〉
〈
x
,
v
2
〉
⋮
〈
x
,
v
N
〉
b
=
〈
x
,
v
1
〉
〈
x
,
v
2
〉
⋮
〈
x
,
v
N
〉
(6)- GG is called the “Grammian” or “Gram matrix” of {vj}{vj}
- One can show since v1,...,vNv1,...,vN are linearly independent that GG is
positive definite, and hence inevitable.
- Also note that by construction, GG is conjugate symmetric, or “Hermitian”, i.e., G=GHG=GH, where HH denotes the conjugate transpose of GG.
Thus, since G-1G-1 exists, we can write c=G-1bc=G-1b to calculate cc.
As a special case, suppose now that {vj}{vj} is an orthobasis for AA? What is GG? It is just the identity matrix II!
Computing cc just got much easier, since now c=bc=b. Plugging this cc back into out formula for PxPx we obtain
P
x
=
∑
k
=
1
N
x
,
v
k
v
k
P
x
=
∑
k
=
1
N
x
,
v
k
v
k
(7)Just to verify, note that PP is indeed a projection matrix:
P
(
P
x
)
=
∑
N
k
=
1
∑
N
j
=
1
x
,
v
j
v
j
,
v
k
v
k
=
∑
N
k
=
1
∑
N
j
=
1
x
,
v
j
v
j
,
v
k
v
k
=
∑
N
j
=
1
x
,
v
j
v
j
=
P
x
.
P
(
P
x
)
=
∑
N
k
=
1
∑
N
j
=
1
x
,
v
j
v
j
,
v
k
v
k
=
∑
N
k
=
1
∑
N
j
=
1
x
,
v
j
v
j
,
v
k
v
k
=
∑
N
j
=
1
x
,
v
j
v
j
=
P
x
.
(8)Example Suppose f∈L2([0,4])f∈L2([0,4]) is given by
Suppose f∈L2([0,4])f∈L2([0,4]) is given by
f
(
t
)
=
t
if
t
∈
[
0
,
1
2
]
1
-
t
if
t
∈
[
1
2
,
1
]
.
f
(
t
)
=
t
if
t
∈
[
0
,
1
2
]
1
-
t
if
t
∈
[
1
2
,
1
]
.
(9)Let A={ piecewise constant functions on [0,14),[14,12),[12,34),[34,1]}A={ piecewise constant functions on [0,14),[14,12),[12,34),[34,1]}.
Our goal is to find the closest (in L2L2) function in AA to f(t)f(t). Using v1,...,v4v1,...,v4 from before, we can calculate
c1=14c1=14, c2=0c2=0, c3=-216c3=-216, c4=216c4=216. Thus, we have that
f
^
(
t
)
=
1
4
v
1
-
2
16
v
3
+
2
16
v
4
.
f
^
(
t
)
=
1
4
v
1
-
2
16
v
3
+
2
16
v
4
.
(10)
