Next we equip a normed vector space VV with a notion of "direction".
-
An inner product is a function (
(· · ·:V×V)→C
:
·
·
V
V
) such that the following properties hold (
,
x⇀∈V∧y⇀∈V∧z⇀∈V
x
y
z
x
V
y
V
z
V
and
,
α∈
α
α
):
-
x⇀ · y⇀=y⇀ · x⇀*
x
y
y
x
-
x⇀ · αy⇀=αx⇀ · y⇀
x
α
y
α
x
y
...implying that
αx⇀ · y⇀=α*x⇀ · y⇀
α
x
y
α
x
y
-
x⇀ · y⇀+z⇀=x⇀ · y⇀+x⇀ · z⇀
x
y
z
x
y
x
z
-
x⇀ · x⇀≥0
x
x
0
with equality iff
x⇀=0
x
0
-
V=RN
V
N
,
x⇀ · y⇀≔x⇀Ty⇀
≔
x
y
x
y
-
V=CN
V
N
,
x⇀ · y⇀≔x⇀Hy⇀
≔
x
y
x
y
-
V=
l2
V
l2
,
xn · yn≔∑n=−∞∞xn*yn
≔
x
n
y
n
n
x
n
y
n
-
V=
ℒ2
V
ℒ2
,
ft · gt≔∫−∞∞ft*gtdt
≔
f
t
g
t
t
f
t
g
t
The inner products above are the "usual" choices for those
spaces.
The inner product naturally defines a norm:
∥x⇀∥≔x⇀ · x⇀
≔
x
x
x
though not every norm can be defined from an inner product.
Thus, an inner product space can be
considered as a normed vector space with additional
structure. Assume, from now on, that we adopt the
inner-product norm when given a choice.
-
The Cauchy-Schwarz inequality says
|x⇀ · y⇀|≤∥x⇀∥∥y⇀∥
x
y
x
y
with equality iff
∃α∈:x⇀=αy⇀
α
x
α
y
.
When
x⇀ · y⇀∈R
x
y
, the inner product can be used to define an "angle"
between vectors:
cosθ=x⇀ · y⇀∥x⇀∥∥y⇀∥
θ
x
y
x
y
-
Vectors x⇀x and
y⇀y are said to be
orthogonal, denoted as
x⇀⊥y⇀
⊥
x
y
, when
x⇀ · y⇀=0
x
y
0
. The Pythagorean theorem says:
∥x⇀+y⇀∥2=∥x⇀∥2+∥y⇀∥2 ,
x⇀⊥y⇀
⊥
x
y
x
y
2
x
2
y
2
Vectors x⇀x and
y⇀y are said to be
orthonormal when
x⇀⊥y⇀
⊥
x
y
and
∥x⇀∥=∥y⇀∥=1
x
y
1
.
-
x⇀⊥S
⊥
x
S
means
x⇀⊥y⇀
⊥
x
y
for all
y⇀∈S
y
S
. SS is an
orthogonal set if
x⇀⊥y⇀
⊥
x
y
for all
x⇀∧y⇀∈S
x
y
S
s.t.
x⇀≠y⇀
x
y
. An orthogonal set SS is an orthonormal
set if
∥x⇀∥=1
x
1
for all
x⇀∈S
x
S
. Some examples of orthonormal sets are
-
R3
3
:
S=(
1
0
0
)(
0
1
0
)
S
1
0
0
0
1
0
-
CN
N
: Subsets of columns from unitary matrices
-
l2
l2
: Subsets of shifted Kronecker delta functions
S⊂
δn−k
k∈Z
S
δ
n
k
k
-
ℒ2
ℒ2
:
S=
1Tft−nT
n∈Z
S
1
T
f
t
n
T
n
for unit pulse
ft=ut−ut−T
f
t
u
t
u
t
T
, unit step
ut
u
t
