Connexions

You are here: Home » Content » Inner Product Space
Content Actions

Inner Product Space

Module by: Phil Schniter

Summary: This module introduces inner product space.

Next we equip a normed vector space VV with a notion of "direction".
  • An inner product is a function ( <·,·>:V×V→ℂ : · · V V ) such that the following properties hold ( ∀x,y,z,x∈V∧y∈V∧z∈V x y z x V y V z V and ∀α,α∈ α α ):
    1. <x,y>=<y,x>¯ x y y x
    2. <x,αy>=α<x,y> x α y α x y ...implying that <αx,y>=α¯<x,y> α x y α x y
    3. <x,y+z>=<x,y>+<x,z> x y z x y x z
    4. <x,x>≥0 x x 0 with equality iff x=0 x 0
    In simple terms, the inner product measures the relative alignment between two vectors. Adding an inner product operation to a vector space yields an inner product space. Important examples include:
    1. V=ℝN V N , <x,y>≔xTy ≔ x y x y
    2. V=ℂN V N , <x,y>≔xHy ≔ x y x y
    3. V=l2 V l2 , <xn,yn>≔∑n=-∞∞xn¯yn ≔ x n y n n x n y n
    4. V=ℒ2 V ℒ2 , <ft,gt>≔∫-∞∞ft¯gtdt ≔ f t g t t f t g t
The inner products above are the "usual" choices for those spaces.
The inner product naturally defines a norm: ∥x∥≔<x,x> ≔ x x x though not every norm can be defined from an inner product. 1 Thus, an inner product space can be considered as a normed vector space with additional structure. Assume, from now on, that we adopt the inner-product norm when given a choice.
  • The Cauchy-Schwarz inequality says
|<x,y>|≤∥x∥∥y∥ x y x y with equality iff ∃,α∈:x=αy α x α y .
When <x,y>∈ℝ x y , the inner product can be used to define an "angle" between vectors: cosθ=<x,y>∥x∥∥y∥ θ x y x y
  • Vectors xx and yy are said to be orthogonal, denoted as x⊥y ⊥ x y , when <x,y>=0 x y 0 . The Pythagorean theorem says: ∀,x⊥y:∥x+y∥2=∥x∥2+∥y∥2 ⊥ x y x y 2 x 2 y 2 Vectors xx and yy are said to be orthonormal when x⊥y ⊥ x y and ∥x∥=∥y∥=1 x y 1 .
  • x⊥S ⊥ x S means x⊥y ⊥ x y for all y∈S y S . SS is an orthogonal set if x⊥y ⊥ x y for all x∧y∈S x y S s.t. x≠y x y . An orthogonal set SS is an orthonormal set if ∥x∥=1 x 1 for all x∈S x S . Some examples of orthonormal sets are
    1. ℝ3 3 : S=100010 S 1 0 0 0 1 0
    2. ℂN N : Subsets of columns from unitary matrices
    3. l2 l2 : Subsets of shifted Kronecker delta functions S⊂{δn-k|k∈ℤ} S δ n k k
    4. ℒ2 ℒ2 : S={1Tft-nT|n∈ℤ} S 1 T f t n T n for unit pulse ft=ut-ut-T f t u t u t T , unit step ut u t
    where in each case we assume the usual inner product.
1. An example for inner product space ℒ 2 ℒ 2 would be any norm ∥f∥≔∫-∞∞|ft|pdtp ≔ f p t f t p such that p>2 p 2 .

Comments, questions, feedback, criticisms?

Send feedback