Inner Product Spaces Inner Product Spaces We have seen that linear vector spaces provide a useful framework for describing a broad range of classes of signals, both discrete-time and continuous-time. The notions of vector length and distances between vectors were introduced with the concept of normed linear vector spaces (i.e., linear vector spaces with a norm). However, optimization and signal approximation are still problematic in many normed linear spaces, as, in general, the solution may not be unique and it may be difficult to find the solution(s). Consider the vector space ℝ 3 with the Euclidean norm x x 1 2 x 2 2 x 3 2 1 2 . This three-dimensional normed linear space, unlike many of the normed linear spaces we have seen, has many of the properties that we expect from everyday 3-D life: The unit sphere (i.e., all vectors x such that x 1 ) is round. Any subspace (i.e., plane or line) has a unique closest point to any element s in ℝ 3 . Practical experience indicates that the line from s to the best approximation s ̂ in the subspace Φ 2 is perpendicular to the plane Φ 2 . Normed linear spaces do not have the necessary structure to handle the concepts of orthogonality or angles between vectors. However, ℝ 3 with the Euclidean norm turns out to be one example of an inner product space, and it is possible to define orthogonality in inner product spaces. Inner Product In a complex linear space S an inner product x y assigns a complex number to any pair of elements x y S . The inner product must satisfy the following three conditions: x y y x For any complex scalars a, b, a x b y z a x z b y z x x 0 and ⇔ x x 0 x 0 . In ℝ N , the usual dot product x y k 0 N x k y k y x is one possible inner product. For complex-valued continuous-time signals in C a b x y t a b x t y t is the most common inner product.