A set of vectors or functions fk(t)fk(t)spans a vector space
FF (or FF is the Span of the set) if any element of
that space can be expressed as a linear combination of members of that
set, meaning: Given the finite or infinite set of functions fk(t)fk(t), we
define Span k{fk}=F Span k{fk}=F as the vector space with all elements of the
space of the form
g
(
t
)
=
∑
k
a
k
f
k
(
t
)
g
(
t
)
=
∑
k
a
k
f
k
(
t
)
(1)with k∈Zk∈Z and t,a∈Rt,a∈R. An inner product
is usually defined for this space and is denoted 〈f(t),g(t)〉〈f(t),g(t)〉. A norm is defined and is denoted by
∥f∥=〈f,f〉∥f∥=〈f,f〉.
We say that the set fk(t)fk(t) is a basis set or a basis
for a
given space FF if the set of {ak}{ak} in Equation 1 are unique
for any particular g(t)∈Fg(t)∈F. The set is called an orthogonal basis
if 〈fk(t),fℓ(t)〉=0〈fk(t),fℓ(t)〉=0 for all k≠ℓk≠ℓ. If we are in three dimensional Euclidean space, orthogonal
basis vectors are coordinate vectors that are at right (90o) angles to
each other. We say the set is an orthonormal basis if 〈fk(t),fℓ(t)〉=δ(k-ℓ)〈fk(t),fℓ(t)〉=δ(k-ℓ) i.e. if, in addition to being
orthogonal, the basis vectors are normalized to unity norm: ∥fk(t)∥=1∥fk(t)∥=1 for all kk.
From these definitions it is clear that if we have an orthonormal basis,
we can express any element in the vector space, g(t)∈Fg(t)∈F, written as
Equation 1 by
g
(
t
)
=
∑
k
〈
g
(
t
)
,
f
k
(
t
)
〉
f
k
(
t
)
g
(
t
)
=
∑
k
〈
g
(
t
)
,
f
k
(
t
)
〉
f
k
(
t
)
(2)since by taking the inner product of fk(t)fk(t) with both sides of Equation 1,
we get
a
k
=
〈
g
(
t
)
,
f
k
(
t
)
〉
a
k
=
〈
g
(
t
)
,
f
k
(
t
)
〉
(3)where this inner product of the signal g(t)g(t) with the basis vector fk(t)fk(t)
“picks out" the corresponding coefficient akak. This expansion
formulation or representation is extremely valuable. It expresses
Equation 2 as an identity operator in the sense that the inner product
operates on g(t)g(t) to produce a set of coefficients that, when used to
linearly combine the basis vectors, gives back the original signal g(t)g(t).
It is the foundation of Parseval's theorem which says the norm or energy
can be partitioned in terms of the expansion coefficients akak. It is
why the interpretation, storage, transmission, approximation, compression, and
manipulation of the coefficients can be very useful. Indeed,
Equation 2 is the form of all Fourier type methods.
Although the advantages of an orthonormal basis are clear, there are cases
where the basis system dictated by the problem is not and cannot (or
should not) be made orthogonal. For these cases, one can still have the
expression of Equation 1 and one similar to Equation 2 by using a dual basis setf˜k(t)f˜k(t) whose elements are not orthogonal to
each other, but to the corresponding element of the expansion set
〈
f
ℓ
(
t
)
,
f
˜
k
(
t
)
〉
=
δ
(
ℓ
-
k
)
〈
f
ℓ
(
t
)
,
f
˜
k
(
t
)
〉
=
δ
(
ℓ
-
k
)
(4)Because this type of “orthogonality" requires two sets of vectors, the
expansion set and the dual set, the system is called biorthogonal.
Using Equation 4 with the expansion in Equation 1 gives
g
(
t
)
=
∑
k
〈
g
(
t
)
,
f
˜
k
(
t
)
〉
f
k
(
t
)
g
(
t
)
=
∑
k
〈
g
(
t
)
,
f
˜
k
(
t
)
〉
f
k
(
t
)
(5)Although a biorthogonal system is more complicated in that it requires,
not only the original expansion set, but the finding, calculating, and
storage of a dual set of vectors, it is very general and allows a larger
class of expansions. There may, however, be greater numerical problems
with a biorthogonal system if some of the basis vectors are strongly
correlated.
The calculation of the expansion coefficients using an inner product in
Equation 3 is called the analysis part of the complete process, and
the calculation of the signal from the coefficients and expansion vectors
in Equation 1 is called the synthesis part.
In finite dimensions, analysis and synthesis operations are simply
matrix–vector multiplications. If the expansion vectors in Equation 1
are a basis, the synthesis matrix has these basis vectors as columns and
the matrix is square and non singular. If the matrix is orthogonal, its
rows and columns are orthogonal, its inverse is its transpose, and the
identity operator is simply the matrix multiplied by its transpose. If it
is not orthogonal, then the identity is the matrix multiplied by its
inverse and the dual basis consists of the rows of the inverse. If the
matrix is singular, then its columns are not independent and, therefore,
do not form a basis.
Using a four dimensional space with matrices to illustrate the ideas of this
chapter, the synthesis formula g(t)=∑kakfk(t)g(t)=∑kakfk(t) becomes
g
(
0
)
g
(
1
)
g
(
2
)
g
(
3
)
=
a
0
f
0
(
0
)
f
0
(
1
)
f
0
(
2
)
f
0
(
3
)
+
a
1
f
1
(
0
)
f
1
(
1
)
f
1
(
2
)
f
1
(
3
)
+
a
2
f
2
(
0
)
f
2
(
1
)
f
2
(
2
)
f
2
(
3
)
+
a
3
f
3
(
0
)
f
3
(
1
)
f
3
(
2
)
f
3
(
3
)
g
(
0
)
g
(
1
)
g
(
2
)
g
(
3
)
=
a
0
f
0
(
0
)
f
0
(
1
)
f
0
(
2
)
f
0
(
3
)
+
a
1
f
1
(
0
)
f
1
(
1
)
f
1
(
2
)
f
1
(
3
)
+
a
2
f
2
(
0
)
f
2
(
1
)
f
2
(
2
)
f
2
(
3
)
+
a
3
f
3
(
0
)
f
3
(
1
)
f
3
(
2
)
f
3
(
3
)
(6)which can be compactly written in matrix form as
g
(
0
)
g
(
1
)
g
(
2
)
g
(
3
)
=
f
0
(
0
)
f
1
(
0
)
f
2
(
0
)
f
3
(
0
)
f
0
(
1
)
f
1
(
1
)
f
2
(
1
)
f
3
(
1
)
f
0
(
2
)
f
1
(
2
)
f
2
(
2
)
f
3
(
2
)
f
0
(
3
)
f
1
(
3
)
f
2
(
3
)
f
3
(
3
)
a
0
a
1
a
2
a
3
g
(
0
)
g
(
1
)
g
(
2
)
g
(
3
)
=
f
0
(
0
)
f
1
(
0
)
f
2
(
0
)
f
3
(
0
)
f
0
(
1
)
f
1
(
1
)
f
2
(
1
)
f
3
(
1
)
f
0
(
2
)
f
1
(
2
)
f
2
(
2
)
f
3
(
2
)
f
0
(
3
)
f
1
(
3
)
f
2
(
3
)
f
3
(
3
)
a
0
a
1
a
2
a
3
(7)The synthesis or expansion Equation 1 or Equation 7 becomes
with the left-hand column vector gg being the signal vector, the
matrix FF formed with the basis vectors fkfk as columns, and
the right-hand vector aa containing the four expansion
coefficients, akak.
The equation for calculating the kthkth expansion coefficient in
Equation 6 is
a
k
=
〈
g
(
t
)
,
f
˜
k
(
t
)
〉
=
f
˜
k
T
g
a
k
=
〈
g
(
t
)
,
f
˜
k
(
t
)
〉
=
f
˜
k
T
g
(9)which can be written in vector form as
a
0
a
1
a
2
a
3
=
f
˜
0
(
0
)
f
˜
0
(
1
)
f
˜
0
(
2
)
f
˜
0
(
3
)
f
˜
1
(
0
)
f
˜
1
(
1
)
f
˜
1
(
2
)
f
˜
1
(
3
)
f
˜
2
(
0
)
f
˜
2
(
1
)
f
˜
2
(
2
)
f
˜
2
(
3
)
f
˜
3
(
0
)
f
˜
3
(
1
)
f
˜
3
(
2
)
f
˜
3
(
3
)
g
(
0
)
g
(
1
)
g
(
2
)
g
(
3
)
a
0
a
1
a
2
a
3
=
f
˜
0
(
0
)
f
˜
0
(
1
)
f
˜
0
(
2
)
f
˜
0
(
3
)
f
˜
1
(
0
)
f
˜
1
(
1
)
f
˜
1
(
2
)
f
˜
1
(
3
)
f
˜
2
(
0
)
f
˜
2
(
1
)
f
˜
2
(
2
)
f
˜
2
(
3
)
f
˜
3
(
0
)
f
˜
3
(
1
)
f
˜
3
(
2
)
f
˜
3
(
3
)
g
(
0
)
g
(
1
)
g
(
2
)
g
(
3
)
(10)where each akak is an inner product of the kthkth row of F˜TF˜T with gg and analysis or coefficient
Equation 3 or Equation 10 becomes
a
=
F
˜
T
g
a
=
F
˜
T
g
(11)which together are Equation 2 or
g
=
F
F
˜
T
g
.
g
=
F
F
˜
T
g
.
(12)Therefore,
F
˜
T
=
F
-
1
F
˜
T
=
F
-
1
(13)is how the dual basis in Equation 4 is found.
If the columns of FF are orthogonal and normalized, then
F
F
T
=
I
.
F
F
T
=
I
.
(14)This means the basis and dual basis are the same, and Equation 12 and
Equation 13 become
g
=
F
F
T
g
g
=
F
F
T
g
(15)and
F
˜
T
=
F
T
F
˜
T
=
F
T
(16)which are both simpler and more numerically stable than Equation 13.
The discrete Fourier transform (DFT) is an interesting example of a finite
dimensional Fourier transform with orthogonal basis vectors where matrix
and vector techniques can be informative as to the DFT's characteristics
and properties. That can be found developed in several signal processing
books.
The Fourier Series is an excellent example of an infinite dimensional
composition (synthesis) and decomposition (analysis). The expansion
formula for an even function g(t)g(t) over 0<x<2π0<x<2π is
g
(
t
)
=
∑
k
a
k
cos
(
k
t
)
g
(
t
)
=
∑
k
a
k
cos
(
k
t
)
(17)where the basis vectors (functions) are
f
k
(
t
)
=
cos
(
k
t
)
f
k
(
t
)
=
cos
(
k
t
)
(18)and the expansion coefficients are obtained as
a
k
=
〈
g
(
t
)
,
f
k
(
t
)
〉
=
2
π
∫
0
π
g
(
t
)
cos
(
k
t
)
d
x
.
a
k
=
〈
g
(
t
)
,
f
k
(
t
)
〉
=
2
π
∫
0
π
g
(
t
)
cos
(
k
t
)
d
x
.
(19)The basis vector set is easily seen to be orthonormal by
verifying
〈
f
ℓ
(
t
)
,
f
k
(
t
)
〉
=
δ
(
k
-
ℓ
)
.
〈
f
ℓ
(
t
)
,
f
k
(
t
)
〉
=
δ
(
k
-
ℓ
)
.
(20)These basis functions span an infinite dimensional vector space and the
convergence of Equation 17 must be examined. Indeed, it is the
robustness of that convergence that is discussed in this section under the topic of
unconditional bases.
Another example of an infinite dimensional orthogonal basis is
Shannon's sampling expansion [9]. If f(t)f(t) is band limited, then
f
(
t
)
=
∑
k
f
(
T
k
)
sin
(
π
T
t
-
π
k
)
π
T
t
-
π
k
f
(
t
)
=
∑
k
f
(
T
k
)
sin
(
π
T
t
-
π
k
)
π
T
t
-
π
k
(21)for a sampling interval T<πWT<πW if the spectrum of f(t)f(t) is
zero for |ω|>W|ω|>W. In this case the basis functions are the sinc
functions with coefficients which are simply samples of the original
function. This means the inner product of a sinc basis function with a
bandlimited function will give a sample of that function. It is easy to
see that the sinc basis functions are orthogonal by taking the inner
product of two sinc functions which will sample one of them at the points
of value one or zero.
