We previously described Shannon's Theorem plus encoding: the
Nyquist sampling rate is the minimal required sampling rate to
recover the entire class of bandlimited signals. We have seen
that this sampling rate may be prohibitively large for
broadband signals. We see a way to improve upon this situation:
we will pose a different model for the signals which is more
restrictive than the assumption that the signals are bandlimited.
Fortunately, there are several real world scenarios in which one
knows much more information about the signals of interest. For
example, they may be written in terms of very few fundamental
building blocks (such as sine waves or chirps). This leads us to define new signal classes based on notions of sparsity and seek to determine if we can improve on sampling and encoding in this new setting.
Let us define the general setting for this section. Let
XX be a Banach space of functions. The typical examples
are X=Lp(R),Lp(Rd),Lp(-T,T)X=Lp(R),Lp(Rd),Lp(-T,T),
1≤p≤∞1≤p≤∞. We denote the norm on XX by
∥⋅∥X∥⋅∥X. We define a dictionary DD as
any collection of functions D⊆XD⊆X
such that ∥g∥X=1∥g∥X=1 for all g∈Dg∈D, i.e.
all the elements of the dictionary are normalized. While the
definition is very broad, in practice dictionaries usually have
more structure. Some examples include D=BD=B, a basis for
XX, such as (i) the Fourier basis on
[-π,π][-π,π], (ii) a wavelet basis, (iii) redundant
families of waveforms of the form ψa,b,σ=e-a(t-b)2eiσxψa,b,σ=e-a(t-b)2eiσx, i.e. D={ψa,b,σ}a,b,σD={ψa,b,σ}a,b,σ, and (iv) wavelet
packets.
We define the class of nn-sparse signals as
Σn:=Σn(D)={s=∑g∈Λcgg,Λ⊆D,♯Λ≤n}Σn:=Σn(D)={s=∑g∈Λcgg,Λ⊆D,♯Λ≤n}. We also
say that ss has sparsity nn in DD if s∈Σn(D)s∈Σn(D), i.e. if it can be written as the linear
combination of nn functions from DD. We note that
ΣnΣn is not a linear space; we instead have
Σn+Σn⊆Σ2nΣn+Σn⊆Σ2n.