# Connexions

You are here: Home » Content » Compressive Sensing » Sparse Approximation and ℓp Spaces

### Lenses

What is a lens?

#### Definition of a lens

##### Lenses

A lens is a custom view of the content in the repository. You can think of it as a fancy kind of list that will let you see content through the eyes of organizations and people you trust.

##### What is in a lens?

Lens makers point to materials (modules and collections), creating a guide that includes their own comments and descriptive tags about the content.

##### Who can create a lens?

Any individual member, a community, or a respected organization.

##### What are tags?

Tags are descriptors added by lens makers to help label content, attaching a vocabulary that is meaningful in the context of the lens.

#### Affiliated with (What does "Affiliated with" mean?)

This content is either by members of the organizations listed or about topics related to the organizations listed. Click each link to see a list of all content affiliated with the organization.
• Rice Digital Scholarship

This collection is included in aLens by: Digital Scholarship at Rice University

Click the "Rice Digital Scholarship" link to see all content affiliated with them.

#### Also in these lenses

• Lens for Engineering

This module and collection are included inLens: Lens for Engineering
By: Sidney Burrus

Click the "Lens for Engineering" link to see all content selected in this lens.

### Recently Viewed

This feature requires Javascript to be enabled.

Inside Collection (Course):

# Sparse Approximation and ℓp Spaces

Module by: Marco F. Duarte, Ronald DeVore. E-mail the authors

We now look at how well fXfX can be approximated by nn functions in the dictionary DD.

## Definition 1

We define the error of nn-term approximation of ff by the elements of the dictionary DD as (1)σn(f)X:=σn(f,D)X:=infsΣnf-sX.(1)σn(f)X:=σn(f,D)X:=infsΣnf-sX.

We also define the class of rr-smooth signals in DD as(2)Ar:=Ar(D):={fX,σn(f)Mn-r  for some  M}(2)Ar:=Ar(D):={fX,σn(f)Mn-r  for some  M},

with the corresponding norm f A r = sup n = 1 , 2 , n r σ n ( f ) X f A r = sup n = 1 , 2 , n r σ n ( f ) X .

In general, the larger rr is, the 'smoother' the function sAr(D)XsAr(D)X. Note also that ArArArAr if r>rr>r. Given ff, let r(f)=sup{r:fAr}r(f)=sup{r:fAr} be a measure of the "smoothness" of ff, i.e. a quantification of compressibility.

Let X=HX=H, a Hilbert space1 such as X=L2(R)X=L2(R), and assume D=BD=B - an orthonormal basis on XX; i.e. if B={ϕi}iB={ϕi}i, then ϕi,ϕj=δi,jϕi,ϕj=δi,j, where δi,jδi,j is the Kronecker delta. This also means that each fXfX has an expansion f=jcj(f)ϕjf=jcj(f)ϕj, where cj(f)=f,ϕjcj(f)=f,ϕj. We also have fX2=j=1|cj(f)|2fX2=j=1|cj(f)|2.

Recall the definition of pp spaces: let (aj)R(aj)R; then (aj)p(aj)p if ( a j ) p < ( a j ) p < with ( a j ) p = ( j | a j | p ) 1 / p ( a j ) p = ( j | a j | p ) 1 / p for p<p< and ( a j ) p = sup j | a j | ( a j ) p = sup j | a j | for p=p=. We also recall that for LpLp spaces on compact sets, LpLpLpLp if p>pp>p. The opposite is true for pp spaces: pppp if p<pp<p. Hence, the smaller the value of pp is, the “smaller” pp is.

## Example 1

Does there exist a sequence (aj)(aj) with ( a j ) 1 =j|aj|< ( a j ) 1 =j|aj|< but with ( a j ) p =(j|aj|p)1p= ( a j ) p =(j|aj|p)1p= for all 0<p<10<p<1? Consider the sequence an=1n(logn)1+δan=1n(logn)1+δ. We see that (an)1(an)1 but ( a n ) p = ( a n ) p = for all 0<p<10<p<1.

## Lemma 1

A sequence (an)(an) is in pp if the sorted magnitudes of the anan decay faster than n-1pn-1p.

Define an*an* as the element of the sequence (an)(an) with the nthnth largest magnitude, and denote (an*)(an*) as the decreasing rearrangement of (an)(an). It is easy to show that k ( a k * ) p n ( a n ) p k ( a k * ) p n ( a n ) p for all kk; also, if (an)p(an)p, then a k * ( a n ) p k - 1 p a k * ( a n ) p k - 1 p .

## Definition 2

A sequence (an)(an) is in weak pp, denoted (an)wp(an)wp, if ak*Mk-1pak*Mk-1p. We also define the quasinorm 2 ( a n ) w p ( a n ) w p as the smallest M>0M>0 such that ak*Mk-1pak*Mk-1p for each kk.

## Example 2

The sequence an=1nan=1n is in weak 11 but not in 11.

## Lemma 2

For pp,pp such that p>pp>p, we have pwpppwpp.

## Theorem 1

Let D=BD=B be an orthonormal basis for the Hilbert space X=HX=H. For fXfX with representation in B=[ϕ1,ϕ2,]B=[ϕ1,ϕ2,] as f=ncn(f)ϕnf=ncn(f)ϕn, we have fAr(B)XfAr(B)X if and only if the sequence (cn(f))wτ(cn(f))wτ, with 1τ=r+121τ=r+12. Moreover, there exist C0,C0RC0,C0R such that C0(cn(f))wτfArC0(cn(f))wτC0(cn(f))wτfArC0(cn(f))wτ.

### Example

Let r=12r=12. fA12fA12 if and only if (cn(f))wτ (cn(f))wτ , i.e. if cn*(f)Mn-1=Mncn*(f)Mn-1=Mn.

### Proof

We prove the converse statement; the forward statement proof is left to the reader. We would like to show that if (cn(f))wτ(cn(f))wτ, then fArfAr, with r=1τ-12r=1τ-12. The best nn-term approximation of ff in BB is of the form s=kΛakϕk,Λns=kΛakϕk,Λn. Therefore, we have: (3) σ n ( f ) X = inf s Σ n f - s X = inf s Σ n k Λ ( c k ( f ) - a k ) ϕ k + k Λ c k ( f ) ϕ k X = inf s Λ k Λ ( c k ( f ) - a k ) 2 + k Λ ( c k ( f ) ) 2 = k = n + 1 | c k * ( f ) | 2 M 2 k = n + 1 k - 2 τ M 2 k = n + 1 k - 2 r - 1  (since  ( c n ( f ) ) w p ) , (3) σ n ( f ) X = inf s Σ n f - s X = inf s Σ n k Λ ( c k ( f ) - a k ) ϕ k + k Λ c k ( f ) ϕ k X = inf s Λ k Λ ( c k ( f ) - a k ) 2 + k Λ ( c k ( f ) ) 2 = k = n + 1 | c k * ( f ) | 2 M 2 k = n + 1 k - 2 τ M 2 k = n + 1 k - 2 r - 1  (since  ( c n ( f ) ) w p ) ,

where M:=(cn(f)wpM:=(cn(f)wp.

We prove the converse statement; the forward statement proof is left to the reader. We would like to show that if (cn(f))wτ(cn(f))wτ, then fArfAr, with r=1τ-12r=1τ-12. The best nn-term approximation of ff in BB is of the form s=kΛakϕk,Λns=kΛakϕk,Λn. Therefore, we have: (3) σ n ( f ) X = inf s Σ n f - s X = inf s Σ n k Λ ( c k ( f ) - a k ) ϕ k + k Λ c k ( f ) ϕ k X = inf s Λ k Λ ( c k ( f ) - a k ) 2 + k Λ ( c k ( f ) ) 2 = k = n + 1 | c k * ( f ) | 2 M 2 k = n + 1 k - 2 τ M 2 k = n + 1 k - 2 r - 1  (since  ( c n ( f ) ) w p ) , (3) σ n ( f ) X = inf s Σ n f - s X = inf s Σ n k Λ ( c k ( f ) - a k ) ϕ k + k Λ c k ( f ) ϕ k X = inf s Λ k Λ ( c k ( f ) - a k ) 2 + k Λ ( c k ( f ) ) 2 = k = n + 1 | c k * ( f ) | 2 M 2 k = n + 1 k - 2 τ M 2 k = n + 1 k - 2 r - 1  (since  ( c n ( f ) ) w p ) ,

where we define C=λ0rC=λ0r. Using this result in the earlier statement, we get(5)k=n+1|ck*(f)|2CM2n-2rM2zn-r;(5)k=n+1|ck*(f)|2CM2n-2rM2zn-r;

this implies by definition that (ck(f))Ar(ck(f))Ar.

## Footnotes

1. A Hilbert space is a complete inner product space with the norm induced by the inner product
2. A quasinorm is has the properties of a norm except that the triangle inequality is replaced by the condition x + y C 0 [ x + y ] x + y C 0 [ x + y ] for some absolute constant C0C0.

## Content actions

PDF | EPUB (?)

### What is an EPUB file?

EPUB is an electronic book format that can be read on a variety of mobile devices.

PDF | EPUB (?)

### What is an EPUB file?

EPUB is an electronic book format that can be read on a variety of mobile devices.

#### Collection to:

My Favorites (?)

'My Favorites' is a special kind of lens which you can use to bookmark modules and collections. 'My Favorites' can only be seen by you, and collections saved in 'My Favorites' can remember the last module you were on. You need an account to use 'My Favorites'.

| A lens I own (?)

#### Definition of a lens

##### Lenses

A lens is a custom view of the content in the repository. You can think of it as a fancy kind of list that will let you see content through the eyes of organizations and people you trust.

##### What is in a lens?

Lens makers point to materials (modules and collections), creating a guide that includes their own comments and descriptive tags about the content.

##### Who can create a lens?

Any individual member, a community, or a respected organization.

##### What are tags?

Tags are descriptors added by lens makers to help label content, attaching a vocabulary that is meaningful in the context of the lens.

| External bookmarks

#### Module to:

My Favorites (?)

'My Favorites' is a special kind of lens which you can use to bookmark modules and collections. 'My Favorites' can only be seen by you, and collections saved in 'My Favorites' can remember the last module you were on. You need an account to use 'My Favorites'.

| A lens I own (?)

#### Definition of a lens

##### Lenses

A lens is a custom view of the content in the repository. You can think of it as a fancy kind of list that will let you see content through the eyes of organizations and people you trust.

##### What is in a lens?

Lens makers point to materials (modules and collections), creating a guide that includes their own comments and descriptive tags about the content.

##### Who can create a lens?

Any individual member, a community, or a respected organization.

##### What are tags?

Tags are descriptors added by lens makers to help label content, attaching a vocabulary that is meaningful in the context of the lens.

| External bookmarks