Skip to content Skip to navigation

Connexions

You are here: Home » Content » Introduction to Concise Signal Models

Navigation

Recently Viewed

This feature requires Javascript to be enabled.

Introduction to Concise Signal Models

Module by: Michael Wakin. E-mail the author

User rating (How does the rating system work?)
Ratings

Ratings allow you to judge the quality of modules. If other users have ranked the module then its average rating is displayed below. Ratings are calculated on a scale from one star (Poor) to five stars (Excellent).

How to rate a module

Hover over the star that corresponds to the rating you wish to assign. Click on the star to add your rating. Your rating should be based on the quality of the content. You must have an account and be logged in to rate content.

:
(0 ratings)

Summary: This collection reviews fundamental concepts underlying the use of concise models for signal processing. Topics are presented from a geometric perspective and include low-dimensional linear, sparse, and manifold-based signal models, approximation, compression, dimensionality reduction, and Compressed Sensing.

Note: Your browser may not currently support MathML. See our browser support page for additional details. You can always view the correct math in the PDF version.

Overview

In characterizing a given problem in signal processing, one is often able to specify a model for the signals to be processed. This model may distinguish (either statistically or deterministically) classes of interesting signals from uninteresting ones, typical signals from anomalies, information from noise, etc.

Very commonly, models in signal processing deal with some notion of structure, constraint, or conciseness. Roughly speaking, one often believes that a signal has “few degrees of freedom” relative to the size of the signal. This notion of conciseness is a very powerful assumption, and it suggests the potential for dramatic gains via algorithms that capture and exploit the true underlying structure of the signal.

In these modules, we survey three common examples of concise models: linear models, sparse nonlinear models, and manifold-based models. In each case, we discuss an important phenomenon: the conciseness of the model corresponds to a low-dimensional geometric structure along which the signals of interest tend to cluster. This low-dimensional geometry again has important implications in the understanding and the development of efficient algorithms for signal processing.

We discuss this low-dimensional geometry in several contexts, including projecting a signal onto the model class (i.e., forming a concise approximation to a signal), encoding such an approximation (i.e., data compression), and reducing the dimensionality of signals and data sets. We conclude with an important and emerging application area known as Compressed Sensing (CS), which is a novel method for data acquisition that relies on concise models and builds upon strong geometric principles. We discuss CS in its traditional, sparsity-based context and also discuss extensions of CS to other concise models such as manifolds.

General Mathematical Preliminaries

Signal notation

We will treat signals as real- or complex-valued functions having domains that are either discrete (and finite) or continuous (and either compact or infinite). Each of these assumptions will be made clear as needed. As a general rule, however, we will use xx to denote a discrete signal in RNRN and ff to denote a function over a continuous domain DD. We also commonly refer to these as discrete- or continuous-time signals, though the domain need not actually be temporal in nature.

Lp and lp norms

As measures for signal energy, fidelity, or sparsity, we will employ the LpLp and ℓpℓp norms. For continuous-time functions, the LpLp norm is defined as

∥ f ∥ L p ( D ) = ∫ D | f | p 1 / p , p ∈ ( 0 , ∞ ) , ∥ f ∥ L p ( D ) = ∫ D | f | p 1 / p , p ∈ ( 0 , ∞ ) , (1)
and for discrete-time functions, the ℓpℓp norm is defined as
∥ x ∥ ℓ p = ( ∑ i = 1 N | x ( i ) | p ) 1 / p , p ∈ ( 0 , ∞ ) , max i = 1 , ⋯ , N | x ( i ) | , p = ∞ , ∑ i = 1 N 1 x ( i ) ≠ 0 , p = 0 , ∥ x ∥ ℓ p = ( ∑ i = 1 N | x ( i ) | p ) 1 / p , p ∈ ( 0 , ∞ ) , max i = 1 , ⋯ , N | x ( i ) | , p = ∞ , ∑ i = 1 N 1 x ( i ) ≠ 0 , p = 0 , (2)
where 11 denotes the indicator function. (While we often refer to these measures as “norms,” they actually do not meet the technical criteria for norms when p<1p<1.)

Linear algebra

Let AA be a real-valued M×NM×N matrix. We denote the nullspace of AA as N(A)N(A) (note that N(A)N(A) is a linear subspace of RNRN), and we denote the transpose of AA as ATAT.

We call AA an orthoprojector from RNRN to RMRM if it has orthonormal rows. From such a matrix we call ATAATA the corresponding orthogonal projection operator onto the MM-dimensional subspace of RNRN spanned by the rows of AA.

Content actions

Give Feedback:

E-mail the module author | Rate module ( How does the rating system work?)

Rating system

Ratings

Ratings allow you to judge the quality of modules. If other users have ranked the module then its average rating is displayed below. Ratings are calculated on a scale from one star (Poor) to five stars (Excellent).

How to rate a module

Hover over the star that corresponds to the rating you wish to assign. Click on the star to add your rating. Your rating should be based on the quality of the content. You must have an account and be logged in to rate content.

(0 ratings)

Download:

Module PDF

Add module to:

My Favorites (?)

'My Favorites' is a special kind of lens which you can use to bookmark modules and collections directly in Connexions. 'My Favorites' can only be seen by you, and collections saved in 'My Favorites' can remember the last module you were on. You need a Connexions account to use 'My Favorites'.

| A lens (?)

Definition of a lens

Lenses

A lens is a custom view of Connexions content. You can think of it as a fancy kind of list that will let you see Connexions through the eyes of organizations and people you trust.

What is in a lens?

Lens makers point to Connexions materials (modules and collections), creating a guide that includes their own comments and descriptive tags about the content.

Who can create a lens?

Any individual Connexions member, a community, or a respected organization.

What are tags? tag icon

Tags are descriptors added by lens makers to help label content, attaching a vocabulary that is meaningful in the context of the lens.

| External bookmarks