Skip to content Skip to navigation Skip to collection information

OpenStax-CNX

You are here: Home » Content » An Introduction to Compressive Sensing » Linear regression and model selection

Navigation

Recently Viewed

This feature requires Javascript to be enabled.
 

Linear regression and model selection

Module by: Mark A. Davenport. E-mail the author

Summary: This module provides a brief overview of the relationship between model selection, sparse linear regression, and the techniques developed in compressive sensing.

Many of the sparse recovery algorithms we have described so far in this course were originally developed to address the problem of sparse linear regression and model selection in statistics. In this setting we are given some data consisting of a set of input variables and response variables. We will suppose that there are a total of NN input variables, and we observe a total of MM input and response pairs. We can represent the set of input variable observations as an M×NM×N matrix ΦΦ, and the set of response variable observations as an M×1M×1 vector yy.

In linear regression, it is assumed that yy can be approximated as a linear function of the input variables, i.e., there exists an xx such that yΦxyΦx. However, when the number of input variables is large compared to the number of observations, i.e., MNMN, this becomes extremely challenging because we wish to estimate NN parameters from far fewer than NN observations. In general this would be impossible to overcome, but in practice it is common that only a few input variables are actually necessary to predict the response variable. In this case the xx that we wish to estimate is sparse, and we can apply all of the techniques that we have learned so far for sparse recovery to estimate xx. In this setting, not only does sparsity aid us in our goal of obtaining a regression, but it also performs model selection by identifying the most relevant variables in predicting the response.

Collection Navigation

Content actions

Download:

Collection as:

PDF | EPUB (?)

What is an EPUB file?

EPUB is an electronic book format that can be read on a variety of mobile devices.

Downloading to a reading device

For detailed instructions on how to download this content's EPUB to your specific device, click the "(?)" link.

| More downloads ...

Module as:

PDF | More downloads ...

Add:

Collection to:

My Favorites (?)

'My Favorites' is a special kind of lens which you can use to bookmark modules and collections. 'My Favorites' can only be seen by you, and collections saved in 'My Favorites' can remember the last module you were on. You need an account to use 'My Favorites'.

| A lens I own (?)

Definition of a lens

Lenses

A lens is a custom view of the content in the repository. You can think of it as a fancy kind of list that will let you see content through the eyes of organizations and people you trust.

What is in a lens?

Lens makers point to materials (modules and collections), creating a guide that includes their own comments and descriptive tags about the content.

Who can create a lens?

Any individual member, a community, or a respected organization.

What are tags? tag icon

Tags are descriptors added by lens makers to help label content, attaching a vocabulary that is meaningful in the context of the lens.

| External bookmarks

Module to:

My Favorites (?)

'My Favorites' is a special kind of lens which you can use to bookmark modules and collections. 'My Favorites' can only be seen by you, and collections saved in 'My Favorites' can remember the last module you were on. You need an account to use 'My Favorites'.

| A lens I own (?)

Definition of a lens

Lenses

A lens is a custom view of the content in the repository. You can think of it as a fancy kind of list that will let you see content through the eyes of organizations and people you trust.

What is in a lens?

Lens makers point to materials (modules and collections), creating a guide that includes their own comments and descriptive tags about the content.

Who can create a lens?

Any individual member, a community, or a respected organization.

What are tags? tag icon

Tags are descriptors added by lens makers to help label content, attaching a vocabulary that is meaningful in the context of the lens.

| External bookmarks