Skip to content Skip to navigation Skip to collection information

OpenStax_CNX

You are here: Home » Content » An Introduction to Compressive Sensing » Hyperspectral imaging

Navigation

Recently Viewed

This feature requires Javascript to be enabled.
 

Hyperspectral imaging

Module by: Marco F. Duarte. E-mail the author

Summary: This module provides an overview of architectures and methods for hyperspectral imaging using the ideas of compressive sensing.

Standard digital color images of a scene of interest consist of three components – red, green and blue – which contain the intensity level for each of the pixels in three different groups of wavelengths. This concept has been extended in the hyperspectral and multispectral imaging sensing modalities, where the data to be acquired consists of a three-dimensional datacube that has two spatial dimensions xx and yy and one spectral dimension λλ.

In simple terms, a datacube is a 3-D function f(x,y,λ)f(x,y,λ) that can be represented as a stacking of intensities of the scene at different wavelengths. An example datacube is shown in Figure 1. Each of its entries is called a voxel. We also define a pixel's spectral signature as the stacking of its voxels in the spectral dimension f(x,y)={f(x,y,λ)}λf(x,y)={f(x,y,λ)}λ. The spectral signature of a pixel can give a wealth of information about the corresponding point in the scene that is not captured by its color. For example, using spectral signatures, it is possible to identify the type of material observed (for example, vegetation vs. ground vs. water), or its chemical composition.

Datacubes are high-dimensional, since the standard number of pixels present in a digitized image is multiplied by the number of spectral bands desired. However, considerable structure is present in the observed data. The spatial structure common in natural images is also observed in hyperspectral imaging, while each pixel's spectral signature is usually smooth.

Figure 1: Example hyperspectral datacube, with labeled dimensions.
Figure 1 (datacube.png)

Compressive sensing (CS) architectures for hyperspectral imaging perform lower-dimensional projections that multiplex in the spatial domain, the spectral domain, or both. Below, we detail three example architectures, as well as three possible models to sparsify hyperspectral datacubes.

Compressive hyperspectral imaging architectures

Single pixel hyperspectral camera

The single pixel camera uses a single photodetector to record random projections of the light emanated from the image, with the different random projections being captured in sequence. A single pixel hyperspectral camera requires a light modulating element that is reflective across the wavelengths of interest, as well as a sensor that can record the desired spectral bands separately [3]. A block diagram is shown in Figure 2.

The single sensor consists of a single spectrometer that spans the necessary wavelength range, which replaces the photodiode. The spectrometer records the intensity of the light reflected by the modulator in each wavelength. The same digital micromirror device (DMD) provides reflectivity for wavelengths from near infrared to near ultraviolet. Thus, by converting the datacube into a vector sorted by spectral band, the matrix that operates on the data to obtain the CS measurements is represented as

Φ = Φ x , y 0 0 0 Φ x , y 0 0 0 Φ x , y . Φ = Φ x , y 0 0 0 Φ x , y 0 0 0 Φ x , y .
(1)

This architecture performs multiplexing only in the spatial domain, i.e. dimensions xx and yy, since there is no mixing of the different spectral bands along the dimension λλ.

Figure 2: Block diagram for a single pixel hyperspectral camera. The photodiode is replaced by a spectrometer that captures the modulated light intensity for all spectral bands, for each of the CS measurements.
Figure 2 (BlockDiagram3.png)

Dual disperser coded aperture snapshot spectral imager

The dual disperser coded aperture snapshot spectral imager (DD-CASSI), shown in Figure 3, is an architecture that combines separate multiplexing in the spatial and spectral domain, which is then sensed by a wide-wavelength sensor/pixel array, thus flattening the spectral dimension [2].

First, a dispersive element separates the different spectral bands, which still overlap in the spatial domain. In simple terms, this element shears the datacube, with each spectral slice being displaced from the previous by a constant amount in the same spatial dimension. The resulting datacube is then masked using the coded aperture, whose effect is to "punch holes" in the sheared datacube by blocking certain pixels of light. Subsequently, a second dispersive element acts on the masked, sheared datacube; however, this element shears in the opposite direction, effectively inverting the shearing of the first dispersive element. The resulting datacube is upright, but features "sheared" holes of datacube voxels that have been masked out.

The resulting modified datacube is then received by a sensor array, which flattens the spectral dimension by measuring the sum of all the wavelengths received; the received light field resembles the target image, allowing for optical adjustments such as focusing. In this way, the measurements consist of full sampling in the spatial xx and yy dimensions, with an aggregation effect in the spectral λλ dimension.

Figure 3: Dual disperser coded aperture snapshot spectral imager (DD-CASSI). (a) Schematic of the DD-CASSI components. (b) Illustration of the datacube processing performed by the components.
(a) (b)
Figure 3(a) (ddcassischematic2.png)Figure 3(b) (ddcassisensing.png)

Single disperser coded aperture snapshot spectral imager

The single disperser coded aperture snapshot spectral imager (SD-CASSI), shown in Figure 4, is a simplification of the DD-CASSI architecture in which the first dispersive element is removed [4]. Thus, the light field received at the sensors does not resemble the target image. Furthermore, since the shearing is not reversed, the area occupied by the sheared datacube is larger than that of the original datacube, requiring a slightly larger number of pixels for the capture.

Figure 4: Single disperser coded aperture snapshot spectral imager (SD-CASSI). (a) Schematic of the SD-CASSI components. (b) Illustration of the datacube processing performed by the components.
(a) (b)
Figure 4(a) (sdcassischematic2.png)Figure 4(b) (sdcassisensing.png)

Sparsity structures for hyperspectral datacubes

Dyadic Multiscale Partitioning

This sparsity structure assumes that the spectral signature for all pixels in a neighborhood is close to constant; that is, that the datacube is piecewise constant with smooth borders in the spatial dimensions. The complexity of an image is then given by the number of spatial dyadic squares with constant spectral signature necessary to accurately approximate the datacube; see Figure 5. A reconstruction algorithm then searches for the signal of lowest complexity (i.e., with the fewest dyadic squares) that generates compressive measurements close to those observed [2].

Figure 5: Example dyadic square partition for piecewise spatially constant datacube.
Figure 5 (dyadic.png)

Spatial-only sparsity

This sparsity structure operates on each spectral band separately and assumes the same type of sparsity structure for each band [4]. The sparsity basis is drawn from those commonly used in images, such as wavelets, curvelets, or the discrete cosine basis. Since each basis operates only on a band, the resulting sparsity basis for the datacube can be represented as a block-diagonal matrix:

Ψ = Ψ x , y 0 0 0 Ψ x , y 0 0 0 Ψ x , y . Ψ = Ψ x , y 0 0 0 Ψ x , y 0 0 0 Ψ x , y .
(2)

Kronecker product sparsity

This sparsity structure employs separate sparsity bases for the spatial dimensions and the spectral dimension, and builds a sparsity basis for the datacube using the Kronecker product of these two [1]:

Ψ = Ψ λ Ψ x , y = Ψ λ [ 1 , 1 ] Ψ x , y Ψ λ [ 1 , 2 ] Ψ x , y Ψ λ [ 2 , 1 ] Ψ x , y Ψ λ [ 2 , 2 ] Ψ x , y . Ψ = Ψ λ Ψ x , y = Ψ λ [ 1 , 1 ] Ψ x , y Ψ λ [ 1 , 2 ] Ψ x , y Ψ λ [ 2 , 1 ] Ψ x , y Ψ λ [ 2 , 2 ] Ψ x , y .
(3)

In this manner, the datacube sparsity bases simultaneously enforces both spatial and spectral structure, potentially achieving a sparsity level lower than the sums of the spatial sparsities for the separate spectral slices, depending on the level of structure between them and how well can this structure be captured through sparsity.

Summary

Compressive sensing will make the largest impact in applications with very large, high dimensional datasets that exhibit considerable amounts of structure. Hyperspectral imaging is a leading example of such applications; the sensor architectures and data structure models surveyed in this module show initial promising work in this new direction, enabling new ways of simultaneously sensing and compressing such data. For standard sensing architectures, the data structures surveyed also enable new transform coding-based compression schemes.

References

  1. Duarte, M. and Baraniuk, R. (2009). Kronecker Compressive Sensing. [Preprint].
  2. Gehm, M. and John, R. and Brady, D. and Willett, R. and Schultz, T. (2007). Single-shot compressive spectral imaging with a dual disperser architecture. Optics Express, 15(21), 14013–14027.
  3. Kelly, K. and Takhar, D. and Sun, T. and Laska, J. and Duarte, M. and Baraniuk, R. (2007, Mar.). Compressed Sensing for Multispectral and Confocal Microscopy. In Proc. American Physical Society Meeting. Denver, CO
  4. Wagadarikar, A. and John, R. and Willett, R. and Brady, D. (2008). Single disperser design for coded aperture snapshot spectral imaging. Appl. Optics, 47(10), B44–51.

Collection Navigation

Content actions

Download:

Collection as:

PDF | EPUB (?)

What is an EPUB file?

EPUB is an electronic book format that can be read on a variety of mobile devices.

Downloading to a reading device

For detailed instructions on how to download this content's EPUB to your specific device, click the "(?)" link.

| More downloads ...

Module as:

PDF | EPUB (?)

What is an EPUB file?

EPUB is an electronic book format that can be read on a variety of mobile devices.

Downloading to a reading device

For detailed instructions on how to download this content's EPUB to your specific device, click the "(?)" link.

| More downloads ...

Add:

Collection to:

My Favorites (?)

'My Favorites' is a special kind of lens which you can use to bookmark modules and collections. 'My Favorites' can only be seen by you, and collections saved in 'My Favorites' can remember the last module you were on. You need an account to use 'My Favorites'.

| A lens I own (?)

Definition of a lens

Lenses

A lens is a custom view of the content in the repository. You can think of it as a fancy kind of list that will let you see content through the eyes of organizations and people you trust.

What is in a lens?

Lens makers point to materials (modules and collections), creating a guide that includes their own comments and descriptive tags about the content.

Who can create a lens?

Any individual member, a community, or a respected organization.

What are tags? tag icon

Tags are descriptors added by lens makers to help label content, attaching a vocabulary that is meaningful in the context of the lens.

| External bookmarks

Module to:

My Favorites (?)

'My Favorites' is a special kind of lens which you can use to bookmark modules and collections. 'My Favorites' can only be seen by you, and collections saved in 'My Favorites' can remember the last module you were on. You need an account to use 'My Favorites'.

| A lens I own (?)

Definition of a lens

Lenses

A lens is a custom view of the content in the repository. You can think of it as a fancy kind of list that will let you see content through the eyes of organizations and people you trust.

What is in a lens?

Lens makers point to materials (modules and collections), creating a guide that includes their own comments and descriptive tags about the content.

Who can create a lens?

Any individual member, a community, or a respected organization.

What are tags? tag icon

Tags are descriptors added by lens makers to help label content, attaching a vocabulary that is meaningful in the context of the lens.

| External bookmarks