Skip to content Skip to navigation Skip to collection information

OpenStax-CNX

You are here: Home » Content » Introduction to Compressive Sensing » Single-pixel camera

Navigation

Recently Viewed

This feature requires Javascript to be enabled.
 

Single-pixel camera

Module by: Marco F. Duarte, Mark A. Davenport. E-mail the authors

Summary: This module describes the application of compressive sensing to the design of a novel imaging architecture called the "single-pixel camera".

Architecture

Several hardware architectures have been proposed that apply the theory of compressive sensing (CS) in an imaging setting [4], [5], [6]. We will focus on the so-called single-pixel camera [4], [7], [8], [9], [10]. The single-pixel camera is an optical computer that sequentially measures the inner products y[j]=x,φjy[j]=x,φj between an NN-pixel sampled version of the incident light-field from the scene under view (denoted by xx) and a set of NN-pixel test functions {φj}j=1M{φj}j=1M. The architecture is illustrated in Figure 1, and an aerial view of the camera in the lab is shown in Figure 2. As shown in these figures, the light-field is focused by a lens (Lens 1 in Figure 2) not onto a CCD or CMOS sampling array but rather onto a spatial light modulator (SLM). An SLM modulates the intensity of a light beam according to a control signal. A simple example of a transmissive SLM that either passes or blocks parts of the beam is an overhead transparency. Another example is a liquid crystal display (LCD) projector.

Figure 1: Single-pixel camera block diagram. Incident light-field (corresponding to the desired image xx) is reflected off a digital micromirror device (DMD) array whose mirror orientations are modulated according to the pseudorandom pattern φjφj supplied by a random number generator. Each different mirror pattern produces a voltage at the single photodiode that corresponds to one measurement y[j]y[j].
Figure 1 (BlockDiagram2.png)

The Texas Instruments (TI) digital micromirror device (DMD) is a reflective SLM that selectively redirects parts of the light beam. The DMD consists of an array of bacterium-sized, electrostatically actuated micro-mirrors, where each mirror in the array is suspended above an individual static random access memory (SRAM) cell. Each mirror rotates about a hinge and can be positioned in one of two states (±10±10 degrees from horizontal) according to which bit is loaded into the SRAM cell; thus light falling on the DMD can be reflected in two directions depending on the orientation of the mirrors.

Each element of the SLM corresponds to a particular element of φjφj (and its corresponding pixel in xx). For a given φjφj, we can orient the corresponding element of the SLM either towards (corresponding to a 1 at that element of φjφj) or away from (corresponding to a 0 at that element of φjφj) a second lens (Lens 2 in Figure 2). This second lens collects the reflected light and focuses it onto a single photon detector (the single pixel) that integrates the product of xx and φjφj to compute the measurement y[j]=x,φjy[j]=x,φj as its output voltage. This voltage is then digitized by an A/D converter. Values of φjφj between 0 and 1 can be obtained by dithering the mirrors back and forth during the photodiode integration time. By reshaping xx into a column vector and the φjφj into row vectors, we can thus model this system as computing the product y=Φxy=Φx, where each row of ΦΦ corresponds to a φjφj. To compute randomized measurements, we set the mirror orientations φjφj randomly using a pseudorandom number generator, measure y[j]y[j], and then repeat the process MM times to obtain the measurement vector yy.

Figure 2: Aerial view of the single-pixel camera in the lab.
Figure 2 (fig1.png)

The single-pixel design reduces the required size, complexity, and cost of the photon detector array down to a single unit, which enables the use of exotic detectors that would be impossible in a conventional digital camera. Example detectors include a photomultiplier tube or an avalanche photodiode for low-light (photon-limited) imaging, a sandwich of several photodiodes sensitive to different light wavelengths for multimodal sensing, a spectrometer for hyperspectral imaging, and so on.

In addition to sensing flexibility, the practical advantages of the single-pixel design include the facts that the quantum efficiency of a photodiode is higher than that of the pixel sensors in a typical CCD or CMOS array and that the fill factor of a DMD can reach 90% whereas that of a CCD/CMOS array is only about 50%. An important advantage to highlight is that each CS measurement receives about N/2N/2 times more photons than an average pixel sensor, which significantly reduces image distortion from dark noise and read-out noise.

The single-pixel design falls into the class of multiplex cameras. The baseline standard for multiplexing is classical raster scanning, where the test functions {φj}{φj} are a sequence of delta functions δ[n-j]δ[n-j] that turn on each mirror in turn. There are substantial advantages to operating in a CS rather than raster scan mode, including fewer total measurements (MM for CS rather than NN for raster scan) and significantly reduced dark noise. See [4] for a more detailed discussion of these issues.

Figure 3 (a) and (b) illustrates a target object (a black-and-white printout of an “R”) xx and reconstructed image x^x^ taken by the single-pixel camera prototype in Figure 2 using N=256×256N=256×256 and M=N/50M=N/50[4]. Figure 3(c) illustrates an N=256×256N=256×256 color single-pixel photograph of a printout of the Mandrill test image taken under low-light conditions using RGB color filters and a photomultiplier tube with M=N/10M=N/10. In both cases, the images were reconstructed using total variation minimization, which is closely related to wavelet coefficient 11 minimization [3].

Figure 3: Sample image reconstructions from single-pixel camera. (a) 256×256256×256 conventional image of a black-and-white “R”. (b) Image reconstructed from M=1300M=1300 single-pixel camera measurements (50×50× sub-Nyquist). (c) 256×256256×256 pixel color reconstruction of a printout of the Mandrill test image imaged in a low-light setting using a single photomultiplier tube sensor, RGB color filters, and M=6500M=6500 random measurements.
(a) (b) (c)
Figure 3(a) (fig2a.png)Figure 3(b) (fig2b.png)Figure 3(c) (fig2c.png)

Discrete formulation

Since the DMD array is programmable, we can employ arbitrary test functions φjφj. However, even when we restrict the φjφj to be {0,1}{0,1}-valued, storing these patterns for large values of NN is impractical. Furthermore, as noted above, even pseudorandom ΦΦ can be computationally problematic during recovery. Thus, rather than purely random ΦΦ, we can also consider ΦΦ that admit a fast transform-based implementation by taking random submatrices of a Walsh, Hadamard, or noiselet transform [1], [2]. We will describe the Walsh transform for the purpose of illustration.

We will suppose that NN is a power of 2 and let Wlog2NWlog2N denote the N×NN×N Walsh transform matrix. We begin by setting W0=1W0=1, and we now define WjWj recursively as

W j = 1 2 W j - 1 W j - 1 W j - 1 - W j - 1 . W j = 1 2 W j - 1 W j - 1 W j - 1 - W j - 1 .
(1)

This construction produces an orthonormal matrix with entries of ±1/N±1/N that admits a fast implementation requiring O(NlogN)O(NlogN) computations to apply. As an example, note that

W 1 = 1 2 1 1 1 - 1 W 1 = 1 2 1 1 1 - 1
(2)

and

W 2 = 1 2 1 1 1 1 1 - 1 1 - 1 1 1 - 1 - 1 1 - 1 - 1 1 . W 2 = 1 2 1 1 1 1 1 - 1 1 - 1 1 1 - 1 - 1 1 - 1 - 1 1 .
(3)

We can exploit these constructions as follows. Suppose that N=2BN=2B and generate WBWB. Let IΓIΓ denote a M×NM×N submatrix of the identity II obtained by picking a random set of MM rows, so that IΓWBIΓWB is the submatrix of WBWB consisting of the rows of WBWB indexed by ΓΓ. Furthermore, let DD denote a random N×NN×N permutation matrix. We can generate ΦΦ as

Φ = 1 2 N I Γ W B + 1 2 D . Φ = 1 2 N I Γ W B + 1 2 D .
(4)

Note that 12NIΓWB+1212NIΓWB+12 merely rescales and shifts IΓWBIΓWB to have {0,1}{0,1}-valued entries, and recall that each row of ΦΦ will be reshaped into a 2-D matrix of numbers that is then displayed on the DMD array. Furthermore, DD can be thought of as either permuting the pixels or permuting the columns of WBWB. This step adds some additional randomness since some of the rows of the Walsh matrix are highly correlated with coarse scale wavelet basis functions — but permuting the pixels eliminates this structure. Note that at this point we do not have any strict guarantees that such ΦΦ combined with a wavelet basis ΨΨ will yield a product ΦΨΦΨ satisfying the restricted isometry property, but this approach seems to work well in practice.

References

  1. Coifman, R. and Geshwind, F. and Meyer, Y. (2001). Noiselets. Appl. Comput. Harmon. Anal., 10, 27–44.
  2. Candès, E. and Romberg, J. (2007). Sparsity and Incoherence in Compressive Sampling. Inverse Problems, 23(3), 969–985.
  3. Candès, E. and Tao, T. (2006). Near Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? IEEE Trans. Inform. Theory, 52(12), 5406–5425.
  4. Duarte, M. and Davenport, M. and Takhar, D. and Laska, J. and Sun, T. and Kelly, K. and Baraniuk, R. (2008). Single-Pixel Imaging via Compressive Sampling. IEEE Signal Processing Mag., 25(2), 83–91.
  5. Marcia, R. and Harmany, Z. and Willett, R. (2009, Jan.). Compressive coded aperture imaging. In Proc. IS&T/SPIE Symp. Elec. Imag.: Comp. Imag. San Jose, CA
  6. Robucci, R. and Chiu, L. and Gray, J. and Romberg, J. and Hasler, P. and Anderson, D. (2008, Apr.). Compressive sensing on a CMOS separable transform image sensor. In Proc. IEEE Int. Conf. Acoust., Speech, and Signal Processing (ICASSP). Las Vegas, NV
  7. Takhar, D. and Laska, J. and Wakin, M. and Duarte, M. and Baron, D. and Kelly, K. and Baraniuk, R. (2006, Jan.). A Compressed Sensing Camera: New Theory and an Implementation using Digital Micromirrors. In Proc. IS&T/SPIE Symp. Elec. Imag.: Comp. Imag. San Jose, CA
  8. Takhar, D. and Laska, J. and Wakin, M. and Duarte, M. and Baron, D. and Sarvotham, S. and Kelly, K. and Baraniuk, R. (2006, Jan.). A New Compressive Imaging Camera Architecture using Optical-Domain Compression. In Proc. IS&T/SPIE Symp. Elec. Imag.: Comp. Imag. San Jose, CA
  9. Wakin, M. and Laska, J. and Duarte, M. and Baron, D. and Sarvotham, S. and Takhar, D. and Kelly, K. and Baraniuk, R. (2006, Oct.). An Architecture for Compressive Imaging. In Proc. IEEE Int. Conf. Image Processing (ICIP). Atlanta, GA
  10. Wakin, M. and Laska, J. and Duarte, M. and Baron, D. and Sarvotham, S. and Takhar, D. and Kelly, K. and Baraniuk, R. (2006, Apr.). Compressive Imaging for Video Representation and Coding. In Proc. Picture Coding Symp. Beijing, China

Collection Navigation

Content actions

Download:

Collection as:

PDF | EPUB (?)

What is an EPUB file?

EPUB is an electronic book format that can be read on a variety of mobile devices.

Downloading to a reading device

For detailed instructions on how to download this content's EPUB to your specific device, click the "(?)" link.

| More downloads ...

Module as:

PDF | More downloads ...

Add:

Collection to:

My Favorites (?)

'My Favorites' is a special kind of lens which you can use to bookmark modules and collections. 'My Favorites' can only be seen by you, and collections saved in 'My Favorites' can remember the last module you were on. You need an account to use 'My Favorites'.

| A lens I own (?)

Definition of a lens

Lenses

A lens is a custom view of the content in the repository. You can think of it as a fancy kind of list that will let you see content through the eyes of organizations and people you trust.

What is in a lens?

Lens makers point to materials (modules and collections), creating a guide that includes their own comments and descriptive tags about the content.

Who can create a lens?

Any individual member, a community, or a respected organization.

What are tags? tag icon

Tags are descriptors added by lens makers to help label content, attaching a vocabulary that is meaningful in the context of the lens.

| External bookmarks

Module to:

My Favorites (?)

'My Favorites' is a special kind of lens which you can use to bookmark modules and collections. 'My Favorites' can only be seen by you, and collections saved in 'My Favorites' can remember the last module you were on. You need an account to use 'My Favorites'.

| A lens I own (?)

Definition of a lens

Lenses

A lens is a custom view of the content in the repository. You can think of it as a fancy kind of list that will let you see content through the eyes of organizations and people you trust.

What is in a lens?

Lens makers point to materials (modules and collections), creating a guide that includes their own comments and descriptive tags about the content.

Who can create a lens?

Any individual member, a community, or a respected organization.

What are tags? tag icon

Tags are descriptors added by lens makers to help label content, attaching a vocabulary that is meaningful in the context of the lens.

| External bookmarks