Several hardware architectures have been proposed that apply the theory of compressive sensing (CS) in an imaging setting [4], [5], [6]. We will focus on the so-called single-pixel camera [4], [7], [8], [9], [10]. The single-pixel camera is an optical computer that sequentially measures the inner products y[j]=〈x,φj〉y[j]=〈x,φj〉 between an NN-pixel sampled version of the incident light-field from the scene under view (denoted by xx) and a set of NN-pixel test functions {φj}j=1M{φj}j=1M. The architecture is illustrated in Figure 1, and an aerial view of the camera in the lab is shown in Figure 2. As shown in these figures, the light-field is focused by a lens (Lens 1 in Figure 2) not onto a CCD or CMOS sampling array but rather onto a spatial light modulator (SLM). An SLM modulates the intensity of a light beam according to a control signal. A simple example of a transmissive SLM that either passes or blocks parts of the beam is an overhead transparency. Another example is a liquid crystal display (LCD) projector.

Figure 1: Single-pixel camera block diagram. Incident light-field (corresponding to the desired image xx) is reflected off a digital micromirror device (DMD) array whose mirror orientations are modulated according to the pseudorandom pattern φjφj supplied by a random number generator. Each different mirror pattern produces a voltage at the single photodiode that corresponds to one measurement y[j]y[j].

The Texas Instruments (TI) digital micromirror device (DMD) is a reflective SLM that selectively redirects parts of the light beam. The DMD consists of an array of bacterium-sized, electrostatically actuated micro-mirrors, where each mirror in the array is suspended above an individual static random access memory (SRAM) cell. Each mirror rotates about a hinge and can be positioned in one of two states (±10±10 degrees from horizontal) according to which bit is loaded into the SRAM cell; thus light falling on the DMD can be reflected in two directions depending on the orientation of the mirrors.

Each element of the SLM corresponds to a particular element of φjφj (and its corresponding pixel in xx). For a given φjφj, we can orient the corresponding element of the SLM either towards (corresponding to a 1 at that element of φjφj) or away from (corresponding to a 0 at that element of φjφj) a second lens (Lens 2 in Figure 2). This second lens collects the reflected light and focuses it onto a single photon detector (the single pixel) that integrates the product of xx and φjφj to compute the measurement y[j]=〈x,φj〉y[j]=〈x,φj〉 as its output voltage. This voltage is then digitized by an A/D converter. Values of φjφj between 0 and 1 can be obtained by dithering the mirrors back and forth during the photodiode integration time. By reshaping xx into a column vector and the φjφj into row vectors, we can thus model this system as computing the product y=Φxy=Φx, where each row of ΦΦ corresponds to a φjφj. To compute randomized measurements, we set the mirror orientations φjφj randomly using a pseudorandom number generator, measure y[j]y[j], and then repeat the process MM times to obtain the measurement vector yy.

Figure 2: Aerial view of the single-pixel camera in the lab.

The single-pixel design reduces the required size, complexity, and cost of the photon detector array down to a single unit, which enables the use of exotic detectors that would be impossible in a conventional digital camera. Example detectors include a photomultiplier tube or an avalanche photodiode for low-light (photon-limited) imaging, a sandwich of several photodiodes sensitive to different light wavelengths for multimodal sensing, a spectrometer for hyperspectral imaging, and so on.

In addition to sensing flexibility, the practical advantages of the single-pixel design include the facts that the quantum efficiency of a photodiode is higher than that of the pixel sensors in a typical CCD or CMOS array and that the fill factor of a DMD can reach 90% whereas that of a CCD/CMOS array is only about 50%. An important advantage to highlight is that each CS measurement receives about N/2N/2 times more photons than an average pixel sensor, which significantly reduces image distortion from dark noise and read-out noise.

The single-pixel design falls into the class of multiplex cameras. The baseline standard for multiplexing is classical raster scanning, where the test functions {φj}{φj} are a sequence of delta functions δ[n-j]δ[n-j] that turn on each mirror in turn. There are substantial advantages to operating in a CS rather than raster scan mode, including fewer total measurements (MM for CS rather than NN for raster scan) and significantly reduced dark noise. See [4] for a more detailed discussion of these issues.

Figure 3 (a) and (b) illustrates a target object (a black-and-white printout of an “R”) xx and reconstructed image x^x^ taken by the single-pixel camera prototype in Figure 2 using N=256×256N=256×256 and M=N/50M=N/50[4]. Figure 3(c) illustrates an N=256×256N=256×256 color single-pixel photograph of a printout of the Mandrill test image taken under low-light conditions using RGB color filters and a photomultiplier tube with M=N/10M=N/10. In both cases, the images were reconstructed using total variation minimization, which is closely related to wavelet coefficient ℓ1ℓ1 minimization [3].

Figure 3: Sample image reconstructions from single-pixel camera. (a) 256×256256×256 conventional image of a black-and-white “R”. (b) Image reconstructed from M=1300M=1300 single-pixel camera measurements (50×50× sub-Nyquist). (c) 256×256256×256 pixel color reconstruction of a printout of the Mandrill test image imaged in a low-light setting using a single photomultiplier tube sensor, RGB color filters, and M=6500M=6500 random measurements.

(a) (b) (c)

Since the DMD array is programmable, we can employ arbitrary test functions φjφj. However, even when we restrict the φjφj to be {0,1}{0,1}-valued, storing these patterns for large values of NN is impractical. Furthermore, as noted above, even pseudorandom ΦΦ can be computationally problematic during recovery. Thus, rather than purely random ΦΦ, we can also consider ΦΦ that admit a fast transform-based implementation by taking random submatrices of a Walsh, Hadamard, or noiselet transform [1], [2]. We will describe the Walsh transform for the purpose of illustration.

We will suppose that NN is a power of 2 and let Wlog2NWlog2N denote the N×NN×N Walsh transform matrix. We begin by setting W0=1W0=1, and we now define WjWj recursively as

This construction produces an orthonormal matrix with entries of ±1/N±1/N that admits a fast implementation requiring O(NlogN)O(NlogN) computations to apply. As an example, note that

and

We can exploit these constructions as follows. Suppose that N=2BN=2B and generate WBWB. Let IΓIΓ denote a M×NM×N submatrix of the identity II obtained by picking a random set of MM rows, so that IΓWBIΓWB is the submatrix of WBWB consisting of the rows of WBWB indexed by ΓΓ. Furthermore, let DD denote a random N×NN×N permutation matrix. We can generate ΦΦ as

Note that 12NIΓWB+1212NIΓWB+12 merely rescales and shifts IΓWBIΓWB to have {0,1}{0,1}-valued entries, and recall that each row of ΦΦ will be reshaped into a 2-D matrix of numbers that is then displayed on the DMD array. Furthermore, DD can be thought of as either permuting the pixels or permuting the columns of WBWB. This step adds some additional randomness since some of the rows of the Walsh matrix are highly correlated with coarse scale wavelet basis functions — but permuting the pixels eliminates this structure. Note that at this point we do not have any strict guarantees that such ΦΦ combined with a wavelet basis ΨΨ will yield a product ΦΨΦΨ satisfying the restricted isometry property, but this approach seems to work well in practice.