6.5 Single-pixel camera  (Page 2/2)

The single-pixel design reduces the required size, complexity, and cost of the photon detector array down to a single unit, which enables the use of exotic detectors that would be impossible in a conventional digital camera. Example detectors include a photomultiplier tube or an avalanche photodiode for low-light (photon-limited) imaging, a sandwich of several photodiodes sensitive to different light wavelengths for multimodal sensing, a spectrometer for hyperspectral imaging, and so on.

In addition to sensing flexibility, the practical advantages of the single-pixel design include the facts that the quantum efficiency of a photodiode is higher than that of the pixel sensors in a typical CCD or CMOS array and that the fill factor of a DMD can reach 90% whereas that of a CCD/CMOS array is only about 50%. An important advantage to highlight is that each CS measurement receives about $N/2$ times more photons than an average pixel sensor, which significantly reduces image distortion from dark noise and read-out noise.

The single-pixel design falls into the class of multiplex cameras. The baseline standard for multiplexing is classical raster scanning, where the test functions $\left\{{\phi }_j\right\}$ are a sequence of delta functions $\delta [n-j]$ that turn on each mirror in turn. There are substantial advantages to operating in a CS rather than raster scan mode, including fewer total measurements ( $M$ for CS rather than $N$ for raster scan) and significantly reduced dark noise. See  [link] for a more detailed discussion of these issues.

[link] (a) and (b) illustrates a target object (a black-and-white printout of an “R”) $x$ and reconstructed image $\widehat{x}$ taken by the single-pixel camera prototype in [link] using $N=256\times 256$ and $M=N/50$ [link] . [link] (c) illustrates an $N=256\times 256$ color single-pixel photograph of a printout of the Mandrill test image taken under low-light conditions using RGB color filters and a photomultiplier tube with $M=N/10$ . In both cases, the images were reconstructed using total variation minimization, which is closely related to wavelet coefficient ${\ell }_1$ minimization   [link] .

Discrete formulation

Since the DMD array is programmable, we can employ arbitrary test functions ${\phi }_j$ . However, even when we restrict the ${\phi }_j$ to be $\{0,1\}$ -valued, storing these patterns for large values of $N$ is impractical. Furthermore, as noted above, even pseudorandom $\Phi$ can be computationally problematic during recovery. Thus, rather than purely random $\Phi$ , we can also consider $\Phi$ that admit a fast transform-based implementation by taking random submatrices of a Walsh, Hadamard, or noiselet transform  [link] , [link] . We will describe the Walsh transform for the purpose of illustration.

We will suppose that $N$ is a power of 2 and let $W_{{log}_{2}N}$ denote the $N\times N$ Walsh transform matrix. We begin by setting $W_0=1$ , and we now define $W_j$ recursively as

This construction produces an orthonormal matrix with entries of $\pm 1/\sqrt{N}$ that admits a fast implementation requiring $O(NlogN)$ computations to apply. As an example, note that

and

We can exploit these constructions as follows. Suppose that $N=2^B$ and generate $W_B$ . Let $I_{\Gamma }$ denote a $M\times N$ submatrix of the identity $I$ obtained by picking a random set of $M$ rows, so that $I_{\Gamma }{W}_B$ is the submatrix of $W_B$ consisting of the rows of $W_B$ indexed by $\Gamma$ . Furthermore, let $D$ denote a random $N\times N$ permutation matrix. We can generate $\Phi$ as

Note that $\frac{1}{2}\sqrt{N}{I}_{\Gamma }{W}_B+\frac{1}{2}$ merely rescales and shifts $I_{\Gamma }{W}_B$ to have $\{0,1\}$ -valued entries, and recall that each row of $\Phi$ will be reshaped into a 2-D matrix of numbers that is then displayed on the DMD array. Furthermore, $D$ can be thought of as either permuting the pixels or permuting the columns of $W_B$ . This step adds some additional randomness since some of the rows of the Walsh matrix are highly correlated with coarse scale wavelet basis functions — but permuting the pixels eliminates this structure. Note that at this point we do not have any strict guarantees that such $\Phi$ combined with a wavelet basis $\Psi$ will yield a product $\Phi \Psi$ satisfying the restricted isometry property , but this approach seems to work well in practice.