Optimizing measurements for feature-specific compressive sensing Abhijit Mahalanobis1,* and Mark Neifeld2 1

Lockheed Martin, Missiles and Fire Control, Orlando, Florida 32819, USA

2

Department of Electrical and Computer Engineering, University of Arizona, Tucson, Arizona 85721, USA *Corresponding author: [email protected] Received 3 March 2014; revised 14 July 2014; accepted 27 July 2014; posted 8 August 2014 (Doc. ID 207444); published 10 September 2014

While the theory of compressive sensing has been very well investigated in the literature, comparatively little attention has been given to the issues that arise when compressive measurements are made in hardware. For instance, compressive measurements are always corrupted by detector noise. Further, the number of photons available is the same whether a conventional image is sensed or multiple coded measurements are made in the same interval of time. Thus it is essential that the effects of noise and the constraint on the number of photons must be taken into account in the analysis, design, and implementation of a compressive imager. In this paper, we present a methodology for designing a set of measurement kernels (or masks) that satisfy the photon constraint and are optimum for making measurements that minimize the reconstruction error in the presence of noise. Our approach finds the masks one at a time, by determining the vector that yields the best possible measurement for reducing the reconstruction error. The subspace represented by the optimized mask is removed from the signal space, and the process is repeated to find the next best measurement. Results of simulations are presented that show that the optimum masks always outperform reconstructions based on traditional feature measurements (such as principle components), and are also better than the conventional images in high noise conditions. © 2014 Optical Society of America OCIS codes: (110.2990) Image formation theory; (110.4280) Noise in imaging systems; (110.6980) Transforms; (170.1630) Coded aperture imaging; (110.1758) Computational imaging. http://dx.doi.org/10.1364/AO.53.006108

1. Introduction

Measurement is the physical process by which a continuous-valued signal is sampled to yield a number. Any such measurement can be mathematically represented as an inner product between the signal of interest xr and a measurement kernel pr, where r is a generalized coordinate vector that could include space, time, andR wavelength, so that the scalar measurement u  xrprdr. It is important to note that many/most conventional measurement systems employ a collection of “local” kernels fpi r; i  1; …; Ng, each of which has nonzero value 1559-128X/14/266108-11$15.00/0 © 2014 Optical Society of America 6108

APPLIED OPTICS / Vol. 53, No. 26 / 10 September 2014

only when r is within some local neighborhood of its “center.” A conventional digital video camera is an example of such a measurement system with each pixel generating a measurement ui according to a local integral in the transverse spatial dimension (i.e., the lateral resolution), in the time dimension (i.e., the frame time), and over some contiguous set of wavelengths (i.e., the color band). It is well known that images obtained in this way are highly redundant/compressible, and for this reason recent work in the domain of compressive sensing (CS) has sought to achieve more efficient use of measurement resources via use of more arbitrary nonlocal measurement kernels. So instead of making conventional measurements (i.e., which may be directly consumed by a human observer) CS advocates collecting data in

the compressed domain (i.e., in a representation that is not human-viewable) and recovering a more conventional image representation as a postprocessing step [1–4]. Although the mathematical underpinnings of CS are very well understood, comparatively little attention has been given to hardware and physical constraints that must be addressed in practice. The CS literature generally uses the same measurement model described above (i.e., after appropriate discretization and signal unrastering), in which the N-dimensional signal vector x is projected onto a set of kernel vectors typically arranged as the rows of the M × N measurement matrix Φ, to yield the M-dimensional measurement vector u  Φx. Unfortunately, much of this theoretical work tends to overlook important constraints associated with the measurement matrices that can be realized in practice, and although the theoretical results are impressive, the optimum solutions may not be realizable in practical systems, or might yield suboptimum results when implemented in real-world hardware. An important constraint associated with any physical measurement system is that the quantity of light energy available to a passive imaging system is limited. In other words, every point in the scene has a fixed brightness (i.e., determined by the scene illumination and reflectance properties at that point), which gives rise to a fixed number of photons being collected by the measurement apparatus within a given collection aperture in space, time, and wavelength. Because this fixed photon budget contributes to several measurements, each such measurement receives only a fraction of the photons from a given point in the scene. Detector noise is also sometimes overlooked in the theoretical CS literature. It is inevitable that measurements contain noise introduced by the detection process, which will impact the accuracy of the results. Although previous works have analyzed the impact of noise on the CS process [5,6], these do not explicitly deal with optimizing the measurement scheme to combat the effects of noise. Neifeld and Shankar [7] were the first to formalize this model, and proposed strategies for feature-specific imaging (FSI) using noisy measurements while imposing constraints to meet the photon budget. They observed that direct feature measurement exploits the multiplex advantage, and for small numbers of projections can provide higher feature fidelity than those systems that post process a conventional image. Since then, further development of the FSI framework has paved the way for compressive imagers that can be realized in optical hardware [6], and information theoretic techniques have been developed for quantifying the information contained in the compressive measurements for performing specific tasks [8]. The primary difference between traditional CS and FSI is in the nature of the sensing matrix employed. In particular, CS assumes sparsity in an unknown basis, and the measurements are designed to be incoherent [9]. In FSI, on the other hand, the sensing

matrix is designed using all available prior statistical knowledge of the scene. Either paradigm can be appropriate for a given situation, but if the particular exploitation task and the prior scene statistics are known, then the FSI paradigm can be very useful. Motivated by the FSI framework, we revisit the task of image reconstruction from compressive measurements informed by signal priors. We begin by observing that given a set of noisy measurements, the optimum linear operator to minimize the reconstruction mean square error (MSE) is well known [10]. It is also important to observe that when second-order signal statistics are available, the principle component analysis (PCA) features are known to be optimal for image reconstruction in a minimum mean square error (MMSE) sense [11]. Thus, it may be tempting to make measurements using PCA projections and then use the LMMSE operator to reconstruct images; however, this approach would be suboptimal even in the LMMSE sense. The key insight here is that the derivation of the PCA projections does not take into account the effect of measurement noise. In fact, it has been shown previously [8] that PCA is indeed suboptimal under high noise conditions, and that the optimum measurement kernels are binary valued. In this paper, our goal is to systematically determine the projections that are optimum for making measurements such that the effect of noise on the reconstructed imagery is minimized, subject to the finite photon budget. The rest of the paper is organized as follows. In Section 2, we present a strategy for mask optimization that takes noise and the photon constraint into account. Assuming a linear MMSE reconstruction model and prior information based on second-order statistics, we show that the optimum masks are binary valued in high noise conditions, but converge to the PCA vectors when noise is absent. For a given signal-to-noise ratio (SNR), the algorithm sequentially determines one optimum mask at a time, removes its contribution from the signal space, and then iterates to find additional masks until the reconstruction error is acceptably small. The results of simulation are presented in Section 3. We find that when SNR is low, the optimized masks not only outperform the PCA, but also yield better results than a conventional imager. Under these conditions, FSI with optimum masks yields a smaller MSE than either the reconstruction obtained with PCA masks or a conventional noisy image at comparable SNR. In high SNR conditions, the reconstruction accuracy of the optimum masks is always equal to or better than that of the PCA for any given number of measurements. Thus, in addition to ensuring that the photon constraint is satisfied, the optimization technique described in Section 2 enables the masks to achieve robust performance in noise without adversely affecting the reconstruction MSE. Although our analysis assumes that the noise is independent of the signal, this is not the case when Poisson noise is present. While a formal treatment of the effects of 10 September 2014 / Vol. 53, No. 26 / APPLIED OPTICS

6109

signal-dependent noise is beyond the scope of the paper, we compare the behavior of optimized masks to the normalized PCA in signal-dependent noise. Section 4 is a summary of the findings of the paper, and contains suggestions for the direction for future work. 2. System Description and Mask Optimization

Before discussing the procedure for optimizing the measurement kernels, we briefly review the architecture of a single-pixel compressive sensor [4,12] so that its implications on the optimization framework are understood. P Assume that we wish to compute the quantity u  m;n xm; nϕm; n, where xm; n is an image of a scene, and ϕm; n is the measurement kernel (interchangeably referred to as a mask), and m and n are the discrete spatial indices. As shown in Fig. 1, this operation can be optically implemented by imaging the scene on a spatial light modulator (SLM), and collecting the modulated light on a photo-detector. The SLM is either a transmissive or a reflective device that is encoded with the mask pattern ϕm; n. The light emerging from the SLM is proportional to the product xm; nϕm; n, and is collected by the integrating lens on the photo-detector. Effectively, the current measured at the output of the photo-detector is proportional to u. Since real-world SLMs can only implement nonnegative masks, the values of ϕm; n must range between 0 and 1, and represent the fraction of the light that is transmitted by the mask at each spatial location. On the other hand, the performance of a mask that contains both positive and negative values may be better than that of those that are strictly positive. To handle the negative values, the mask can be expressed in terms of two nonnegative quantities, i.e., ϕm; n  ϕ m; n − ϕ− m; n, where  ϕ m; n

ϕm; n 0

if ϕm; n > 0 otherwise

and  ϕm; nj ϕ− m; n 0

if ϕm; n < 0 ; otherwise

and the final result can be obtained by subtracting a partial measurement made using ϕ− m; n from

x(m,n)

another obtained using ϕ m; n. The optimization technique described in this paper can be formulated to yield both nonnegative and real-valued masks. The noise characteristics of a photo-detector depend on the type of device and the wavelength of the light. In particular, we are interested in detectors that operate in the midwave infrared (IR). Cooled sensors are much more sensitive and exhibit relatively low levels of noise that has predominantly signal-dependent Poisson characteristics. Thus cooled detectors provide significantly higher SNRs in many applications, and the impact of noise on CS measurements is less severe. On the other hand, uncooled sensors (such as lead selenide detectors [13]) are relatively inexpensive, but are also substantially noisier. In such cases, the dominant noise is pink noise (also known as 1∕f noise), which does not depend on the signal, and can be modeled as an additive process. The analysis presented in this paper deals with the impact of such signal-independent additive noise on compressive measurements, and is motivated by the goal of using inexpensive IR detectors in compressive sensors. A. Mathematical Framework

Let ϕ be a column vector that is obtained by lexicographically rearranging the elements of the mask ϕm; n. Similarly, assume that the scene xm; n is represented by the vector x that belongs to a class of images whose correlation matrix Rx  EfxxT g; is known, and Ef·g; is the expectation operator over the class of images. We wish to make compressive measurements by projecting x on a set of masks  ϕ1    ϕN  with the intent of reconstructing the original image from these measurements. Arranging the mask vectors as rows of the sensing matrix Φ, the vector of noisy measurements is given by u  Φx  v;

(1)

where ν is the detector noise vector, assumed to be additive uncorrelated noise with correlation matrix Rν  EfννT g. Our goal is to determine the kernels (i.e., to be implemented as physical masks in the compressive optical imaging system) that minimize the MSE between the ideal and reconstructed images in the presence of noise. Toward this end, we first note that for a given measurement matrix Φ the linear operator W that minimizes MSE  Efjx − Wuj2 g

(2)

between the ideal and reconstructed images is given by [8] φ(m,n)

Fig. 1. Basic architecture of a single-pixel compressive sensor that integrates the light from the scene, modulated by codes written on the SLM. 6110

APPLIED OPTICS / Vol. 53, No. 26 / 10 September 2014

W  Rx ΦT ΦRx ΦT  Rν −1 :

(3)

Assuming that the noise is zero mean and uncorrelated with the signal, the MSE can be expressed as

MSE  EfjWΦx  v − xj2 g  EfxT ΦT W T WΦx  νT W T Wν  xT x − 2xT WΦxg:

(4)

By noting that xT W T Wx  trfWxxT W T g, the MSE can be written as

are used in both the conventional and the featurespecific measurements. The mathematical implication of this photon-count constraint is that the maximum absolute column sum of Φ must be less than or equal to 1.0. Therefore, to impose the photon constraint on a single measurement vector, we restate the problem as minimize

MSE  EftrfWΦxxT ΦT W T  WννT W T  xxT g

over all Φ

− 2trfΦxxT Wgg

MSE  trfRx g −

subject to:maxjΦj j  1;

 trfWΦRx ΦT W T  WRν W T  Rx g

j

− 2trfΦRx Wg:

(5)

For a given choice of Φ setting the derivative of the above expression to zero yields the LMMSE operator W in Eq. (3). However, this depends on the choice of Φ, and clearly the MSE is also a function of the sensing matrix. Therefore, the question remains: what is the optimum choice of Φ that will yield the minimum value of MSE? To answer this question, let us consider the special case in which only one measurement is being made, and Φ is therefore a vector. Let us first determine what is the optimum mask if the reconstruction were to be based on this single measurement. In this case W reduces to W

Rx ΦT ; ασ

6

where α  ΦRx ΦT , and σ is the noise variance. Substituting this for W in Eq. (5), we get 

R ΦT ΦRx R ΦT ΦRx  x σ  Rx MSE  tr x α ασ ασ ασ ασ   Rx ΦT − 2tr ΦRx ασ   Rx ΦT ΦRx Rx ΦT ΦRx  σ  Rx  tr α α  σ2 α  σ2   ΦRx Rx ΦT − 2tr ασ     T Rx Φ ΦRx ΦRx Rx ΦT  Rx − 2tr  tr α  σ ασ 

B.



ΦRx Rx ΦT : ΦRx ΦT  σ

(8)

where Φj is the jth element of Φ, and Rxx  Rx 2 . To minimize MSE, the second term has to be maximized. Therefore, the above problem is equivalent to maximize over all Φ

J σ;Φ 

ΦRxx ΦT ΦRx ΦT  σ

subject to: maxjΦj j  1: j

(9)

One way to ensure that jΦj j ≤ 1 is to assume that Φj  1 − exp−αyj ∕1  exp−αyj , where yj are dummy variables free to be taken on any real value [14]. Thus, we can maximize the ratio with respect to yj , with the understanding that the mask values are a sigmoid function of the optimum values of yj. In terms of the vector elements, the objective function in Eq. (9) is P P xx ΦRxx ΦT i j Φi Φj rij P P  ; Jσ  ΦRx ΦT  σ i j Φi Φj rij  σ

(10)

where rxx ij and rij are the elements in the ith row and jth column of Rxx and Rx , respectively. The subscript J σ in J σ emphasizes that the objective function also depends on the noise level, which plays an important role in the behavior of its gradient. Taking the derivative of J σ with respect to yi , we see that " P xx −2α exp−αyi  j Φj rij P P ∇yi J σ  1  expαyi 2 i j Φi Φj rij  σ # P P  xx X i j Φi Φj rij 2 − P P Φj rij : i j Φi Φj rij  σ j

ΦRx Rx ΦT ΦRx Rx ΦT  trfRx g − 2 α  σ ασ

 trfRx g −

ΦRxx ΦT ΦRx ΦT  σ

(7)

Optimization with Photon Constraint

A consequence of the conservation of energy is that no column in the sensing matrix Φ can sum (in absolute value) to greater than one. This constraint ensures that for a given object irradiance distribution and measurement time, the same number of photons

(11)

If we define a diagonal matrix S with sii  −2α exp−αyi ∕1  exp−αyi 2 as its diagonal element in the ith row and column, then ∇y J σ 

1 · S · Rx Rx ΦT − J σ ΦT : ΦRx ΦT  σ

(12)

This gradient can be zero in one of two ways. Either (i) Rx ΦT − J σ ΦT   0, which implies that ΦT is a eigenvector of Rx with eigenvalue J σ , or (ii) the 10 September 2014 / Vol. 53, No. 26 / APPLIED OPTICS

6111

diagonal elements of S are zero. When σ is small, J σ as defined in Eq. (10) depends mostly on Φ. In the limiting case when σ  0, choosing Φ to be the dominant eigenvector of Rx equates J σ to its largest eigenvalue, and the first condition is satisfied. However, when σ becomes larger, J σ decreases and can no longer equal the largest eigenvalue of Rx. In this case, condition (i) no longer holds, and the solution pivots to satisfy condition (ii) i.e., −2α exp−αyj ∕ 1  exp−αyj 2  0, which implies that Φi  1 − exp −αyj ∕1  exp−αyj   1, and we have proof that the optimum values for the element of Φ must be binary. Note that this formulation unifies the two different possible solutions for the optimum vector; i.e., whether the optimum solutions are eigenvectors (scaled to meet the photon constraint), or binary vectors with values limited to 1, setting this gradient to zero yields both types of solutions. In fact, simulations show that in the low noise case, the solution converges to the eigenvectors of Rx, whereas in the high noise case it leads to the binary-valued solutions. Although the eigenvector solution can be readily found when σ  0, we require a numerical procedure for estimating the binary masks for all other cases. While the above analysis reveals that the optimum masks for handling noisy conditions are binary, it does not explicitly yield the values of the masks. This requires us to use the gradient expression in Eq. (12) with various numerical search and optimization routines available in MATLAB to find the optimum solution for Φ. For simplicity, we employ a gradient descent rule for finding the optimum solution as follows. We define a vector yk   yk1    ykn  that has the values of the underlying dummy variables at the kth iteration. The iterative gradient descent update rule is then given by yk1  yk − μ · ∇yk J:

(13)

Although the original MSE criterion is quadratic in Φ, it is not so in terms of yi. Therefore, the convergence of this algorithm may not always be guaranteed, and needs to be examined further. However, preliminary numerical simulations appear to converge to meaningful results and are presented in Section 3.

The correlation matrix for this residual is     Rx ΦT Φx Rx ΦT Φx T Rxˆ  E x − x− ΦRx ΦT  σ ΦRx ΦT  σ   R ΦT ΦxxT ΦT ΦRx xxT ΦT ΦRx  E xxT  x − 2 ΦRx ΦT  σ2 ΦRx ΦT  σ  Rx 

Rx ΦT ΦRx ΦT ΦRx Rx ΦT ΦRx − 2 ΦRx ΦT  σ2 ΦRx ΦT  σ

ΦRx ΦT Rx ΦT ΦRx  R ΦT ΦR −2 x T x 2 T ΦRx Φ  σ ΦRx Φ  σ    T T Rx Φ ΦRx ΦRx Φ −2 :  Rx  ΦRx ΦT  σ ΦRx ΦT  σ

 Rx 

(15)

To find the next best measurement, the optimization algorithm can be repeated by using Rxˆ in lieu of Rx so that the MSE is minimized with respect to the residual xˆ ;. If the noise is negligible or zero, then ΦRx ΦT ∕ΦRx ΦT  σ ≅ 1, and the expression reduces to Rxˆ  Rx −

Rx ΦT ΦRx : ΦRx ΦT

(16)

Also, in the absence of noise, ΦT converges to an eigenvector of Rx, so that ΦRx ΦT  λ is an eigenvalue, and Rx ΦT  λΦT , which yields Rxˆ  Rx − λΦT Φ:

(17)

As expected, this is just removing the subspace represented by the first measurement. However, when the noise is large, the result is modified so that the subspace is weighted by the effective SNR for that measurement. 3. Results of Simulation

A database of thermal images of six different types of vehicular targets, some of which are shown in Fig. 2, was used to create the noise optimized masks for FSI. It should be noted that such thermal images in the long-wave IR spectrum have relatively low spatial

1. Finding Additional Measurement Vectors Given that the single best measurement vector is obtained as described above in Section 2.A, the question remains as to what should be the next best measurement. To answer this, we observe that the residual signal after subtracting the reconstruction based on the first vector is

xˆ  x −

6112

Rx ΦT Φx : ΦRx ΦT  σ

(14)

APPLIED OPTICS / Vol. 53, No. 26 / 10 September 2014

Fig. 2. Typical images used for estimating target statistics.

(a)

(b)

5

5

10

10

15

15

20

20 5

10

15

20

25

30

35

40

(c)

5

10

15

20

25

30

35

40

5

10

15

20

25

30

35

40

(d) 5

5

10

10

15

15

20

20 5

10

15

20

25

30

35

40

Fig. 3. Conventional images of a target for various noise levels: (a) no noise; (b) SNR  5; (c) SNR  30; and (d) SNR  90.

SNR in terms of the signal and the detector noise in a conventional camera. Assume that the ideal image x obtained using a conventional imager is corrupted by detector noise that is zero mean additive white Gaussian noise (AWGN) with standard deviation σ. If xmax and xmin represent the maximum and minimum signal amplitudes, then the SNR is defined as xmax − xmin ∕2σ. Once σ is determined, the same noise level is used for making the compressive measurements, and for assessing the performance of the masks. For illustrative purposes, a conventional image of a target with no noise is shown in Fig. 3, along with versions of the same image corrupted

frequency content, and the information contained in the pixels is often redundant. Our goal is to examine the results of FSI using far fewer measurements than the number of conventional pixels contained in these images. The images are of size 20 × 40 pixels, and consequently the target correlation matrix Rx estimated from these images is of size 800 × 800. Since the location of the target in the scene is assumed to be unknown, all possible shifted versions of the targets (within the 20 × 40 window) were also included in the estimation of Rx. To compare the results of CS and reconstruction to those of a conventional imaging sensor, we define a SNR=5,Mask 1

SNR=5,Mask 2

10

SNR=5,Mask 3

10

20

10

20 10

20

30

40

20

30

40

10 20 20

30

40

20

30

40

20 20

30

40

PCA Mask 1

20

30

40

20 20

30

40

30

20

30

40

30

40

30

40

20

30

40

10 20 10

20

30

40

10

20

30

40

PCA Mask 5

10

20 20

10

SNR=90,Mask 5

10

10

40

20 20

PCA Mask 4

20 10

30

10

10

40

20

SNR=30,Mask 5

20 20

10

SNR=90,Mask 4

10

20 10

40

PCA Mask 3

10

40

10

10

PCA Mask 2

10

30

20 10

30

20 20

10

20 10

20

SNR=30,Mask 4

SNR=90,Mask 3

10

20 10

10

10

SNR=90,Mask 2

10

40

20 10

SNR=90,Mask 1

30

10

20 10

20

SNR=30,Mask 3

10

10

20 10

SNR=30,Mask 2

SNR=5,Mask 5

10

20 10

SNR=30,Mask 1

SNR=5,Mask 4

20 10

20

30

40

10

20

30

40

Fig. 4. Masks in the top three rows are optimized for SNR values of 5, 30, and 90, respectively. The PCA masks are shown in the bottom row. Note that the masks appear to be almost binary valued for SNR  5 (high noise) and essentially the same as the PCA for SNR  90 (low noise). 10 September 2014 / Vol. 53, No. 26 / APPLIED OPTICS

6113

the number of measurements is increased. It is clear that the masks optimized to handle the high noise case perform the best and always yield the smallest MSE. On the other hand, the normalized PCA yields the worst performance, as expected. It is also noteworthy that the masks optimized for moderate and low noise cases also outperform the normalized PCA, although to a lesser extent. Hence, even if the masks are optimized for SNR values that are different from the actual noise level, their performance is better than that of the normalized PCA when the measurements are corrupted by noise. The performance of the masks in moderate and low noise levels is shown in Figs. 6(a) and 6(b), respectively. In Fig. 6(a), σ is chosen to correspond to an SNR of 30. Although all sets perform comparably with the few initial measurements, we see that as the number of features increases, the set optimized for moderate noise level (i.e., for SNR  30) outperforms the others. Not surprisingly, the normalized PCA produces the largest MSE as the number of measurements is increased. In low noise conditions, the performance of all masks is comparable to the PCA, as shown in Fig. 6(b). This implies that the optimization algorithm does not produce any appreciable loss of performance in the absence of noise, but yet provides considerable improvements when noise is present in the measurements. We now discuss the effect of imposing a photon constraint on the measurements made using the optimized masks. In IR systems, the integration time of the detector determines the number of photons collected over that time interval. The plots shown in Figs. 5 and 6 assume that the same integration time is used for every measurement. This implies that as the number of masks increases, more photons are utilized in the final reconstruction result. Whether this is feasible depends on the design of the imaging system. For example, if a final frame rate of no more

by noise at SNR values of 5, 30, and 90. For the purposes of our analysis, these are considered to be the high, moderate, and low noise cases, respectively. We created three sets of masks optimized for various SNR values of 5, 30, and 90 using the algorithm described in Section 2. The first five masks for each case are shown in Fig. 4, along with the masks based on the PCA vectors. It is interesting to note that for the low SNR of 5 (high noise case), the optimum masks are practically binary valued. At the moderate SNR of 30, the masks are more saturated than the PCA, but also exhibit some fractional gray values. However, at high SNR of 90 (low noise case), the masks appear to be essentially the same as the PCA vectors. The logarithm of the reconstruction MSE obtained using these masks for the high noise case is shown in Fig. 5. Here, σ was chosen to yield an SNR of 5, and the MSE in Eq. (5) is plotted for different numbers of measurements. In all cases, the MSE is reduced as 19.7 Normalized PCA Optimized SNR 5 Optimized SNR 30 Optimized SNR 90

19.6

log (Mean Square Error)

19.5 19.4 19.3 19.2 19.1 19 18.9 18.8

0

10

20

30

40

50

60

70

80

90

100

Number of Masks

Fig. 5. In high noise, the masks optimized for an SNR of 5 yield the smallest MSE. Masks optimized for moderate and high SNR also perform better than the PCA.

(a)

20

(b) Normalized PCA Optimized SNR 5 Optimized SNR 30 Optimized SNR 90

Normalized PCA Optimized SNR 5 Optimized SNR 30 Optimized SNR 90

19.5

log (Mean Square Error)

log (Mean Square Error)

19.5

20

19

18.5

19

18.5

18

18 17.5

17.5

0

10

20

30

40

50

60

70

80

90

100

Number of Masks

17

0

10

20

30

40

50

60

70

80

90

100

Number of Masks

Fig. 6. In moderate noise (a) the mask optimized for SNR  30 yields the best performance as the number of features is increased, while the PCA yields the highest MSE. For SNR  90 (b) (i.e., in low noise) the performance of all sets of masks is comparable. 6114

APPLIED OPTICS / Vol. 53, No. 26 / 10 September 2014

than 10 Hz is desired using an integration time of a millisecond per measurement, then up to 100 masks can be used for reconstructing each image frame. In this scenario, images reconstructed with fewer measurements essentially receive fewer photons, and the time required for making all the measurements depends on the number of masks to be used. On the other hand, if all measurements must be made within a fixed interval of time, then the integration time for each measurement depends on the number of masks to be employed. Essentially, this ensures that the number of photons utilized for reconstructing a complete image is always the same. The effect of reducing the integration time for each mask can be treated as a multiplicative scale factor applied to the corresponding measurement prior to adding noise. Therefore, if N measurements are used to reconstruct an image, noise with the same 19.8 19.7

Log (Mean Square Error)

19.6 19.5 19.4 19.3 19.2 19.1

Optimized SNR 5 Optimized SNR 30 Optimized SNR 90 Normalized PCA

19 18.9 0

10

20

30

40

50

60

70

80

90

100

Number of Masks

Fig. 7. When a photon constraint is imposed by limiting the integration time allocated to each mask, the MSE initially decreases but then increases again as more noisy measurements are included. In high noise conditions, the best results are obtained using the masks optimized for SNR  5, although the masks optimized for low and medium SNR still outperform the PCA.

(a)

(b)

20 Optimized SNR 5 Optimized SNR 30 Optimized SNR 90 Normalized PCA

20 Optimized SNR 5 Optimized SNR 30 Optimized SNR 90 Normalized PCA

19.5

Log (Mean Square Error)

Log (Mean Square Error)

19.5

19

18.5

18

17.5 0

standard deviation σ (determined as before based on the SNR of a conventional image) is added to each measurement scaled by 1∕N. Figure 7 shows the effect of imposing this photon constraint on the behavior of the reconstruction MSE for the high noise case SNR  5. Under these circumstances, we see that the MSE initially decreases as more masks are used until a minimum value is achieved for 17 measurements. This occurs as long as additional measurements contain more signal information than noise. After that, the MSE starts to increase when more measurements are included since these introduce more noise than useful signal information. It is noteworthy that in Fig. 7, the masks optimized for high noise (i.e., the plot indicated with the “+” symbol) outperform the PCA (solid line) and other masks. This set not only provides a smaller MSE at any number of measurements, but also achieves the smallest overall MSE, and hence provides the best possible reconstruction. The results for the moderate and low noise cases are shown in Fig. 8. Here, the mask optimized for SNR  30 yields the smallest reconstruction MSE in moderate noise using 85 measurements as shown in Fig. 8(a). For SNR  90 there is no appreciable difference between the performance of the masks as shown in Fig. 8(b). After comparing these results to those obtained earlier in Figs. 5 and 6, we conclude that although the photon constraint impacts the behavior of the MSE as the number of measurements is increased, the best results for a given SNR are obtained using the masks that were optimized for that noise level. It should be noted that if two measurements are made for every mask (to account for values) based on architectural considerations, then the integration time for each measurement is reduced by a factor of 2, which in turn reduces the measurement SNR. Although this results in a larger MSE for all masks, the overall trends shown in Figs. 7 and 8, and our conclusions, remain the same.

19

18.5

18

17.5

17 10

20

30

40

50

60

Number of Masks

70

80

90

100

0

10

20

30

40

50

60

70

80

90

100

Number of Masks

Fig. 8. Under the photon constraint at SNR  30, the mask optimized for moderate noise yields the best result, compared to the PCA and the masks optimized for other noise levels, as shown in (a). In low noise conditions (SNR  90), there is no appreciable difference between any of the masks, as shown in (b). 10 September 2014 / Vol. 53, No. 26 / APPLIED OPTICS

6115

a zero mean signal-dependent noise with standard p









T deviation σ η  γ jΦ xj, and v is the signalindependent noise process (as before) with standard deviation σ. Thus, σ η is proportional to the square root of the magnitude of the projection of the signal on the mask. We chose the constant γ such that σ η  σ; when Φ is the first mask, and is left unchanged for all other masks. The experiment described in Fig. 7 is repeated using the modified noise model, and the results are shown in Fig. 9. While the minimum MSE is somewhat greater for all masks, their behavior differs significantly as the number of measurements is increased. The masks optimized for higher levels of noise exhibit a relatively smaller growth in MSE as the number of measurements is increased. For instance, in Fig. 9, the normalized PCA achieves the minimum log(MSE) of 19.25 for about 10 measurements, whereas the mask optimized for SNR  5 achieves an even smaller minimum log(MSE) of 19.1 with about 13 measurements. It is clear from these results that the masks designed to minimize MSE in signal-independent noise continue to perform better than the normalized PCA, even when signal-dependent noise is present.

Although the masks have been optimized for signal-independent noise, we now examine their behavior in noise that has a signal-dependent component. Toward this end, Eq. (1) is modified so that the measurement obtained with the mask Φ is given by u  ΦT x  v  η, where the term η represents 21 Optimized SNR 5 Optimized SNR 30 Optimized SNR 90 Normalized PCA

20.8

Log (Mean Square Error)

20.6 20.4 20.2 20 19.8 19.6 19.4 19.2 19

0

10

20

30

40

50

60

70

80

90

100

Number of Measurements

Fig. 9. Masks designed to minimize MSE in signal-independent noise continue to perform better than the normalized PCA, even when signal-dependent noise is present.

SNR

Optimized Mask

5 10

5

15

5

5

10

10

15

15

20

20 5

10

15

20

25

30

35

5

40

15

20

25

30

35

40

20 5

5

5

5

10

10

15

15

15

10

20

30

40

20 20

30

40

5

5

5

10

10

10

15

15

15

20 30

MSE=4.61 Compression Ratio=8x

40

20

25

30

35

40

15

20

25

30

35

40

MSE=4.6

5

20

10

MSE=14.6 Compression Ratio=9.4

20

15

20

10

MSE=11.1 Compression Ratio = 9.4

10

10

MSE=26.2

10

20

90

10

MSE=28.8 Compression Ratio =47

MSE=20.15 Compression Ratio= 47

30

Conventional Image (no compression)

Normalized PCA

20

10

20

30

MSE=4.62 Compression Ratio=8x

40

10

20

30

40

MSE=1.5

Fig. 10. Results of reconstructing the ideal image in Fig. 3(a) using noisy feature-specific measurements (using the optimized mask and masks based on the PCA) are compared to the conventional image at the same SNR. The results show that the optimized masks always outperform the PCA by yielding a smaller MSE at the same compression ratio. The results are also better than the conventional image in high noise, and visually comparable to the conventional image in moderate and low noise conditions. In these cases, reconstruction based on FSI exhibits a residual MSE due to the compressive nature of the measurements. 6116

APPLIED OPTICS / Vol. 53, No. 26 / 10 September 2014

For visual comparison, several examples of reconstructed and conventional images are shown in the table in Fig. 10 for different noise levels. For each case, we also note the MSE with respect to the noise-free ideal image, as well as the effective compression ratio (i.e., the ratio between the number of pixels in the image and the number of measurements that were used to obtain the reconstruction). Under very noisy conditions at SNR  5, the optimized masks yield the smallest MSE (20.15) outperforming both the PCA (MSE  28.8) as well as a conventional imager (MSE  26.2). Additionally, only 17 measurements were used to obtain this result, which represents a compression ratio of 47. The reconstruction using the optimized masks has the best visual quality; the reconstruction using the PCA does not resemble the original object, and the conventional image appears to be very noisy. In moderate noise (SNR  30), the optimized masks (MSE  11.1) still outperform the PCA (MSE  14.6). Although the conventional image has a lower MSE, the reconstruction using the optimized mask and the PCA provides a compression ratio of 9.4. The shape of the target obtained with the optimized masks is visually comparable to the ideal image, whereas the PCA-based reconstruction appears more distorted. The target can be clearly seen in the traditional image, albeit with the noise also visually evident. Finally, in low noise conditions, very comparable results are obtained using both the optimized masks (MSE  4.61) and the PCA masks (MSE  4.62). Both exhibit a slightly larger MSE than the conventional image (MSE  1.5), mainly because of the compressive nature of the measurements (compression ratio  8). 4. Summary

While the theory of CS has been very well investigated in the literature, comparatively little attention has been given to the issues that arise when compressive measurements are made in hardware. For instance, compressive measurements are corrupted by detector noise. Further, the number of photons available is the same whether a conventional image is sensed or multiple coded measurements are made in the same interval of time. Thus it is essential that the effects of noise and the constraint on the number of photons must be taken into account in the analysis, design, and implementation of a compressive imager. FSI is a form of CS in which the measurement kernels are not random, but are based on prior knowledge of the information we are interested in sensing. Working in this FSI framework, we have developed a methodology for designing a set of masks that satisfy the photon constraint and are optimum for making measurements that minimize the reconstruction MSE in the presence of noise. The measurement matrix is obtained by arranging these masks as its rows. A criteria for meeting the photon constraint is that the maximum L1 norm of the

columns of the measurement matrix is less than or equal to 1.0. To simplify the optimization process, we employed an analytical mapping that ensures the masks can take on any value between 1 and formulated a quadratic objective function that can be minimized using gradient descent. The process then finds the masks one at a time, by determining the vector that yields the best possible measurement for reducing the MSE. The subspace represented by the optimized mask is removed from the signal space, and the process is repeated to find the next best measurement. Of course, the primary reason for CS is to make fewer measurements than the number of pixels in the reconstructed image, and to collect the information more efficiently than a conventional image. One might intuitively expect that utilizing more features (or measurements) in the reconstruction process will yield a smaller reconstruction error. We demonstrated, however, that the photon constraint limits the number of masks that can be used at a particular SNR to reduce the reconstruction MSE. In noisy conditions, the MSE initially decreases as the number of measurements is increased, but then increases when measurements that contain more noise than signal information are included. The simulations showed that the optimum masks always outperform masks based on the PCA vectors, and also yield a smaller MSE than the conventional image in high noise conditions. Although a formal treatment of the effects of signal-dependent noise is beyond the scope of the paper, we found that the optimized masks perform better than the normalized PCA, even in signaldependent noise. Future work will focus on extending the optimization algorithm so that multiple masks are jointly optimized subject to the photon constraint, rather than one at a time. Other types of noise models will also be considered, such as signal-dependent shot noise. This material is based upon work supported by DARPA and the SPAWAR System Center Pacific under Contract No. N66001-11-C-4092. The views expressed are those of the author and do not reflect the official policy or position of the Department of Defense or the U.S. Government. References and Note 1. E. Candès and T. Tao, “Near optimal signal recovery from random projections: universal encoding strategies,” IEEE Trans. Inf. Theory 52, 5406–5425 (2006). 2. D. Donoho, “Compressed sensing,” IEEE Trans. Inf. Theory 52, 1289–1306 (2006). 3. R. Baraniuk, “Compressive sensing,” IEEE Signal Process. Mag. 24(4), 118–121 (2007). 4. M. Wakin, J. Laska, M. Duarte, D. Baron, S. Sarvotham, D. Takhar, K. Kelly, and R. Baraniuk, “An architecture for compressive imaging,” presented at the International Conference on Image Processing (ICIP), Atlanta, Georgia, October 2006. 5. P. T. Boufounos, M. F. Duarte, and R. G. Baraniuk, “Sparse signal reconstruction from noisy compressive measurements using cross validation,” in Proceedings of the IEEE Workshop on Statistical Signal Processing (SSP) (IEEE, 2007), pp. 299–303. 10 September 2014 / Vol. 53, No. 26 / APPLIED OPTICS

6117

6. M. Raginsky, R. M. Willett, Z. T. Harmany, and R. F. Marcia, “Compressed sensing performance bounds under Poisson noise,” IEEE Trans. Signal Process. 58, 3990–4002 (2010). 7. M. A. Neifeld and P. Shankar, “Feature-specific imaging,” Appl. Opt. 42, 3379–3389 (2003). 8. A. Ashok, L.-C. Huang, and M. A. Neifeld, “Information optimal compressive sensing: static measurement design,” J. Opt. Soc. Am. A 30, 831–853 (2013). 9. R. Baraniuk, M. Davenport, R. DeVore, and M. Wakin, “A simple proof of the restricted isometry property for random matrices,” Constr. Approx. 28, 253–263 (2008). 10. H. C. Andrews and B. R. Hunt, Digital Image Restoration, Signal Processing Series (Prentice-Hall, 1977).

6118

APPLIED OPTICS / Vol. 53, No. 26 / 10 September 2014

11. R. J. Clarke, Transform Coding of Images (Academic, 1985). 12. D. Takhar, J. Laska, M. Wakin, M. Duarte, D. Baron, S. Sarvotham, K. Kelly, and R. Baraniuk, “A new compressive imaging camera architecture using optical-domain compression,” presented at Computational Imaging IV at SPIE Electronic Imaging, San Jose, California, January 2006. 13. G. Vergara, “VPD PbSe technology fills the existing gap in uncooled, low cost and fast IR imagers,” Proc. SPIE 8012, 80121Q (2011). 14. Any differentiable function whose magnitude ranges in the interval should yield similar results. The function can be readily modified to limit the optimum mask values to [0,1] if a non-negative solution is desired.

Optimizing measurements for feature-specific compressive sensing.

While the theory of compressive sensing has been very well investigated in the literature, comparatively little attention has been given to the issues...
721KB Sizes 2 Downloads 6 Views