Dual-camera design for coded aperture snapshot spectral imaging Lizhi Wang,1 Zhiwei Xiong,2,* Dahua Gao,3 Guangming Shi,1 and Feng Wu4 1

Xidian University, Xi’an, China

2

Microsoft Research, Beijing, China

3

Air Force Engineering University, Xi’an, China

4

University of Science and Technology of China, Hefei, China *Corresponding author: [email protected]

Received 24 July 2014; revised 7 December 2014; accepted 14 December 2014; posted 15 December 2014 (Doc. ID 217578); published 28 January 2015

Coded aperture snapshot spectral imaging (CASSI) provides an efficient mechanism for recovering 3D spectral data from a single 2D measurement. However, since the reconstruction problem is severely underdetermined, the quality of recovered spectral data is usually limited. In this paper we propose a novel dual-camera design to improve the performance of CASSI while maintaining its snapshot advantage. Specifically, a beam splitter is placed in front of the objective lens of CASSI, which allows the same scene to be simultaneously captured by a grayscale camera. This uncoded grayscale measurement, in conjunction with the coded CASSI measurement, greatly eases the reconstruction problem and yields high-quality 3D spectral data. Both simulation and experimental results demonstrate the effectiveness of the proposed method. © 2015 Optical Society of America OCIS codes: (110.4234) Multispectral and hyperspectral imaging; (110.1758) Computational imaging; (120.6200) Spectrometers and spectroscopic instrumentation; (150.4232) Multisensor methods. http://dx.doi.org/10.1364/AO.54.000848

1. Introduction

Spectral imaging has played an important role in various areas such as medicine, astrophysics, manufacturing, and remote sensing [1–4]. To obtain a 3D data cube consisting of 2D spatial and 1D spectral information, conventional spectral imagers usually scan the scene along certain dimensions sequentially [5,6], which is time-consuming and not suitable for measuring dynamic scenes. Recently, there have emerged some computational-based spectral imaging techniques [7–9] among which coded aperture snapshot spectral imaging (CASSI) has attracted a lot of attention due to its ability to recover the whole 3D data cube from a single 2D image [10–15]. The secret behind CASSI is that the spectral data of natural scenes is intrinsically sparse, and this sparsity 1559-128X/15/040848-11$15.00/0 © 2015 Optical Society of America 848

APPLIED OPTICS / Vol. 54, No. 4 / 1 February 2015

can be coped well with by the compressive sampling (CS) theory [16,17]. In practice, however, the required condition for high-fidelity CS reconstruction is difficult to verify through coded aperture design. As a result, CASSI suffers from limited quality of the recovered spectral data, especially for the applications demanding high spatial and spectral resolution. An existing solution to this problem is multi-frame CASSI, which takes multiple shots of the same scene with different coded apertures [18–21]. Although the reconstruction quality can be improved with increased measurements, this solution compromises the distinct snapshot advantage of CASSI. In this paper we propose a novel dual-camera design to improve the performance of CASSI while maintaining its snapshot advantage. This new design, as shown in Fig. 1, merely includes a beam splitter and an ordinary grayscale camera in addition to a suite of CASSI. The grayscale branch of the system uses a detector identical to that of the

spectral information before the focal plane can be expressed as f x; y; λ  f 0 x − ϕλ; y; λTx − ϕλ; y;

Fig. 1. Schematic of dual-camera design for CASSI.

CASSI branch. The beam splitter is placed in front of the objective lens of CASSI and equally splits the incident light from the scene toward two directions. Light in the two directions is then captured by CASSI and the grayscale camera simultaneously, which are synchronized by a computing unit. The measurement of the grayscale camera can be regarded as a linear sampling over all spectral bands, which is complementary to the CS as conducted by CASSI. This uncoded grayscale measurement, in conjunction with the coded CASSI measurement, greatly eases the reconstruction problem. The proposed method thus yields much higher-quality spectral data compared with 1-frame CASSI. It is even competitive to multi-frame CASSI, yet with only a single shot. Our work is motivated by the hybrid camera design in previous spectral imaging systems. The pansharpening technique employs a low-resolution spectral imager and a high-resolution panchromatic camera to capture spectral and panchromatic images of the same geographic area, respectively [22,23]. With some postprocessing methods, the resolution of spectral images can be enhanced by utilizing the corresponding panchromatic images. However, these systems are not capable of snapshots. Recently, Ma et al. presented a hybrid camera system for capturing spectral video [24,25]. This system consists of a high-resolution RGB camera and a customized, lowresolution spectral imager developed in [26]. The final high-resolution spectral images are generated by employing a propagation algorithm based on trilateral filtering. In comparison, our proposed dualcamera design inherits the advantage of CASSI that high-resolution spectral images can be obtained directly. 2. System Principles A.

CASSI

A schematic of the CASSI system is included in Fig. 1. Denote the 3D source information entering the objective lens as f 0 x; y; λ, where x and y index the spatial coordinates and λ indexes the wavelength coordinate. The objective lens projects the source information onto the coded aperture, which performs a spatial modulation. After propagation through the coded aperture, the spatially modulated information is spectrally dispersed by the prism. Then the

(1)

where Tx; y denotes the transmission function of the coded aperture, and ϕλ denotes the wavelengthdependent dispersion function induced by the prism. It is generally supposed the dispersion is along one direction of the detector (here the x coordinate). The resultant intensity image at the detector plane is the integration of the field f x; y; λ over the detector’s spectral response range Λ, which can be represented as Z ωλf x; y; λdλ; (2) gx; y  Λ

where ωλ is the spectral response function of the detector. Since the detector array is spatially pixelated, gx; y is sampled and integrated across the grid of the detector. The m; nth pixel measurement is given by Z m1Δ Z n1Δ gmn  gx; ydxdy; (3) mΔ



with Δ being the pitch of the detector. Generally, the spectral dimension is discretized into L bands. Thus, the discrete formation of a 2D CASSI measurement can be written as gmn 

L−1 X

ωk f m−knk T m−kn ;

(4)

k0

where ωk , f mnk , and T mn are the discretized representation of the detector’s spectral response, the underlying scene, and the coded aperture, respectively. It can be further expressed in a linear matrix form as g  Hf;

(5)

where H represents the system forward response of CASSI. For multi-frame CASSI, the system model is extended as 2

g1 6 g2 6 6 .. 4 .

gN

3

2

H1 7 6 H2 7 6 7  6 .. 5 4 .

3 7 7 7f; 5

(6)

HN

where N is the frame number used for reconstruction. Note each frame requires a distinct coded aperture. B. Dual-Camera Design

In our proposed dual-camera design the incident light from the scene is split toward two directions. 1 February 2015 / Vol. 54, No. 4 / APPLIED OPTICS

849

Light in one direction is captured by CASSI, which is formulated as spatial modulation, spectral dispersion, and detector integration. This process conducts a CS of the 3D spectral data cube. Light in the other direction is directly captured by a grayscale camera that has the same detector as in the CASSI branch. This grayscale measurement, without spatial modulation and spectral dispersion, can be regarded as a uniform sampling along the spectral dimension. Consequently, the scene is imaged twice through different sampling mechanisms. For an easy interpretation, the formation process of these two types of measurements is illustrated in Fig. 2. Specifically, the grayscale camera integrates the source information f 0 x; y; λ as Z ~ y  ωλf 0 x; y; λdλ: (7) gx; Λ

After spatial pixelation the grayscale measurement can be written as

C.

Reconstruction Algorithm

Given the system forward response H0 , our goal is to recover the whole spectral-data cube f from the incomplete measurements g0. Due to the intrinsic sparsity of the spectral data, CS theory ensures that a unique solution can be found for this underdetermined system by solving the following optimization problem [27]: fˆ  arg min‖g0 − H0 f‖22  τΓf; f

(12)

where Γf is a regularization term imposing sparsity and τ is a weighting factor between fidelity and sparsity. To be consistent with the original multi-frame CASSI work [18] and thus facilitate the performance comparison, we adopt the total variation (TV) regularizer [28] as ΓTV f 

X X q f m1nk − f mnk 2  f mn1k − f mnk 2 ; k m;n

g~ mn 

L−1 X

ωk f mnk .

k0

Eq. (8) can be expressed in a linear matrix form as ~ g~  Hf;

(9)

~ represents the system forward response of where H the grayscale camera. With the dual-camera design, we can obtain two kinds of measurements corresponding to Eqs. (5) and (9), which obey the observation model as follows: g0  H0 f  ν0 ;

(10)

where g0 

  g ; g~

 H0 

 H ~ ; H

ν0 

  ν ; ν~

(11)

and ν and ν~ represent the additive noise of the two detectors, which are generally modeled as white Gaussian [13,14].

Fig. 2. Formation process of uncoded grayscale measurement and coded CASSI measurement (1D view). 850

(13)

(8)

APPLIED OPTICS / Vol. 54, No. 4 / 1 February 2015

and the two-step iterative shrinkage/thresholding (TwIST) algorithm [29] to solve the optimization problem in Eq. (12). D.

Performance Analysis

Since the dual-camera design doubles the number of measurements in 1-frame CASSI, it is not surprising that the recovered image quality should be better. However, as can been seen later, the dual-camera design significantly outperforms 2-frame CASSI with the same number of measurements. The underlying reason is that the dual-camera design provides another kind of sampling that is complementary to CASSI. Specifically, in the CASSI model each pixel on the detector corresponds to a solid set of voxels in the 3D data cube, as shown in the lower branch in Fig. 2. For 2-frame CASSI, measurements at the same pixel of the two frames are contributed by the same set of voxels just with different weights controlled by the coded apertures. In contrast, in the dual-camera design, measurements at the same pixel of the two detectors are contributed by two sets of different voxels in the data cube. This property indicates a distinctive superposition of the source information, and thus relieves the burden of reconstruction. On the other hand, it should be noted that CASSI involves voxel shifts along one spatial dimension. So one main distortion introduced by CASSI is spatial blurring, which tends to make the recovered images overly smooth. In contrast, the grayscale camera in our proposed method integrates unshifted voxels solely along the spectral dimension. This complementary measurement thus plays an important role during the reconstruction, suppressing the spatial blurring in the recovered images. This will be demonstrated in the following simulation and experimental results.

Table 1.

Coherence Comparison of Sensing Matrices

Dual-Camera

2-Frame CASSI

0.6817

0.9345

Coherence

In CS theory, given the measurement matrix H and a sparsity basis D, we can simply obtain the sensing matrix Φ  HD. To efficiently evaluate the performance of Φ, a summary parameter of columns of Φ is used, which is called coherence and defined as the largest inner product between any two columns of Φ: uΦ  max

jhϕi ; ϕj ij

1≤i≠j≤N ‖ϕi ‖2 ‖ϕj ‖

:

(14)

Fig. 3. (a) System forward response of CASSI under 605 nm monochromatic light. (b) Spectral response curve of detector (Sony ICX414) used in CASSI. Both are from [11].

2

The lower coherence indicates the lower correlation between the measurements and superior performance of the whole sensing matrix [30–32]. We compare the coherence of the sensing matrices of a dual-camera design and 2-frame CASSI. The basis is chosen as D  D1 ⊗ D1 ⊗ D1 , where D1 is the Symmlet 8 wavelet basis. The measurement matrix H tested here corresponds to a 16 × 16 × 4 spectral cube. We run 1000 times Monte Carlo simulation to calculate the coherence and with an independent binary coded aperture each time. The average coherence result is shown in Table 1. We can see that the coherence of the sensing matrix of a dual-camera design is smaller than that of 2-frame CASSI, which proves that the correlation between an uncoded grayscale sampling and a coded and shifted sampling is lower than that between two coded and shifted samplings. So the performance of a dual-camera design should be superior to 2-frame CASSI.

PSNR  10  log10 I 2max ∕MSE;

(15)

where MSE 

−1 X n−1 X 1 m Ii; j − Ki; j2 ; m  n i0 j0

(16)

and I represents the reference band and K represents the corresponding reconstruction band. We fix τ at 0.2 (which is suggested in the original multiframe CASSI work [18]) and then calculate the objective function value and PSNR of the reconstruction result. As for the influence of τ, we fix the iteration

3. Simulation Results

In this section we evaluate the performance of the proposed dual-camera design in comparison with CASSI via simulation. We use the real system forward response of CASSI as reported in the seminal work [11], which consists of a matrix set with each matrix corresponding to one spectral band. As an example, the calibrated response under 605 nm monochromatic light is shown in Fig. 3(a). The system forward response of the graysale camera, which uses the same detector as in CASSI, is obtained by discretizing the detector’s spectral-response curve shown in Fig. 3(b). We use the spectral-image data set published in [33] for simulation. To match the real system response of CASSI, the test images are tailored with a 224 × 234 spatial resolution and 31 spectral bands spanning from 400 to 700 nm. All the intensity values are scaled to the interval from 0 to 1, and white Gaussian noise is added in generating the measurements. Typically, the weighting factor τ and the iteration number will influence the reconstruction fidelity. We test the algorithm performance versus τ and the iteration number, respectively. For the convenience of comparison we define the peak signal-to-noise value (PSNR) of each band as

Fig. 4. (a) Objective function value and PSNR versus iteration. (b) PSNR versus weighting factor τ. 1 February 2015 / Vol. 54, No. 4 / APPLIED OPTICS

851

Table 2.

PSNR (Db) Results of the Recovered Images with Different Levels of Noise (Averaging over 31 Spectral Bands)

σ  0.1 Image Set Balloons Beads CD Toy Egyptian Face Beer Lemon Straw-berry Sushi Average

σ  0.2

σ  0.4

DualCamera

1-Frame CASSI

2-Frame CASSI

DualCamera

1-Frame CASSI

2-Frame CASSI

DualCamera

1-Frame CASSI

2-Frame CASSI

33.4 23.0 28.8 26.7 38.7 35.7 34.1 32.5 33.3 34.8 32.1

25.6 17.0 24.5 19.5 30.4 27.8 25.1 21.9 24.9 26.6 24.3

28.8 19.1 26.9 22.3 32.0 30.3 27.9 25.0 27.2 29.1 26.9

32.5 22.6 28.3 25.4 37.2 33.7 32.5 30.2 32.1 33.6 30.8

25.3 16.4 24.4 18.1 29.2 27.5 24.8 21.7 24.7 26.1 23.8

28.1 18.9 26.2 19.6 31.1 29.5 27.3 24.4 26.4 28.1 26.0

30.3 21.5 26.6 23.8 35.4 31.7 30.9 28.3 29.9 32.1 29.0

24.5 16.3 23.6 17.9 27.9 26.3 23.8 21.2 23.6 24.5 23.0

26.8 18.0 25.0 19.2 29.4 27.8 26.0 23.4 25.1 26.1 24.7

number at 200 and then calculate the final PSNR. The results of performance evaluation for 2-frame CASSI are shown in Fig. 4, for example. From Fig. 4(a) it is obvious that when the iteration number is smaller than 50, the objective function value and PSNR change dramatically. When the iteration number exceeds 100, the objective function value and PSNR both converge. From Fig. 4(b) we can see the PSNR dependence on the weighting factor τ (the similar trend is found in the original multiframe CASSI work [18]). According to our test, the iteration number of TwIST is set as 100 for all methods. And the weighting factor τ is selected to achieve the best performance for each method, i.e., 0.5, 0.2, and 0.1 for 1-frame CASSI, 2-frame CASSI, and dual-camera design, respectively. Table 2 shows the PSNR results with different levels of noise. Note that the noise variance is within the same 0-to-1 numerical scale as the spectral image

intensity. Considering that the beam splitter will cut the light intensity by half in the proposed method, the noise level in the dual-camera measurements should double for a fairer comparison. Even so, it can be found that the proposed method significantly outperforms 1-frame CASSI and 2-frame CASSI. When the noise level is σ  0.2 for dual-camera design and σ  0.1 for the other two methods, the average PSNR gain of dual-camera design is 6.5 dB over 1-frame CASSI and 3.9 dB over 2-frame CASSI. As for the performance compared with multi-frame CASSI that uses more than two frames, we found that the result may change for different test images. For example, for the beads cube, the dual-camera design is competitive with 8-frame CASSI, while for the toy cube, the dual-camera design is competitive with up to 20-frame CASSI. Figure 5 gives the visual results of recovered images in one spectral band from two test-data cubes.

Fig. 5. Visual results of recovered images in one spectral band from two test-data cubes (top: beads; bottom: toy) with the noise level σ  0.1 for 1-frame CASSI and 2-frame CASSI, and σ  0.2 for dual-camera design. 852

APPLIED OPTICS / Vol. 54, No. 4 / 1 February 2015

Fig. 6. (a) Spectral signature at the center point of the selected lines in Fig. 5. (b) RMSE of spectral signature at all 20 points along the selected line. Top row is for the beads cube and bottom row is for the toy cube.

As can be seen, it is difficult for 1-frame CASSI to faithfully reconstruct the detailed information in original-data cubes. Specifically, the above mentioned spatial blurring can be observed in the recovered images. The quality of reconstruction is improved by 2-frame CASSI, yet the improvement is limited. In comparison, the proposed dual-camera design successfully reproduces the rich details of the underlying scene, e.g., the edges of the beads and the hair of the toy. Such superiority on the perceptual quality can be also found in other bands and data cubes not displayed. To further analyze the spectral performance of dual-camera design, we select a line consisting of 20 consecutive points with its center point at high spatial frequency (i.e., edges) as labeled in Fig. 5. For each method the spectral signature of this center point is plotted in Fig. 6(a). It can be easily noticed that the spectral signature of dual-camera design is much closer to the ground truth, which suggests the spectrum accuracy at high spatial frequency is significantly improved. On the other hand, we also show the spectral accuracy profile of the selected lines in terms of the root mean square error (RMSE) of the spectral signature at the 20 points versus the spatial location in Fig. 6(b). It can be seen that the RMSE is significantly reduced with dual-camera design, which suggests that the spectral performance is improved at different spatial locations.

4. System Implementation A. Setup

Figure 7 shows our system built for dual-camera experiments, which consists of a beam splitter in front of a suite of CASSI and a grayscale camera in the reflection direction of the beam splitter. In this setup the incident light can be simultaneously captured by the grayscale camera and CASSI with the portions assigned by the beam splitter. Here we choose a halfreflect, half-pass beam splitter to assign equal light for the two cameras. In this work the optical filter’s passband is from 460 to 650 nm. The manufactured

Fig. 7. System of dual-camera design for CASSI. 1 February 2015 / Vol. 54, No. 4 / APPLIED OPTICS

853

Fig. 8. (a) Image of CASSI coded aperture. (b) Image of secondary coded aperture on grayscale detector. (c) Image of secondary coded aperture on CASSI detector. All are under 584 nm monochromatic light.

coded aperture is a random binary pattern with 300 × 300 elements by realizing the Bernoulli distribution with a probability p  0.5. We use a twodirectional translator (Thorlabs CXY1) to maneuver the coded aperture. By adjusting this translator the coded aperture can be translated horizontally and vertically with high precision, which facilitates the implementation of 2-frame CASSI. Owing to the relay prism, each element of the coded aperture is optically mapped to 2 × 2 pixels on the detector. A double Amici prism vertically disperses the spectral information with the center wavelength at 550 nm. Both cameras are identical PointGray detectors (model FL3-U3-13Y3M), which are synchronized using the PointGray MultiSync software. B.

Calibration

CASSI calibration has already been well studied in [12,18,21]. Following the same procedures, we calibrate CASSI with particular attention to the impact of nonuniform illumination, nonlinear dispersion,

exposure time control, and noise reduction. The coded aperture is illuminated with different monochromatic light emitted by a commercial monochromator (Zolix Omni-λ 300). The illumination intensity of the monochromator at each wavelength is obtained using a spectrometer. By recording the image of the coded aperture, the whole spectrum spanning over the optical filter’s passband is discretized into 25 bands with different intervals. The image of the coded aperture at the center of each interval is used to act as the system forward operator. Figure 8(a) exhibits the image of the coded aperture under the monochromatic light of 584 nm. To implement 2-frame CASSI with distinct coded apertures, we slightly translate the coded aperture between two CASSI measurements to ensure a unique contribution to the measurement matrix. Two sets of calibration matrices accounting for the corresponding measurements are necessary information for 2-frame CASSI reconstruction. So we conduct calibration twice right before and after obtaining the two CASSI measurements, respectively. The calibration between the grayscale camera and CASSI is essential for dual-camera design. As mentioned above the dispersed spectrum is not linearly distributed on the CASSI image plane with respect to wavelength. Originally, we checked the correspondence between the grayscale image and the projection of each spectral wavelength on the CASSI image plane. Owing to the calibration of CASSI, however, we only need to align the grayscale image with the projection of one wavelength, and the alignment with other wavelengths can be easily deduced. To achieve this we place a secondary coded aperture in front of the beam splitter to act as an objective scene. This secondary coded aperture is illuminated by the monochromatic light and captured by both cameras, resulting in images as shown in Figs. 8(b) and 8(c). Note that here we merely care about the boundaries of the secondary coded aperture, not its content. Once the CASSI suite is fixed, we first fine tune the position of the grayscale camera so that the secondary coded aperture occupies a square area with the same resolution on the two detectors. Then, under the illumination at one specified wavelength, the two images jointly determine the correspondence

Fig. 9. (a) Test scene of X-rite Colorchecker. (b) Spectral signature in area 1. (c) Spectral signature in area 2. 854

APPLIED OPTICS / Vol. 54, No. 4 / 1 February 2015

Fig. 10. Reconstruction results of all 25 spectral bands for X-rite Colorchecker by the proposed method.

between the grayscale measurement and the CASSI measurement. 5. Experimental Results

To validate the effectiveness of dual-camera design, we test our system for different scenes and compare

the results with those from 1-frame CASSI and 2-frame CASSI, both qualitatively and quantitatively. In real experiments each branch of a dualcamera system receives 50% of the power, while every shot of 2-frame CASSI receives 100% of the power. (In other words, every shot of 2-frame CASSI

Fig. 11. Reconstruction results of spectral band at 595 nm. (a) 1-frame CASSI, (b) 2-frame CASSI, and (c) dual-camera design.

Fig. 12. (a) Test scene of paper flowers. (b) Spectral signature in area 1. (c) Spectral signature in area 2. 1 February 2015 / Vol. 54, No. 4 / APPLIED OPTICS

855

Fig. 13. Reconstruction results of 12 spectral bands from 534 to 650 nm. (a) 1-frame CASSI, (b) 2-frame CASSI, and (c) dual-camera design.

856

APPLIED OPTICS / Vol. 54, No. 4 / 1 February 2015

uses the same setting as 1-frame CASSI.) Therefore, if we want to obtain the same exposure for the above systems we can, (1) under the same ISO of the sensor, set the exposure time of a dual-camera system double that of 1-frame CASSI, or (2) under the same exposure time, set the ISO of a dual-camera system double that of 1-frame CASSI. We choose the latter to guarantee that the dual-camera system has the same acquisition speed as 1-frame CASSI. The result will be, naturally, that the noise level in a dualcamera system doubles when compared with both 1-frame and 2-frame CASSI. The first scene we test is a commonly used X-rite Colorchecker, as shown in Fig. 9(a). Under the laboratory lighting condition it takes 50 ms for 1-frame CASSI and a dual-camera system to achieve a proper exposure. For 2-frame CASSI, it takes 100 ms to conduct twice the exposure and extra time to translate the coded aperture. The reconstruction algorithm and parameter setting are the same as used in simulation. Figure 10 shows all 25 spectral bands reconstructed from the proposed method. Due to space limitation we compare the recovered image of one selected band with that from 1-frame CASSI and 2-frame CASSI. Figure 11 shows the three results of the band at 595 nm. We can see that while 2-frame CASSI provides limited improvement over 1-frame CASSI, dual-camera design significantly outperforms both of them in terms of the perceptual quality. Specifically, the spatial blurring introduced by CASSI has been eliminated in the dual-camera design, e.g., around the square boundaries marked by the red box and the characters marked by the green box. Note that other bands not shown here also exhibit similar comparison results. For quantitative assessment of the proposed method, we further compare the averaging spectral signatures in two areas as labeled in Fig. 9(a). The ground-truth value of the spectral signature is measured by a scanning spectrometer (StellarNet BLKCXR-SR-50 with 1.3 nm spectral resolution and 220–1100 nm range). Note that the non-uniform band intervals and the quantum efficiency of the spectrometer should be taken into consideration. All spectral signatures are normalized by the total energy of each area. The plots in Fig. 9 show the spectral signature comparison for two selected areas and the superior performance of the dual-camera design compared with 1-frame CASSI and 2-frame CASSI is evident. For the second experiment we test two color flowers made from fluorescent paper as shown in Fig. 12(a). The reconstruction results from three methods are displayed in Fig. 13. Since little content is present at wavelengths shorter than 534 nm, we select the last 12 spectral bands for comparison. Apparently, the highest-quality reconstruction is obtained by the proposed method. Furthermore, the plots in Fig. 12 give the spectral signatures in two selected areas in the scene, from which we can see that the improvement of the dual-camera design is in accordance with the visual results in Fig. 13.

Table 3.

RMSE Comparison of Spectral Signature in Selected Areas

1-Frame CASSI 2-Frame CASSI Dual-Camera Colorchecker 1 Colorchecker 2 Flower 1 Flower 2

0.054 0.0414 0.0456 0.0425

0.0471 0.0404 0.0396 0.0388

0.0284 0.0166 0.0188 0.0197

In addition, for the selected areas in both test scenes, quantitative evaluation for the spectrum accuracy in terms of the normalized RMSE is given in Table 3, from which the superiority of dual-camera design is demonstrated quantitatively. 6. Discussion

In this section we conduct a summary for the comparison between 1-frame CASSI, multi-frame CASSI, and the proposed dual-camera design in terms of five aspects: reconstruction accuracy, exposure time, light efficiency, calibration effort, and extra device. Reconstruction accuracy. As demonstrated in the simulation and experimental results, the reconstruction accuracy of dual-camera design is much higher compared with 1-frame CASSI and also superior to 2-frame CASSI. Still, as the frame number increases, the accuracy of multi-frame CASSI is expected to improve gracefully and outperform dualcamera design eventually. Exposure time. The main advantage of dualcamera design over multi-frame CASSI is snapshot. The former only requires one time exposure as for 1-frame CASSI, while the latter requires multiple exposures as well as additional time for coded aperture change. Snapshot is considered an essential attribute of CASSI, which makes it capable of dynamic scene capture. Light efficiency. Both 1-frame and multi-frame CASSI employ random binary coded apertures, which lead to 50% loss of light intensity. In dualcamera design the overall light efficiency is 75% with an equally dividing beam splitter, as the grayscale camera without a coded aperture will not sacrifice light intensity. In this sense, dual-camera design makes better use of light. Calibration effort. Compared with 1-frame CASSI, dual-camera design involves more calibration effort to align the CASSI and grayscale measurements. (For multi-frame CASSI, it depends on the device used for coded aperture change.) Fortunately, the calibration process only needs to be performed once in advance. Extra device. The main disadvantage of dualcamera design over 1-frame CASSI is that a second camera is needed. However, this additional cost is limited. Compared with multi-frame CASSI using a translation stage or digital micro-mirror device (DMD), dual-camera design provides an alternative solution with potentially lower cost, and most importantly, the snapshot merit. 1 February 2015 / Vol. 54, No. 4 / APPLIED OPTICS

857

7. Conclusion and Future Work

The performance of CASSI can be further improved with more shots of the scene, yet at the cost of compromising the snapshot advantage, which makes CASSI distinct from traditional spectral imagers. The proposed dual-camera design, by introducing a grayscale measurement for a complementary uniform sampling, greatly improves the reconstruction quality of CASSI while maintaining its snapshot advantage. Besides acting directly as extra measurements for the reconstruction, the grayscale image may provide other auxiliary information to guide the reconstruction. For example, edge and texture regions can be partitioned from the grayscale image, based on which advanced CS algorithms may be applied for even improved performance. This direction is considered as our future work. This work was performed during the internship of Lizhi Wang at Microsoft Research. This work was supported by the Major State Basic Research Development Program of China (973 Program, No. 2013CB329402), the National Natural Science Foundation of China (Nos. 61425026, 61227004, 61372131, and 31300473), and the Fund of New Scientific Stars of Shanxi Province (No. 2013KJXX-80). References 1. N. Gat, S. Subramanian, J. Barhen, and N. B. Toomarian, “Spectral imaging applications: remote sensing, environmental monitoring, medicine, military operations, factory automation and manufacturing,” Proc. SPIE 2962, 63–77 (1997). 2. C. Chang, Hyperspectral Imaging: Techniques for Spectral Detection and Classification (Springer, 2003), Vol. 1. 3. A. Plaza, J. A. Benediktsson, J. W. Boardman, J. Brazile, L. Bruzzone, G. Camps-Valls, J. Chanussot, M. Fauvel, P. Gamba, A. Gualtieri, M. Marconcini, J. C. Tilton, and G. Trianni, “Recent advances in techniques for hyperspectral image processing,” in Remote Sensing of Environment (Elsevier, 2009), Vol. 113, pp. S110–S122. 4. A. Gowen, C. O’Donnell, P. Cullen, G. Downey, and J. Frias, “Hyperspectral imaging–an emerging process analytical tool for food quality and safety control,” in Trends in Food Science & Technology (Elsevier, 2007), Vol. 18, pp. 590–598. 5. Y. Garini, I. T. Young, and G. McNamara, “Spectral imaging: principles and applications,” Cytometry Part A 69, 735–747 (2006). 6. D. J. Brady, Optical Imaging and Spectroscopy (Wiley, 2008). 7. B. Ford, M. Descour, and R. Lynch, “Large-image-format computed tomography imaging spectrometer for fluorescence microscopy,” Opt. Express 9, 444–453 (2001). 8. J. Yue, J. Han, Y. Zhang, and L. Bai, “High-throughput deconvolution-resolved computational spectrometer,” Chin. Opt. Lett. 12, 043001 (2014). 9. X. Lin, G. Wetzstein, Y. Liu, and Q. Dai, “Dual-coded compressive hyperspectral imaging,” Opt. Lett. 39, 2044–2047 (2014). 10. M. E. Gehm, R. John, D. J. Brady, R. M. Willett, and T. J. Schulz, “Single-shot compressive spectral imaging with a dual-disperser architecture,” Opt. Express 15, 14013–14027 (2007). 11. A. A. Wagadarikar, R. John, R. Willett, and D. J. Brady, “Single disperser design for coded aperture snapshot spectral imaging,” Appl. Opt 47, B44–B51 (2008).

858

APPLIED OPTICS / Vol. 54, No. 4 / 1 February 2015

12. A. A. Wagadarikar, N. P. Pitsianis, X. Sun, and D. J. Brady, “Video rate spectral imaging using a coded aperture snapshot spectral imager,” Opt. Express 17, 6368–6388 (2009). 13. H. Arguello, C. V. Correa, and G. R. Arce, “Fast lapped block reconstructions in compressive spectral imaging,” Appl. Opt. 52, D32–D45 (2013). 14. H. Arguello and G. R. Arce, “Colored coded aperture design by concentration of measure in compressive spectral imaging,” IEEE Trans. Image Process. 23, 1896–1908 (2014). 15. G. R. Arce, D. J. Brady, L. Carin, H. Arguello, and D. S. Kittle, “Compressive coded aperture spectral imaging: an introduction,” IEEE Signal Process. Mag. 31(1), 105–115 (2014). 16. D. L. Donoho, “Compressed sensing,” IEEE Trans. Inf. Theory 52, 1289–1306 (2006). 17. E. J. Candes, “Compressive sampling,” in Proceedings of the International Congress of Mathematicians (European Mathematical Society, 2006), pp. 1433–1452. 18. D. Kittle, K. Choi, A. A. Wagadarikar, and D. J. Brady, “Multiframe image estimation for coded aperture snapshot spectral imagers,” Appl. Opt. 49, 6824–6833 (2010). 19. H. Arguello and G. R. Arce, “Code aperture optimization for spectrally agile compressive imaging,” J. Opt. Soc. Am. A 28, 2400–2413 (2011). 20. Y. Wu, I. O. Mirza, G. R. Arce, and D. W. Prather, “Development of a digital-micromirror-device-based multishot snapshot spectral imaging system,” Opt. Lett. 36, 2692–2694 (2011). 21. H. Arguello, H. Rueda, Y. Wu, D. W. Prather, and G. R. Arce, “Higher-order computational model for coded aperture spectral imaging,” Appl. Opt. 52, D12–D21 (2013). 22. B. Aiazzi, S. Baronti, and M. Selva, “Improving component substitution pansharpening through multivariate regression of ms+pan data,” IEEE Trans. Geosci. Remote Sens. 45, 3230– 3239 (2007). 23. L. Alparone, L. Wald, J. Chanussot, C. Thomas, P. Gamba, and L. M. Bruce, “Comparison of pansharpening algorithms: Outcome of the 2006 grs-s data-fusion contest,” IEEE Trans. Geosci. Remote Sens. 45, 3012–3021 (2007). 24. C. Ma, X. Cao, X. Tong, Q. Dai, and L. Stephen, “Acquisition of high spatial and spectral resolution video with a hybrid camera system,” Int. J. Comput. Vis. 110, 141–155 (2014). 25. C. Ma, X. Cao, R. Wu, and Q. Dai, “Content-adaptive highresolution hyperspectral video acquisition with a hybrid camera system,” Opt. Lett. 39, 937–940 (2014). 26. X. Cao, H. Du, X. Tong, Q. Dai, and S. Lin, “A prism-mask system for multispectral video acquisition,” IEEE Trans. Pattern Anal. Mach. Intell. 33, 2423–2435 (2011). 27. E. J. Candes, J. Romberg, and T. Tao, “Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information,” IEEE Trans. Inf. Theory 52, 489–509 (2006). 28. A. Chambolle, “An algorithm for total variation minimization and applications,” J. Math. Imaging Vis. 20, 89–97 (2004). 29. J. Bioucas-Dias and M. Figueiredo, “A new twist: two-step iterative shrinkage/thresholding algorithms for image restoration,” IEEE Trans. Image Process. 16, 2992–3004 (2007). 30. M. F. Duarte and Y. C. Eldar, “Structured compressed sensing: from theory to applications,” IEEE Trans. Signal Process. 59, 4053–4085 (2011). 31. E. J. Cands and J. Romberg, “Sparsity and incoherence in compressive sampling,” Inverse Probl. 23, 969–985 (2007). 32. L. Carin, D. Liu, and B. Guo, “Coherence, compressive sensing, and random sensor arrays,” IEEE Antennas Propag. Mag. 53(4), 28–39 (2011). 33. F. Yasuma, T. Mitsunaga, D. Iso, and S. K. Nayar, “Generalized assorted pixel camera: postcapture control of resolution, dynamic range, and spectrum,” IEEE Trans. Image Process. 19, 2241–2253 (2010).

Dual-camera design for coded aperture snapshot spectral imaging.

Coded aperture snapshot spectral imaging (CASSI) provides an efficient mechanism for recovering 3D spectral data from a single 2D measurement. However...
2MB Sizes 0 Downloads 10 Views