Compressive sensing in the EO/IR M. E. Gehm* and D. J. Brady Duke University ECE Department, Box 90291, Durham, North Carolina 27708, USA *Corresponding author: [email protected] Received 4 November 2014; revised 13 February 2015; accepted 17 February 2015; posted 19 February 2015 (Doc. ID 226157); published 10 March 2015
We investigate the utility of compressive sensing (CS) to electro-optic and infrared (EO/IR) applications. We introduce the field through a discussion of historical antecedents and the development of the modern CS framework. Basic economic arguments (in the broadest sense) are presented regarding the applicability of CS to the EO/IR and used to draw conclusions regarding application areas where CS would be most viable. A number of experimental success stories are presented to demonstrate the overall feasibility of the approaches, and we conclude with a discussion of open challenges to practical adoption of CS methods. © 2015 Optical Society of America OCIS codes: (110.1758) Computational imaging; (120.0280) Remote sensing and sensors. http://dx.doi.org/10.1364/AO.54.000C14
1. Introduction
In April of 2014, the Optical Society of America (OSA) sponsored an incubator meeting [Implications of Compressive Sensing Concepts to Imaging Systems]. The goal of the meeting was to explore the broad future potential of compressive sensing (CS) to imaging by building upon a frank assessment of the accomplishments of the field to date. This manuscript attempts to capture the essence of the incubator presentations and discussion related to CS in electro-optic and infrared (EO/IR) applications. It is not a true review article of the field—while we attempt to present a historical timeline that we believe elucidates the general progress of the field, we do not claim that the works we mention reflect the definitive and full history of the subject or of the many researchers who have contributed to the overall progress. Rather, the items we cite are merely those that most directly influenced our own understanding of the history and present state of the field. The manuscript is organized into four main parts: the aforementioned historical overview; a discussion of the relevance of CS to EO/IR; an investigation of
1559-128X/15/080C14-09$15.00/0 © 2015 Optical Society of America C14
APPLIED OPTICS / Vol. 54, No. 8 / 10 March 2015
several success stories; and an overview of several open challenges for the future. 2. Antecedents and Emergence of the Field
It is rare for a scientific discovery or engineering development to have absolutely no connection to ideas and achievements of the past. Compressive sensing is no exception to this general trend; there exist numerous antecedents in the literature that touch upon many key ideas of CS. Nonetheless, we believe that the modern conception and formulation of CS definitively represents something intellectually distinct from those earlier works. This section describes several of those antecedents and key points in the development of modern CS framework. A. Antecedents
Generally speaking, CS addresses the challenge of recovering “sensible” solutions to an underdetermined inverse problem through the use of additional information regarding precisely what constitutes such a sensible solution. Given such a broadly applicable problem statement, it is no surprise that attacks on this and related problems have a long history in the literature. Indeed, the OSA has a history of supporting the development of such methods that dates back to at least 1983 through its Signal Recovery and Synthesis technical area.
In the optical domain, primary antecedents to CS include super resolution/deconvolution, color imaging, and phase retrieval. The relationship between these technologies and CS is illuminated with reference to basic sampling theory. Shannon’s sampling theorem states that a band-limited function can be exactly recovered from periodic samples taken at a rate equal to or greater than Nyquist. Under classical theory, the number of samples taken from a Nyquist sampled signal is N 2XB, where X is the spatial or temporal extent of the signal and B is the band limit. While compressive sampling consists of estimating N signal values from M < N discrete samples, most CS systems do not attempt to increase the physically limited bandwidth of imaging systems. Super resolution, in contrast, does not generally concern itself with discrete analysis or the number of samples taken. Rather, super resolution focuses explicitly on transformations that produce estimated signals with band limits in excess of the measured signal. Optical signals may be band limited by diffraction or by geometric aberration. Efforts to achieve resolution in excess of the diffraction limit date to the dawn of the computational age [1]. Deconvolution strategies to overcome optical blur also have ancient lineage, most famously in the context of aberrated images from the Hubble space telescope. Super resolution remains a vibrant and growing field [2–4]; indeed, it was the focus of the 2014 Nobel Prize in Chemistry. In contrast with algorithmic approaches, however, the Nobel-winning studies consist of methods for physically increasing the bandpass via near-field [5,6] or nonlinear [7–11] techniques. While super resolution has been presented from a CS perspective [12], deconvolution as a method for increasing signal band limits is not a generally accepted technique. The primary difference between CS and super resolution can be understood metaphorically as the difference between interpolation and extrapolation. Conventional super resolution attempts to extrapolate signal values in unsampled regions (e.g., Fourier space beyond the band limit) from densely sampled data. Most CS systems, in contrast, attempt to interpolate signal values residing between measurements in sparsely sampled regions. CS shares its focus on interpolation with color imaging techniques. Color imaging, most typically achieved using the Bayer color filter array, consists of estimating a 3D spectral data cube from measurements taken with a 2D sensor array. Most typically, three colors are estimated from data “demosaiced” from interlaced red, green, and blue measurements. As with super resolution, color demosaicing has been implemented to positive effect using modern decompressive inference algorithms [13]. The main distinction between classical color imaging and modern CS resides in the use of regular periodic sampling. Compressive sensing theory, and several recent studies of quasi-random color filter arrays, obtains signal fidelity benefits from less structured sampling approaches.
In electromagnetics and optics, phase retrieval is the subfield concerned with estimating the electric field given only irradiance measurements (as the irradiance measurements fix the amplitude of the unknown field, full field recovery reduces to the problem of estimating the phase content). Given the Fourier relationship that holds in the context of Fraunhofer propagation, this problem is also analogous to the general problem of determining a Fourier transform pair that satisfies fairly arbitrary domain constraints (e.g., modulus/support constraints). Foundational contributions to the field of phase retrieval were made in the 1970s and 1980s by a host of authors, with particularly relevant contributions from Fienup and Gonsalves [14–17]. Decades of advancement and important results followed—with a particularly famous success being the remote diagnosis of the unexpected aberrations in the Hubble Space Telescope [18]. These works, and others, clearly explore the role that prior knowledge can play in finding the solution to an underdetermined problem. Despite this similarity, however, we argue that the work in phase retrieval is clearly an antecedent to CS and not a prior discovery. In 1984, Johnson and Lindenstrauss published their eponymous lemma (first stated in 1982) regarding approximate distance-preserving projections from high to low dimension [19]. In a linear space, approximate preservation of distance implies (e.g., through the law of cosines) the approximate preservation of angles and, hence, inner products. This, in turn, implies the approximate preservation of linear functionals (the mapping of a vector to a scalar—e.g., measurement) Thus, this work provided evidence that underdetermined measurement problems could be formulated that were approximately information preserving and, therefore, that the solution of such general underdetermined problems was, in fact, possible—provided that the class of “sensible” solutions was not nearly all-encompassing. The 1980s and 1990s also saw the blossoming of wavelet concepts and their concurrent analysis through the lens of frame theory, with key works by Coifman, Daubechies, Mallat, Meyer, and others [20–26]. This development created an understanding of how multiresolution signal representations in terms of jointly localized functions provided an extremely efficient descriptive mechanism for broad classes of naturally occurring signals—the emergence of the concept of generalized signal sparsity. This stands in stark contrast to the impulse/sinusoid decompositions of conventional Fourier theory, where sparsity [such as under the discrete cosine transform (DCT)] exists only for band-limited/lowpass signals. These two underlying insights—that certain classes of signals admit the possibility of general underdetermined-yet-information-preserving problems, along with generalized, sparsifying signal representations that strongly suggest natural signals are such a class—form the true foundation of CS 10 March 2015 / Vol. 54, No. 8 / APPLIED OPTICS
C15
and mark it as a distinct break from the historical antecedents. B.
Emergence of the Field
Despite these developments, formalization of modern CS was still a number of years in the future. The following subsections provide a brief timeline of the emergence of the field from these early insights. 1. Rise of Intuition In the optical domain, these insights eventually precipitated the understanding that natural signals look nothing like random signals (static) and that using a naïve basis to measure a signal that was easily compressible via conventional algorithms was wasteful of sensor resources—if the signal was ultimately representable by a smaller set of numbers, the sensor should seek to measure only those numbers in the first place. As early as 2002–2004, Brady and Neifeld were drawing upon physical intuition to motivate exploration of sensor systems that applied these ideas (and related concepts from combinatorial group testing) to the acquisition of compressed measurements and subsequent reconstruction of the signals. Their methods, referred to at the time as physical-layer compression, sensor-layer compression, and featurespecific imaging were ultimately applied to 2D imagers, 3D imagers, wavemeters, and motion detectors [27–31]. The use of coded apertures or reference structures to condition ill-posed tomographic imaging systems in these studies was anticipated two decades earlier by Smith and co-workers [32,33]. Roughly concurrently, Christensen and collaborators were developing sensor systems that explicitly relied on the fact that the information content in a scene (hyperspectral, in their specific case) was not equivalent to the number of signal elements in the naïve representation [34]. Their implementation however, did not use sparsity or localization in some generalized transform basis but rather in the conventional spatial/spectral domains of the hyperspectral datacube. Nonetheless, this work also represents a clear demonstration of the growing physical intuition related to natural signals. 2. Random Sampling and L1 In 2004–2006, the formalization of CS took a major step forward with the near-simultaneous development and publication of the works by Donoho [35] and Candes et al. [36]. These works built upon years of continued mathematical development (by the authors and others in the field) from the seeds of Johnson–Lindenstrauss and wavelet theory and provided three key results. First, provided that certain conditions were met, the solution of underdetermined problems could be achieved with probabilistic guarantees of success. Second, that random sampling was a sufficient form of measurement to meet the required conditions; measurement strategies did not need to be tailored to the specific nature of the signals C16
APPLIED OPTICS / Vol. 54, No. 8 / 10 March 2015
of interest. Third, that, although finding the sparsest solution (minimum L0-norm) that was consistent with the measurements was in general a computationally intractable problem, finding the signal with the minimum L1-norm was both computationally tractable via linear programming and frequently yielded the same answer as the L0 approach. Thus, data consistency combined with L1-regularization was a viable reconstruction method. Almost immediately, Baraniuk and Kelly produced a concrete demonstration of this approach in the form of the Rice single-pixel camera, which remains the most wellknown implementation of CS to date [37]. 3. Modern Framework The modern form of CS looks very similar to the version set forth in 2004–2006, albeit with the addition of a number of important insights and improvements. These include: • The recovery guarantees have proven to be of greater theoretical interest to the signal processing community than of practical interest to the optics community. • It has been demonstrated that designed measurements outperform random measurements (indeed the random measurements were primarily considered only because they facilitated the proof of the recovery guarantees). See [38] for an early exploration of this effect. However, the performance gain from designed measurements over random measurements varies dramatically based on sensor task (e.g., detection versus reconstruction), naïve signal dimension, noise, and other signal and system properties. In cases where the gain is minimal, the ease of random design may outweigh any potential benefit from design. • A variety of other signal priors beyond basic sparsity have been developed and/or used for regularization—structured sparsity [39] and gradient sparsity (i.e., total variation, a previously developed concept applied to signal recovery in e.g., [40]) being of particular interest. • Many recovery algorithms now exist (see [41] for a noncomprehensive list), with varying performance and resource trade-offs with respect to the original linear programming solution. In addition, methods based on Bayesian inference have also become popular for certain categories of sensor tasks (e.g., classification) [42]. • The time-sequential nature of the single pixel framework is now understood to limit measurement SNR and/or the ability to measure dynamic systems. In the optical domain where detector arrays are plentiful and inexpensive, space-parallel solutions have emerged that remove these limitations in exchange for an increased aperture requirement. • Adaptive (on-line) measurement design and inference has been shown to provide significant performance improvements, particularly in sensor tasks other than reconstruction [43].
3. Relevance of CS to EO/IR
To address the relevance of CS to EO/IR applications, we must first answer the question “Where does CS offer potential value?” Applying basic economic thought to that question results in the nearly tautological conclusion that CS has potential value whenever the marginal cost (in size, weight, power, price, etc.) of a measurement is high. While this is obviously highly situation-dependent, we can infer certain general trends. First, we consider component costs. We focus on four aspects: detectors (price), detectors (size and power), optical elements (price), and bandwidth (price, size, and power). Detector price varies dramatically across the electromagnetic spectrum. Approximate per-pixel prices for detector elements in different spectral bands are shown in Table 1. From this, we see immediately that measurements in the near-infrared (NIR), visible (VIS), near ultraviolet (NUV), and medium ultraviolet (MUV) are extremely inexpensive and that the per-pixel price grows rapidly as we move to shorter or longer wavelengths (the one potential bright spot being the availability of Si-based direct detection methods in the soft x-ray band, which is primarily used for x-ray diffraction studies and not medical or security imaging). Detector size and power generally follow similar trends as price, given the inability to leverage the tremendous investments in Si process development outside the NIR/VIS/NUV and MUV. Likewise, optical element prices also have a similar variation as a function of spectral band— although the element prices are largely a fixed cost driven by aperture size and do not vary dramatically in a per-pixel sense. There is also a rollover in the cost trend for components at the extreme long-wavelength end, as simple plastics serve as functional dielectrics and the required fabrication tolerances are low. Finally, bandwidth costs scale with the number of pixels, but they are independent of wavelength and so have no impact on the applicability of CS to a particular spectral band. Combining these various component costs, we come to three conclusions regarding different spectral bands: First, in the NIR/VIS/NUV and Table 1.
Approximate Per-Pixel Price of Detector Elements in Various Spectral Bands
Spectral Band mmW/THz LWIR MWIR SWIR NIR/VIS/NUV MUV EUV Soft x ray Hard x ray/gamma
Detector Technology
Approx. Per-Pixel Price ($/pix)
Multiple HgCdTe Bolometer InSb/PbSe InGaAs/PbSe Si Si (thinned) Si-PIN/CdTe Si (thinned) Si-PIN/CdTe Multiple
102 –104
We investigate the utility of compressive sensing (CS) to electro-optic and infrared (EO/IR) applications. We introduce the field through a discussion of historical antecedents and the development of the modern CS framework. Basic economic arguments (in the broadest sense) are presented regarding the applicability of CS to the EO/IR and used to draw conclusions regarding application areas where CS would be most viable. A number of experimental success stories are presented to demonstrate the overall feasibility of the approaches, and we conclude with a discussion of open challenges to practical adoption of CS methods. © 2015 Optical Society of America OCIS codes: (110.1758) Computational imaging; (120.0280) Remote sensing and sensors. http://dx.doi.org/10.1364/AO.54.000C14
1. Introduction
In April of 2014, the Optical Society of America (OSA) sponsored an incubator meeting [Implications of Compressive Sensing Concepts to Imaging Systems]. The goal of the meeting was to explore the broad future potential of compressive sensing (CS) to imaging by building upon a frank assessment of the accomplishments of the field to date. This manuscript attempts to capture the essence of the incubator presentations and discussion related to CS in electro-optic and infrared (EO/IR) applications. It is not a true review article of the field—while we attempt to present a historical timeline that we believe elucidates the general progress of the field, we do not claim that the works we mention reflect the definitive and full history of the subject or of the many researchers who have contributed to the overall progress. Rather, the items we cite are merely those that most directly influenced our own understanding of the history and present state of the field. The manuscript is organized into four main parts: the aforementioned historical overview; a discussion of the relevance of CS to EO/IR; an investigation of
1559-128X/15/080C14-09$15.00/0 © 2015 Optical Society of America C14
APPLIED OPTICS / Vol. 54, No. 8 / 10 March 2015
several success stories; and an overview of several open challenges for the future. 2. Antecedents and Emergence of the Field
It is rare for a scientific discovery or engineering development to have absolutely no connection to ideas and achievements of the past. Compressive sensing is no exception to this general trend; there exist numerous antecedents in the literature that touch upon many key ideas of CS. Nonetheless, we believe that the modern conception and formulation of CS definitively represents something intellectually distinct from those earlier works. This section describes several of those antecedents and key points in the development of modern CS framework. A. Antecedents
Generally speaking, CS addresses the challenge of recovering “sensible” solutions to an underdetermined inverse problem through the use of additional information regarding precisely what constitutes such a sensible solution. Given such a broadly applicable problem statement, it is no surprise that attacks on this and related problems have a long history in the literature. Indeed, the OSA has a history of supporting the development of such methods that dates back to at least 1983 through its Signal Recovery and Synthesis technical area.
In the optical domain, primary antecedents to CS include super resolution/deconvolution, color imaging, and phase retrieval. The relationship between these technologies and CS is illuminated with reference to basic sampling theory. Shannon’s sampling theorem states that a band-limited function can be exactly recovered from periodic samples taken at a rate equal to or greater than Nyquist. Under classical theory, the number of samples taken from a Nyquist sampled signal is N 2XB, where X is the spatial or temporal extent of the signal and B is the band limit. While compressive sampling consists of estimating N signal values from M < N discrete samples, most CS systems do not attempt to increase the physically limited bandwidth of imaging systems. Super resolution, in contrast, does not generally concern itself with discrete analysis or the number of samples taken. Rather, super resolution focuses explicitly on transformations that produce estimated signals with band limits in excess of the measured signal. Optical signals may be band limited by diffraction or by geometric aberration. Efforts to achieve resolution in excess of the diffraction limit date to the dawn of the computational age [1]. Deconvolution strategies to overcome optical blur also have ancient lineage, most famously in the context of aberrated images from the Hubble space telescope. Super resolution remains a vibrant and growing field [2–4]; indeed, it was the focus of the 2014 Nobel Prize in Chemistry. In contrast with algorithmic approaches, however, the Nobel-winning studies consist of methods for physically increasing the bandpass via near-field [5,6] or nonlinear [7–11] techniques. While super resolution has been presented from a CS perspective [12], deconvolution as a method for increasing signal band limits is not a generally accepted technique. The primary difference between CS and super resolution can be understood metaphorically as the difference between interpolation and extrapolation. Conventional super resolution attempts to extrapolate signal values in unsampled regions (e.g., Fourier space beyond the band limit) from densely sampled data. Most CS systems, in contrast, attempt to interpolate signal values residing between measurements in sparsely sampled regions. CS shares its focus on interpolation with color imaging techniques. Color imaging, most typically achieved using the Bayer color filter array, consists of estimating a 3D spectral data cube from measurements taken with a 2D sensor array. Most typically, three colors are estimated from data “demosaiced” from interlaced red, green, and blue measurements. As with super resolution, color demosaicing has been implemented to positive effect using modern decompressive inference algorithms [13]. The main distinction between classical color imaging and modern CS resides in the use of regular periodic sampling. Compressive sensing theory, and several recent studies of quasi-random color filter arrays, obtains signal fidelity benefits from less structured sampling approaches.
In electromagnetics and optics, phase retrieval is the subfield concerned with estimating the electric field given only irradiance measurements (as the irradiance measurements fix the amplitude of the unknown field, full field recovery reduces to the problem of estimating the phase content). Given the Fourier relationship that holds in the context of Fraunhofer propagation, this problem is also analogous to the general problem of determining a Fourier transform pair that satisfies fairly arbitrary domain constraints (e.g., modulus/support constraints). Foundational contributions to the field of phase retrieval were made in the 1970s and 1980s by a host of authors, with particularly relevant contributions from Fienup and Gonsalves [14–17]. Decades of advancement and important results followed—with a particularly famous success being the remote diagnosis of the unexpected aberrations in the Hubble Space Telescope [18]. These works, and others, clearly explore the role that prior knowledge can play in finding the solution to an underdetermined problem. Despite this similarity, however, we argue that the work in phase retrieval is clearly an antecedent to CS and not a prior discovery. In 1984, Johnson and Lindenstrauss published their eponymous lemma (first stated in 1982) regarding approximate distance-preserving projections from high to low dimension [19]. In a linear space, approximate preservation of distance implies (e.g., through the law of cosines) the approximate preservation of angles and, hence, inner products. This, in turn, implies the approximate preservation of linear functionals (the mapping of a vector to a scalar—e.g., measurement) Thus, this work provided evidence that underdetermined measurement problems could be formulated that were approximately information preserving and, therefore, that the solution of such general underdetermined problems was, in fact, possible—provided that the class of “sensible” solutions was not nearly all-encompassing. The 1980s and 1990s also saw the blossoming of wavelet concepts and their concurrent analysis through the lens of frame theory, with key works by Coifman, Daubechies, Mallat, Meyer, and others [20–26]. This development created an understanding of how multiresolution signal representations in terms of jointly localized functions provided an extremely efficient descriptive mechanism for broad classes of naturally occurring signals—the emergence of the concept of generalized signal sparsity. This stands in stark contrast to the impulse/sinusoid decompositions of conventional Fourier theory, where sparsity [such as under the discrete cosine transform (DCT)] exists only for band-limited/lowpass signals. These two underlying insights—that certain classes of signals admit the possibility of general underdetermined-yet-information-preserving problems, along with generalized, sparsifying signal representations that strongly suggest natural signals are such a class—form the true foundation of CS 10 March 2015 / Vol. 54, No. 8 / APPLIED OPTICS
C15
and mark it as a distinct break from the historical antecedents. B.
Emergence of the Field
Despite these developments, formalization of modern CS was still a number of years in the future. The following subsections provide a brief timeline of the emergence of the field from these early insights. 1. Rise of Intuition In the optical domain, these insights eventually precipitated the understanding that natural signals look nothing like random signals (static) and that using a naïve basis to measure a signal that was easily compressible via conventional algorithms was wasteful of sensor resources—if the signal was ultimately representable by a smaller set of numbers, the sensor should seek to measure only those numbers in the first place. As early as 2002–2004, Brady and Neifeld were drawing upon physical intuition to motivate exploration of sensor systems that applied these ideas (and related concepts from combinatorial group testing) to the acquisition of compressed measurements and subsequent reconstruction of the signals. Their methods, referred to at the time as physical-layer compression, sensor-layer compression, and featurespecific imaging were ultimately applied to 2D imagers, 3D imagers, wavemeters, and motion detectors [27–31]. The use of coded apertures or reference structures to condition ill-posed tomographic imaging systems in these studies was anticipated two decades earlier by Smith and co-workers [32,33]. Roughly concurrently, Christensen and collaborators were developing sensor systems that explicitly relied on the fact that the information content in a scene (hyperspectral, in their specific case) was not equivalent to the number of signal elements in the naïve representation [34]. Their implementation however, did not use sparsity or localization in some generalized transform basis but rather in the conventional spatial/spectral domains of the hyperspectral datacube. Nonetheless, this work also represents a clear demonstration of the growing physical intuition related to natural signals. 2. Random Sampling and L1 In 2004–2006, the formalization of CS took a major step forward with the near-simultaneous development and publication of the works by Donoho [35] and Candes et al. [36]. These works built upon years of continued mathematical development (by the authors and others in the field) from the seeds of Johnson–Lindenstrauss and wavelet theory and provided three key results. First, provided that certain conditions were met, the solution of underdetermined problems could be achieved with probabilistic guarantees of success. Second, that random sampling was a sufficient form of measurement to meet the required conditions; measurement strategies did not need to be tailored to the specific nature of the signals C16
APPLIED OPTICS / Vol. 54, No. 8 / 10 March 2015
of interest. Third, that, although finding the sparsest solution (minimum L0-norm) that was consistent with the measurements was in general a computationally intractable problem, finding the signal with the minimum L1-norm was both computationally tractable via linear programming and frequently yielded the same answer as the L0 approach. Thus, data consistency combined with L1-regularization was a viable reconstruction method. Almost immediately, Baraniuk and Kelly produced a concrete demonstration of this approach in the form of the Rice single-pixel camera, which remains the most wellknown implementation of CS to date [37]. 3. Modern Framework The modern form of CS looks very similar to the version set forth in 2004–2006, albeit with the addition of a number of important insights and improvements. These include: • The recovery guarantees have proven to be of greater theoretical interest to the signal processing community than of practical interest to the optics community. • It has been demonstrated that designed measurements outperform random measurements (indeed the random measurements were primarily considered only because they facilitated the proof of the recovery guarantees). See [38] for an early exploration of this effect. However, the performance gain from designed measurements over random measurements varies dramatically based on sensor task (e.g., detection versus reconstruction), naïve signal dimension, noise, and other signal and system properties. In cases where the gain is minimal, the ease of random design may outweigh any potential benefit from design. • A variety of other signal priors beyond basic sparsity have been developed and/or used for regularization—structured sparsity [39] and gradient sparsity (i.e., total variation, a previously developed concept applied to signal recovery in e.g., [40]) being of particular interest. • Many recovery algorithms now exist (see [41] for a noncomprehensive list), with varying performance and resource trade-offs with respect to the original linear programming solution. In addition, methods based on Bayesian inference have also become popular for certain categories of sensor tasks (e.g., classification) [42]. • The time-sequential nature of the single pixel framework is now understood to limit measurement SNR and/or the ability to measure dynamic systems. In the optical domain where detector arrays are plentiful and inexpensive, space-parallel solutions have emerged that remove these limitations in exchange for an increased aperture requirement. • Adaptive (on-line) measurement design and inference has been shown to provide significant performance improvements, particularly in sensor tasks other than reconstruction [43].
3. Relevance of CS to EO/IR
To address the relevance of CS to EO/IR applications, we must first answer the question “Where does CS offer potential value?” Applying basic economic thought to that question results in the nearly tautological conclusion that CS has potential value whenever the marginal cost (in size, weight, power, price, etc.) of a measurement is high. While this is obviously highly situation-dependent, we can infer certain general trends. First, we consider component costs. We focus on four aspects: detectors (price), detectors (size and power), optical elements (price), and bandwidth (price, size, and power). Detector price varies dramatically across the electromagnetic spectrum. Approximate per-pixel prices for detector elements in different spectral bands are shown in Table 1. From this, we see immediately that measurements in the near-infrared (NIR), visible (VIS), near ultraviolet (NUV), and medium ultraviolet (MUV) are extremely inexpensive and that the per-pixel price grows rapidly as we move to shorter or longer wavelengths (the one potential bright spot being the availability of Si-based direct detection methods in the soft x-ray band, which is primarily used for x-ray diffraction studies and not medical or security imaging). Detector size and power generally follow similar trends as price, given the inability to leverage the tremendous investments in Si process development outside the NIR/VIS/NUV and MUV. Likewise, optical element prices also have a similar variation as a function of spectral band— although the element prices are largely a fixed cost driven by aperture size and do not vary dramatically in a per-pixel sense. There is also a rollover in the cost trend for components at the extreme long-wavelength end, as simple plastics serve as functional dielectrics and the required fabrication tolerances are low. Finally, bandwidth costs scale with the number of pixels, but they are independent of wavelength and so have no impact on the applicability of CS to a particular spectral band. Combining these various component costs, we come to three conclusions regarding different spectral bands: First, in the NIR/VIS/NUV and Table 1.
Approximate Per-Pixel Price of Detector Elements in Various Spectral Bands
Spectral Band mmW/THz LWIR MWIR SWIR NIR/VIS/NUV MUV EUV Soft x ray Hard x ray/gamma
Detector Technology
Approx. Per-Pixel Price ($/pix)
Multiple HgCdTe Bolometer InSb/PbSe InGaAs/PbSe Si Si (thinned) Si-PIN/CdTe Si (thinned) Si-PIN/CdTe Multiple
102 –104
