Compressive Sensing Focus Issue: introduction Joseph N. Mait,1,* Abhijit Mahalanobis,2 Mark A. Neifeld,3 and Ravindra A. Athale4 1
US Army Research Laboratory, RDRL-D, 2800 Powder Mill Road, Adelphi, Maryland 20783, USA 2
Lockheed Martin Corporation, MS 450, 5600 Sandlake Rd., Orlando, Florida 32819-0000, USA
3
University of Arizona, Department of Electrical and Computer Engineering, 1230 E. Speedway Blvd., Tucson, Arizona 85721, USA 4
Office of Naval Research, 875 N. Randolph St., Arlington, Virginia 22203, USA *Corresponding author: [email protected] Received 20 February 2015; posted 20 February 2015 (Doc. ID 235065); published 10 March 2015
On 10–11 April 2014, the Optical Society of America (OSA) hosted an incubator on compressive sensing at its Washington, D.C., headquarters. © 2015 Optical Society of America OCIS codes: (000.1200) Announcements, awards, news, and organizational activities; (000.1780) Conferences, lectures, and institutes; (000.5360) Physics literature and publications. http://dx.doi.org/10.1364/AO.54.000CS1
On 10–11 April 2014, the Optical Society of America (OSA) hosted an incubator on compressive sensing (CS) at its Washington, D.C., headquarters. The goal of this narrowly focused meeting was to ask tough questions about the promise and reality of CS, and to understand its potential impact on future sensor technology. To provide as complete an examination of the topic as possible, attendees represented different fields, such as visible imaging, medical imaging, and radio frequency imaging. The meeting was co-sponsored by the Office of Naval Research, the Army Research Laboratory, and Lockheed Martin Corporation. We organized the incubator in response to the claims of both the success and the failure of CS. This phenomenon, well documented in other fields, is characterized by the Gartner Technology Hype Cycle represented in Fig. 1. The claimed potential of a new, but untested, technology drives up its visibility. However, increased visibility implies increased scrutiny, which can drive a technology into a trough of disillusionment. From there it must retrench before emerging into some steady level of productivity and payoff.
1559-128X/15/080CS1-03$15.00/0 © 2015 Optical Society of America
We felt that to guide CS through this trough, it was worthwhile to consider the applications and markets that might help CS reach its plateau of productivity. The papers published in this special feature hopefully capture the present understanding of the field and provide a perspective for moving forward. Before summarizing the papers, we provide a context for them. With reference to Fig. 1, the technology triggers for CS are electronic image detection and post-detection electronic processing. In the beginning of optical design, optical instruments only formed images. There was no post-detection processing. With the development of chemical-based recorded imaging, image detection and storage were combined into a single step. Post-detection processing was off-line, manual, and analog, e.g., changes between images were noted by a human who viewed a series of photographs. Holography also incorporated an off-line optical form of post-detection processing, i.e., hologram recording and playback are two distinct steps. By separating storage from detection, electronic detection enabled electronic post-detection processing prior to storage and further transmission. Early approaches to post-detection processing made assumptions about signal priors and noise to improve estimates of conventional measurements. Common assumptions included non-negativity of 10 March 2015 / Vol. 54, No. 8 / APPLIED OPTICS
CS1
Fig. 1. Gartner Hype Cycle that describes the visibility of a technology as it matures. Untested claims increase visibility to a peak before increased inquiry drives the technology toward retrenchment. Enlightened understanding eventually leads to stable productivity.
measurements over a bounded extent and the presence or absence of specific spectra. However, the most fundamental assumption was that the processed imagery was meant for human use. In contrast, at the time computational imaging was introduced, machine consumption of imagery was possible. Under this assumption, designers are no longer constrained by conventional notions of imaging, such as point-spread functions. Instead, they can design measurements that emphasize the information in which one is interested. Consequently, if one exploits redundancy in the scene of interest, it is possible to reduce the number of measurements required to produce an image with a certain signal-to-noise ratio. As straight forward as this explanation of CS appears, its simplicity is exactly what contributes to inflated expectations. Stated more positively, it gives rise to skepticism. As we know, the devil lies in the details, which is what we heard at the incubator. For the benefit of the community, we asked several of the incubator speakers to co-author papers that summarize their assessment of CS applied to electrooptic imaging, medical imaging, and radio frequency imaging. The papers are “Compressive sensing in the EO/IR,” by Michael Gehm and David Brady, “Pitfalls and possibilities of radio frequency compressive sensing,” by Nathan Goodman and Lee Potter, and “Compressive sensing in medical imaging,” by Christian Graff and Emil Sidky. Instead of summarizing each paper here, we note several overarching themes and provide some general conclusions. Each of the papers notes yet another trigger for CS, the papers by Candes et al. and Donoho [1,2], which provided a mathematical foundation for CS. Candes et al. and Donoho derived conditions under which CS can be proven to provide an advantage. Although this prompted a flurry of activity, the authors note that Candes et al. and Donoho were not the first to introduce techniques to maintain performance using a small set of measurements. Further, the theoretical CS2
APPLIED OPTICS / Vol. 54, No. 8 / 10 March 2015
limits on performance quickly encounter the practicalities of implementation, which each of the papers addresses. From a technical standpoint, compressive techniques are at a disadvantage when they are retrofitted into commercially available and affordable technology. It is therefore difficult to show any advantage of CS for visible imaging, where current approaches allow complete modularity. In visible imaging, one must find an application where forming an image is undesired, e.g., security or surveillance where maintaining privacy is an issue that increases cost. CS can be used to augment, not replace, current imaging systems, e.g., in navigation. Systems incorporating compressive and computational processing offer an advantage when they can be redesigned from the ground up. It is evident that improved performance can be obtained if one redesigns the fields in a magnetic resonance imager or the beam patterns in a computerized axial tomography scan, but this requires considerable investment from companies such as Siemens and GE. The portion of the spectrum in which CS has perhaps the greatest potential for application includes the x-ray and the millimeter wave, where no commercial systems exist, where the technology to form images does not exist, and where arrays of detectors do not exist. New sensor designs based on coded apertures, computational optics, and information processing have led to multiframe coded aperture spectral snapshot imaging, compressive x-ray imaging, and knowledge-enhanced exapixel photography, among others. While these prototypes are novel, they have not yet entered the product space; they may do so in the future as the operational benefits of such devices become clear. We believe that the next development in optical sensing will be adaptive sensing, where the next measurement depends upon past measurements, and optics, detectors, and information processing will function together to optimize task performance, whether it is imaging or estimating relevant information such as patterns of interests, motion, or behavior. We conclude that CS will most likely make an impact in new and emerging areas, rather than in retrofit designs. Three areas that seem most promising include applications where privacy is a major issue, three-dimensional imaging for medicine and security, and wide-area, long-range IR sensing. CS can aid these areas in two ways. For cases where the cost of measurement is high, multiplexing information with codes reduces the number of such measurements that need to be made and offers the potential to reduce the overall system cost. CS can also reduce the penalties associated with measurements, e.g., reduce exposure to harmful radiation in medical imaging, reduce the data acquisition bottleneck in high-throughput systems, for example, baggage scanners, and reduce concerns about loss of privacy in security systems. In the last case, CS
techniques make it possible to avoid forming images at all. Finally, for CS to have an impact, it must overcome organizational and opportunity costs, not just the technical ones. Design teams founded on exploiting physics in conjunction with computational techniques require a different organizational structure than conventional teams. It is not possible to develop CS systems using separate departments for optics, detectors, and signal processing. To realize any
payoff, the three disciplines must be integrated. Each must feed off of an understanding of the other. References 1. E. Candes, J. Romberg, and T. Tao, “Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information,” IEEE Trans. Inf. Theory 52, 489–509 (2006). 2. D. Donoho, “Compressed sensing,” IEEE Trans. Inf. Theory 52, 1289–1306 (2006).
10 March 2015 / Vol. 54, No. 8 / APPLIED OPTICS
CS3
US Army Research Laboratory, RDRL-D, 2800 Powder Mill Road, Adelphi, Maryland 20783, USA 2
Lockheed Martin Corporation, MS 450, 5600 Sandlake Rd., Orlando, Florida 32819-0000, USA
3
University of Arizona, Department of Electrical and Computer Engineering, 1230 E. Speedway Blvd., Tucson, Arizona 85721, USA 4
Office of Naval Research, 875 N. Randolph St., Arlington, Virginia 22203, USA *Corresponding author: [email protected] Received 20 February 2015; posted 20 February 2015 (Doc. ID 235065); published 10 March 2015
On 10–11 April 2014, the Optical Society of America (OSA) hosted an incubator on compressive sensing at its Washington, D.C., headquarters. © 2015 Optical Society of America OCIS codes: (000.1200) Announcements, awards, news, and organizational activities; (000.1780) Conferences, lectures, and institutes; (000.5360) Physics literature and publications. http://dx.doi.org/10.1364/AO.54.000CS1
On 10–11 April 2014, the Optical Society of America (OSA) hosted an incubator on compressive sensing (CS) at its Washington, D.C., headquarters. The goal of this narrowly focused meeting was to ask tough questions about the promise and reality of CS, and to understand its potential impact on future sensor technology. To provide as complete an examination of the topic as possible, attendees represented different fields, such as visible imaging, medical imaging, and radio frequency imaging. The meeting was co-sponsored by the Office of Naval Research, the Army Research Laboratory, and Lockheed Martin Corporation. We organized the incubator in response to the claims of both the success and the failure of CS. This phenomenon, well documented in other fields, is characterized by the Gartner Technology Hype Cycle represented in Fig. 1. The claimed potential of a new, but untested, technology drives up its visibility. However, increased visibility implies increased scrutiny, which can drive a technology into a trough of disillusionment. From there it must retrench before emerging into some steady level of productivity and payoff.
1559-128X/15/080CS1-03$15.00/0 © 2015 Optical Society of America
We felt that to guide CS through this trough, it was worthwhile to consider the applications and markets that might help CS reach its plateau of productivity. The papers published in this special feature hopefully capture the present understanding of the field and provide a perspective for moving forward. Before summarizing the papers, we provide a context for them. With reference to Fig. 1, the technology triggers for CS are electronic image detection and post-detection electronic processing. In the beginning of optical design, optical instruments only formed images. There was no post-detection processing. With the development of chemical-based recorded imaging, image detection and storage were combined into a single step. Post-detection processing was off-line, manual, and analog, e.g., changes between images were noted by a human who viewed a series of photographs. Holography also incorporated an off-line optical form of post-detection processing, i.e., hologram recording and playback are two distinct steps. By separating storage from detection, electronic detection enabled electronic post-detection processing prior to storage and further transmission. Early approaches to post-detection processing made assumptions about signal priors and noise to improve estimates of conventional measurements. Common assumptions included non-negativity of 10 March 2015 / Vol. 54, No. 8 / APPLIED OPTICS
CS1
Fig. 1. Gartner Hype Cycle that describes the visibility of a technology as it matures. Untested claims increase visibility to a peak before increased inquiry drives the technology toward retrenchment. Enlightened understanding eventually leads to stable productivity.
measurements over a bounded extent and the presence or absence of specific spectra. However, the most fundamental assumption was that the processed imagery was meant for human use. In contrast, at the time computational imaging was introduced, machine consumption of imagery was possible. Under this assumption, designers are no longer constrained by conventional notions of imaging, such as point-spread functions. Instead, they can design measurements that emphasize the information in which one is interested. Consequently, if one exploits redundancy in the scene of interest, it is possible to reduce the number of measurements required to produce an image with a certain signal-to-noise ratio. As straight forward as this explanation of CS appears, its simplicity is exactly what contributes to inflated expectations. Stated more positively, it gives rise to skepticism. As we know, the devil lies in the details, which is what we heard at the incubator. For the benefit of the community, we asked several of the incubator speakers to co-author papers that summarize their assessment of CS applied to electrooptic imaging, medical imaging, and radio frequency imaging. The papers are “Compressive sensing in the EO/IR,” by Michael Gehm and David Brady, “Pitfalls and possibilities of radio frequency compressive sensing,” by Nathan Goodman and Lee Potter, and “Compressive sensing in medical imaging,” by Christian Graff and Emil Sidky. Instead of summarizing each paper here, we note several overarching themes and provide some general conclusions. Each of the papers notes yet another trigger for CS, the papers by Candes et al. and Donoho [1,2], which provided a mathematical foundation for CS. Candes et al. and Donoho derived conditions under which CS can be proven to provide an advantage. Although this prompted a flurry of activity, the authors note that Candes et al. and Donoho were not the first to introduce techniques to maintain performance using a small set of measurements. Further, the theoretical CS2
APPLIED OPTICS / Vol. 54, No. 8 / 10 March 2015
limits on performance quickly encounter the practicalities of implementation, which each of the papers addresses. From a technical standpoint, compressive techniques are at a disadvantage when they are retrofitted into commercially available and affordable technology. It is therefore difficult to show any advantage of CS for visible imaging, where current approaches allow complete modularity. In visible imaging, one must find an application where forming an image is undesired, e.g., security or surveillance where maintaining privacy is an issue that increases cost. CS can be used to augment, not replace, current imaging systems, e.g., in navigation. Systems incorporating compressive and computational processing offer an advantage when they can be redesigned from the ground up. It is evident that improved performance can be obtained if one redesigns the fields in a magnetic resonance imager or the beam patterns in a computerized axial tomography scan, but this requires considerable investment from companies such as Siemens and GE. The portion of the spectrum in which CS has perhaps the greatest potential for application includes the x-ray and the millimeter wave, where no commercial systems exist, where the technology to form images does not exist, and where arrays of detectors do not exist. New sensor designs based on coded apertures, computational optics, and information processing have led to multiframe coded aperture spectral snapshot imaging, compressive x-ray imaging, and knowledge-enhanced exapixel photography, among others. While these prototypes are novel, they have not yet entered the product space; they may do so in the future as the operational benefits of such devices become clear. We believe that the next development in optical sensing will be adaptive sensing, where the next measurement depends upon past measurements, and optics, detectors, and information processing will function together to optimize task performance, whether it is imaging or estimating relevant information such as patterns of interests, motion, or behavior. We conclude that CS will most likely make an impact in new and emerging areas, rather than in retrofit designs. Three areas that seem most promising include applications where privacy is a major issue, three-dimensional imaging for medicine and security, and wide-area, long-range IR sensing. CS can aid these areas in two ways. For cases where the cost of measurement is high, multiplexing information with codes reduces the number of such measurements that need to be made and offers the potential to reduce the overall system cost. CS can also reduce the penalties associated with measurements, e.g., reduce exposure to harmful radiation in medical imaging, reduce the data acquisition bottleneck in high-throughput systems, for example, baggage scanners, and reduce concerns about loss of privacy in security systems. In the last case, CS
techniques make it possible to avoid forming images at all. Finally, for CS to have an impact, it must overcome organizational and opportunity costs, not just the technical ones. Design teams founded on exploiting physics in conjunction with computational techniques require a different organizational structure than conventional teams. It is not possible to develop CS systems using separate departments for optics, detectors, and signal processing. To realize any
payoff, the three disciplines must be integrated. Each must feed off of an understanding of the other. References 1. E. Candes, J. Romberg, and T. Tao, “Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information,” IEEE Trans. Inf. Theory 52, 489–509 (2006). 2. D. Donoho, “Compressed sensing,” IEEE Trans. Inf. Theory 52, 1289–1306 (2006).
10 March 2015 / Vol. 54, No. 8 / APPLIED OPTICS
CS3
