Ptychographic microscope for three-dimensional imaging T. M. Godden,1,∗ R. Suman,1,2 M. J. Humphry,1 J. M. Rodenburg,1,3 and A. M. Maiden,3 1
3
Phase Focus Limited, The Electric works, Sheffield Digital Campus Sheffield, S1 2BJ , UK 2 Department of Biology, The University of York, Wentworth Way, York, YO10 5DD, UK Department of Electronic and Electrical Engineering, University of Sheffield, Mappin Street, Sheffield, S1 3JD ∗ [email protected]
http://www.phasefocus.com
Abstract: Ptychography is a coherent imaging technique that enables an image of a specimen to be generated from a set of diffraction patterns. One limitation of the technique is the assumption of a multiplicative interaction between the illuminating coherent beam and the specimen, which restricts ptychography to samples no thicker than a few tens of micrometers in the case of visible-light imaging at micron-scale resolution. By splitting a sample into axial sections, we demonstrated in recent work that this thickness restriction can be relaxed and whats-more, that coarse optical sectioning can be realized using a single ptychographic data set. Here we apply our technique to data collected from a modified optical microscope to realize a reduction in the optical sectioning depth to 2 μ m in the axial direction for samples up to 150 μ m thick. Furthermore, we increase the number of sections that are imaged from 5 in our previous work to 34 here. Our results compare well with sectioned images collected from a confocal microscope but have the added advantage of strong phase contrast, which removes the need for sample staining. © 2014 Optical Society of America OCIS codes: (100.5070) Phase retrieval; (180.6900) Three-dimensional microscopy; (100.3190) Inverse problems.
References and links 1. J. Marrison, L. Raty, P. Marriott, and P. O’Toole, “Ptychography a label free, high-contrast imaging technique for live cells using quantitative phase information,” Nature Scientific Reports 3, 1–7 (2013). 2. H. M. L. Faulkner and J. M. Rodenburg, “Movable aperture lensless transmission microscopy: a novel phase retrieval algorithm,” Phys. Rev. Lett. 93, 023903 (2004). 3. P. Thibault, M. Dierolf, A. Menzel, O. Bunk, C. David, and F. Pfeiffer, “High-resolution scanning x-ray diffraction microscopy,” Science 321, 379–382 (2008). 4. M. J. Humphry, B. Kraus, A. C. Hurst, A. M. Maiden, and J. M. Rodenburg, “Ptychographic electron microscopy using high-angle dark-field scattering for sub-nanometre resolution imaging,” Nature Comms. 3, 730 (2012). 5. M. Guizar-Sicairos and J. R. Fienup, “Phase retrieval with transverse translation diversity: a nonlinear optimization approach,” Opt. Express 16, 7264–7278 (2008). 6. A. M. Maiden and J.M. Rodenburg, “An improved ptychographical phase retrieval algorithm for diffractive imaging,” Ultramicroscopy 109, 1256–1262 (2009).
#209079 - $15.00 USD (C) 2014 OSA
Received 1 Apr 2014; revised 6 May 2014; accepted 7 May 2014; published 15 May 2014 19 May 2014 | Vol. 22, No. 10 | DOI:10.1364/OE.22.012513 | OPTICS EXPRESS 12513
7. A. M. Maiden, M. J. Humphry, F. Zhang, and J. M. Rodenburg, “Superresolution imaging via ptychography,” J. Opt. Soc. Am. A 28, 604–612 (2011). 8. P. Thibault and A. Menzel, “Reconstructing state mixtures from diffraction measurements,” Nature 494, 68–71 (2013). 9. P. Thibault and M. Guizar-Sicairos, “Maximum-likelihood refinement for coherent diffractive imaging,” New J. Phys 14, 063004 (2012). 10. H. Liu, Z. Xu, X. Zhang, Y. Wu, Z. Guo, and R. Tai, “Effects of missing low-frequency information on ptychographic and plane-wave coherent diffraction imaging,” Applied Optics 52, 2416–2427 (2013). 11. D. J. Batey, D. Claus, and J. M. Rodenburg, “Information multiplexing in ptychography,” Ultramicroscopy 138, 13–21 (2014). 12. A. M. Maiden, M. J. Humphry, M. C. Sarahan, B. Kraus, and J.M Rodenburg, “An annealing algorithm to correct positioning errors in ptychography,” Ultramicroscopy 120, 64–72 (2012). 13. M. Beckers, T. Senkbeil, T. Gorniak, K. Giewekemeyer, T. Salditt, and A. Rosenhahn, “Drift correction in ptychographic diffractive imaging,” Ultramicroscopy 126, 44–47 (2013). 14. F. Zhang, I. Peterson, J. Vila-Comamala, A. Diaz, F. Berenguer, R. Bean, B. Chen, A. Menzel, I. K. Robinson, and J. M. Rodenburg, “Translation position determination in ptychographic coherent diffractive imaging,” Opt. Express 21, 13592–13606 (2013). 15. A. Tripathi, I. McNulty, and O. G. Shpyrko, “Ptychographic overlap constraint errors and the limits of their numerical recovery using conjugate gradient descent methods,” Opt. Express 22(2), 1452-1466 (2014). 16. P. Thibault, V. Elser, C. Jacobsen, D. Shapiro, and D. Sayre, “Reconstruction of a yeast cell from X-ray diffraction data,” Acta Crystallogr. Sec. A 62, 248–261 (2006). 17. A. M. Maiden, M. J. Humphry, and J. M. Rodenburg, “Ptychographic transmission microscopy in three dimensions using a multi-slice approach,” J. Opt. Soc. Am. A 29, 1606–1614, (2012). 18. A. Suzuki, S. Furutaku, K. Shimomura, K. Yamauchi, Y. Kohmura, T. Ishikawa, and Y. Takahashi, “Highresolution multislice x-ray ptychography of extended thick objects,” Phys. Rev. Lett. 112, 053903 (2014). 19. M. Dierolf, A. Menzel, P. Thibault, P. Schneider, C. M. Kewish, R. Wepf, O. Bunk, and F. Pfeiffer, “Ptychographic x-ray computed tomography at the nanoscale,” Nature 467, 436–440 (2010). 20. Y. Cotte, F. Toy, P. Jourdain, N. Pavillon, D. Boss, P. Magistretti, P. Marquet, and C. Depeursinge, “Marker-free phase nanoscopy,” Nature Photonics 7, 113–117 (2013). 21. J. M. Rodenburg and R. H. T. Bates, “The theory of super-resolution electron microscopy via Wigner-distribution deconvolution,” Phil. Trans. R. Soc. Lond. A 339, 521 (1992). 22. J. M. Cowley and A. F. Moodie, “The scattering of electrons by atoms and crystals. A new theoretical approach,” Acta Crystallogr. Sec. A 10, 609–619 (1957). 23. J. W. Goodman, Introduction to Fourier Optics 3rd edition (Roberts and Company Publishers, 2005) pp. 55–73 24. M. Dierolf, P. Thibault, A. Menzel, C. M. Kewish, K. Jefimovs, I. Schlichting, K. Von Konig, O. Bunk, and F. Pfeiffer, “Ptychographic coherent diffractive imaging of weakly scattering specimens,” New J. Phys. 12, 035017 (2010). 25. C. L. Wenzel, J. Marrison, J. Mattsson, J. Haseloff, and S. M. Bougourd, “Ectopic divisions in vascular and ground tissues of Arabidopsis Thaliana result in distinct leaf venation defects,” J. Exp. Bot. 63, 5351–5363 (2012).
1.
Introduction
Ptychography is a form of Coherent Diffractive Imaging (CDI) rapidly gaining in popularity thanks to a simple experimental procedure and robust associated image reconstruction algorithms. In the visible light regime it offers quantitative phase imaging with extremely low noise levels [1]; for X-rays and electrons it also removes the need for (relatively poor) imaging optics [2–4]. A ptychographic experiment involves translating a specimen through a grid of positions and recording at each position the diffraction pattern that results from the specimen’s interaction with a localized coherent illuminating beam. A spacing of the translation grid that is a fraction of the diameter of the beam ensures that a given region of the specimen is illuminated at several specimen positions, thereby introducing redundancy into the recorded data. Structuring the illuminating beam so that a given region of the specimen is illuminated in a different way at each position introduces diversity into the data. The redundancy within a ptychographic data set provides robustness to the image reconstruction process, whilst the inherent diversity of the data provides a particularly rich source of supplementary information [5–16]. A crucial early example of this richness came with the realization that both the specimen transmission function and the illuminating probe wavefront #209079 - $15.00 USD (C) 2014 OSA
Received 1 Apr 2014; revised 6 May 2014; accepted 7 May 2014; published 15 May 2014 19 May 2014 | Vol. 22, No. 10 | DOI:10.1364/OE.22.012513 | OPTICS EXPRESS 12514
could be reconstructed from the diffraction data [3, 5, 6]. This made ptychography practical and straightforward. Subsequently, a string of results have shown that within the ptychographic data set lies sufficient information to extract super resolution data [7] and the coherent modes of a partially coherent probe [8], to recover diffraction data lost to shot noise [9] or due to the presence of a beam-stop [10], to separate contributions from a number of probes of different wavelengths [11], to correct errors in the measurement of the specimen positions [5, 12–15] and to divide the reconstructed image of a thick sample into a number of axial sections [17,18]. This last point we will address further here, showing that as many as 34 sections through a thick specimen can be reconstructed using a single ptychographic data set. We will also show how a conventional microscope platform can be adapted to collect the necessary data and reduce the section separation to around 2 μ m for penetration depths of up to 150 μ m. Ptychography has been combined with rotational tomography to realize high contrast 3D imaging in the X-ray regime [19], but the success of this method relies on the applicability of the projection approximation, which allows back projection of ptychographically reconstructed images taken at a series of specimen rotation angles. In our method the propagation and scattering of the probe as it passes through a thick sample is taken into account, so that strongly scattering, thick samples can be imaged. Since the technique does not require staining or fluorescent makers, and the illumination intensity required is on the order of 10 nW μ m−2 , our work has important applications in the field of label-free live cell imaging [1, 20]. We also envisage it finding applications when combined with rotational tomography to image thick samples with coherent X-rays, and in the electron regime where the majority of samples scatter strongly. 2.
Description of the method
Two-dimensional ptychography models the interaction of a transmissive specimen with a localized probe wavefront as a multiplication. This means that at each of the j = 1 . . . J positions within a specimen translation grid, the intensity, I j (u), of the wavefront incident upon a detector placed a distance d downstream of the specimen is described by: I j (u) ≈ |Fd [P(r − R j )O(r)]|2
(1)
Where O(r) represents the transmission function of the specimen, and P(r) the illuminating probe, with r = (x, y) a coordinate in the two-dimensional plane of the object. Rj = (Rx, j , Ry, j ) is the jth position of the specimen within the translation grid, and Fd is a free-space propagator over the distance d. u = (u, v) is a coordinate in the plane of the detector. 2D ptychographic reconstruction algorithms solve the inverse problem of determining which specimen and probe, when fed into the forward model of equation 1, would result in the recorded diffraction intensities. However, the multiplicative approximation used in equation 1 is applicable only when the specimen is suitably thin [3,18,21]. As a rough guide, the supplement of [3] derives the following upper limit on the thickness of sample that can be accommodated within the multiplicative approximation: T
3
Phase Focus Limited, The Electric works, Sheffield Digital Campus Sheffield, S1 2BJ , UK 2 Department of Biology, The University of York, Wentworth Way, York, YO10 5DD, UK Department of Electronic and Electrical Engineering, University of Sheffield, Mappin Street, Sheffield, S1 3JD ∗ [email protected]
http://www.phasefocus.com
Abstract: Ptychography is a coherent imaging technique that enables an image of a specimen to be generated from a set of diffraction patterns. One limitation of the technique is the assumption of a multiplicative interaction between the illuminating coherent beam and the specimen, which restricts ptychography to samples no thicker than a few tens of micrometers in the case of visible-light imaging at micron-scale resolution. By splitting a sample into axial sections, we demonstrated in recent work that this thickness restriction can be relaxed and whats-more, that coarse optical sectioning can be realized using a single ptychographic data set. Here we apply our technique to data collected from a modified optical microscope to realize a reduction in the optical sectioning depth to 2 μ m in the axial direction for samples up to 150 μ m thick. Furthermore, we increase the number of sections that are imaged from 5 in our previous work to 34 here. Our results compare well with sectioned images collected from a confocal microscope but have the added advantage of strong phase contrast, which removes the need for sample staining. © 2014 Optical Society of America OCIS codes: (100.5070) Phase retrieval; (180.6900) Three-dimensional microscopy; (100.3190) Inverse problems.
References and links 1. J. Marrison, L. Raty, P. Marriott, and P. O’Toole, “Ptychography a label free, high-contrast imaging technique for live cells using quantitative phase information,” Nature Scientific Reports 3, 1–7 (2013). 2. H. M. L. Faulkner and J. M. Rodenburg, “Movable aperture lensless transmission microscopy: a novel phase retrieval algorithm,” Phys. Rev. Lett. 93, 023903 (2004). 3. P. Thibault, M. Dierolf, A. Menzel, O. Bunk, C. David, and F. Pfeiffer, “High-resolution scanning x-ray diffraction microscopy,” Science 321, 379–382 (2008). 4. M. J. Humphry, B. Kraus, A. C. Hurst, A. M. Maiden, and J. M. Rodenburg, “Ptychographic electron microscopy using high-angle dark-field scattering for sub-nanometre resolution imaging,” Nature Comms. 3, 730 (2012). 5. M. Guizar-Sicairos and J. R. Fienup, “Phase retrieval with transverse translation diversity: a nonlinear optimization approach,” Opt. Express 16, 7264–7278 (2008). 6. A. M. Maiden and J.M. Rodenburg, “An improved ptychographical phase retrieval algorithm for diffractive imaging,” Ultramicroscopy 109, 1256–1262 (2009).
#209079 - $15.00 USD (C) 2014 OSA
Received 1 Apr 2014; revised 6 May 2014; accepted 7 May 2014; published 15 May 2014 19 May 2014 | Vol. 22, No. 10 | DOI:10.1364/OE.22.012513 | OPTICS EXPRESS 12513
7. A. M. Maiden, M. J. Humphry, F. Zhang, and J. M. Rodenburg, “Superresolution imaging via ptychography,” J. Opt. Soc. Am. A 28, 604–612 (2011). 8. P. Thibault and A. Menzel, “Reconstructing state mixtures from diffraction measurements,” Nature 494, 68–71 (2013). 9. P. Thibault and M. Guizar-Sicairos, “Maximum-likelihood refinement for coherent diffractive imaging,” New J. Phys 14, 063004 (2012). 10. H. Liu, Z. Xu, X. Zhang, Y. Wu, Z. Guo, and R. Tai, “Effects of missing low-frequency information on ptychographic and plane-wave coherent diffraction imaging,” Applied Optics 52, 2416–2427 (2013). 11. D. J. Batey, D. Claus, and J. M. Rodenburg, “Information multiplexing in ptychography,” Ultramicroscopy 138, 13–21 (2014). 12. A. M. Maiden, M. J. Humphry, M. C. Sarahan, B. Kraus, and J.M Rodenburg, “An annealing algorithm to correct positioning errors in ptychography,” Ultramicroscopy 120, 64–72 (2012). 13. M. Beckers, T. Senkbeil, T. Gorniak, K. Giewekemeyer, T. Salditt, and A. Rosenhahn, “Drift correction in ptychographic diffractive imaging,” Ultramicroscopy 126, 44–47 (2013). 14. F. Zhang, I. Peterson, J. Vila-Comamala, A. Diaz, F. Berenguer, R. Bean, B. Chen, A. Menzel, I. K. Robinson, and J. M. Rodenburg, “Translation position determination in ptychographic coherent diffractive imaging,” Opt. Express 21, 13592–13606 (2013). 15. A. Tripathi, I. McNulty, and O. G. Shpyrko, “Ptychographic overlap constraint errors and the limits of their numerical recovery using conjugate gradient descent methods,” Opt. Express 22(2), 1452-1466 (2014). 16. P. Thibault, V. Elser, C. Jacobsen, D. Shapiro, and D. Sayre, “Reconstruction of a yeast cell from X-ray diffraction data,” Acta Crystallogr. Sec. A 62, 248–261 (2006). 17. A. M. Maiden, M. J. Humphry, and J. M. Rodenburg, “Ptychographic transmission microscopy in three dimensions using a multi-slice approach,” J. Opt. Soc. Am. A 29, 1606–1614, (2012). 18. A. Suzuki, S. Furutaku, K. Shimomura, K. Yamauchi, Y. Kohmura, T. Ishikawa, and Y. Takahashi, “Highresolution multislice x-ray ptychography of extended thick objects,” Phys. Rev. Lett. 112, 053903 (2014). 19. M. Dierolf, A. Menzel, P. Thibault, P. Schneider, C. M. Kewish, R. Wepf, O. Bunk, and F. Pfeiffer, “Ptychographic x-ray computed tomography at the nanoscale,” Nature 467, 436–440 (2010). 20. Y. Cotte, F. Toy, P. Jourdain, N. Pavillon, D. Boss, P. Magistretti, P. Marquet, and C. Depeursinge, “Marker-free phase nanoscopy,” Nature Photonics 7, 113–117 (2013). 21. J. M. Rodenburg and R. H. T. Bates, “The theory of super-resolution electron microscopy via Wigner-distribution deconvolution,” Phil. Trans. R. Soc. Lond. A 339, 521 (1992). 22. J. M. Cowley and A. F. Moodie, “The scattering of electrons by atoms and crystals. A new theoretical approach,” Acta Crystallogr. Sec. A 10, 609–619 (1957). 23. J. W. Goodman, Introduction to Fourier Optics 3rd edition (Roberts and Company Publishers, 2005) pp. 55–73 24. M. Dierolf, P. Thibault, A. Menzel, C. M. Kewish, K. Jefimovs, I. Schlichting, K. Von Konig, O. Bunk, and F. Pfeiffer, “Ptychographic coherent diffractive imaging of weakly scattering specimens,” New J. Phys. 12, 035017 (2010). 25. C. L. Wenzel, J. Marrison, J. Mattsson, J. Haseloff, and S. M. Bougourd, “Ectopic divisions in vascular and ground tissues of Arabidopsis Thaliana result in distinct leaf venation defects,” J. Exp. Bot. 63, 5351–5363 (2012).
1.
Introduction
Ptychography is a form of Coherent Diffractive Imaging (CDI) rapidly gaining in popularity thanks to a simple experimental procedure and robust associated image reconstruction algorithms. In the visible light regime it offers quantitative phase imaging with extremely low noise levels [1]; for X-rays and electrons it also removes the need for (relatively poor) imaging optics [2–4]. A ptychographic experiment involves translating a specimen through a grid of positions and recording at each position the diffraction pattern that results from the specimen’s interaction with a localized coherent illuminating beam. A spacing of the translation grid that is a fraction of the diameter of the beam ensures that a given region of the specimen is illuminated at several specimen positions, thereby introducing redundancy into the recorded data. Structuring the illuminating beam so that a given region of the specimen is illuminated in a different way at each position introduces diversity into the data. The redundancy within a ptychographic data set provides robustness to the image reconstruction process, whilst the inherent diversity of the data provides a particularly rich source of supplementary information [5–16]. A crucial early example of this richness came with the realization that both the specimen transmission function and the illuminating probe wavefront #209079 - $15.00 USD (C) 2014 OSA
Received 1 Apr 2014; revised 6 May 2014; accepted 7 May 2014; published 15 May 2014 19 May 2014 | Vol. 22, No. 10 | DOI:10.1364/OE.22.012513 | OPTICS EXPRESS 12514
could be reconstructed from the diffraction data [3, 5, 6]. This made ptychography practical and straightforward. Subsequently, a string of results have shown that within the ptychographic data set lies sufficient information to extract super resolution data [7] and the coherent modes of a partially coherent probe [8], to recover diffraction data lost to shot noise [9] or due to the presence of a beam-stop [10], to separate contributions from a number of probes of different wavelengths [11], to correct errors in the measurement of the specimen positions [5, 12–15] and to divide the reconstructed image of a thick sample into a number of axial sections [17,18]. This last point we will address further here, showing that as many as 34 sections through a thick specimen can be reconstructed using a single ptychographic data set. We will also show how a conventional microscope platform can be adapted to collect the necessary data and reduce the section separation to around 2 μ m for penetration depths of up to 150 μ m. Ptychography has been combined with rotational tomography to realize high contrast 3D imaging in the X-ray regime [19], but the success of this method relies on the applicability of the projection approximation, which allows back projection of ptychographically reconstructed images taken at a series of specimen rotation angles. In our method the propagation and scattering of the probe as it passes through a thick sample is taken into account, so that strongly scattering, thick samples can be imaged. Since the technique does not require staining or fluorescent makers, and the illumination intensity required is on the order of 10 nW μ m−2 , our work has important applications in the field of label-free live cell imaging [1, 20]. We also envisage it finding applications when combined with rotational tomography to image thick samples with coherent X-rays, and in the electron regime where the majority of samples scatter strongly. 2.
Description of the method
Two-dimensional ptychography models the interaction of a transmissive specimen with a localized probe wavefront as a multiplication. This means that at each of the j = 1 . . . J positions within a specimen translation grid, the intensity, I j (u), of the wavefront incident upon a detector placed a distance d downstream of the specimen is described by: I j (u) ≈ |Fd [P(r − R j )O(r)]|2
(1)
Where O(r) represents the transmission function of the specimen, and P(r) the illuminating probe, with r = (x, y) a coordinate in the two-dimensional plane of the object. Rj = (Rx, j , Ry, j ) is the jth position of the specimen within the translation grid, and Fd is a free-space propagator over the distance d. u = (u, v) is a coordinate in the plane of the detector. 2D ptychographic reconstruction algorithms solve the inverse problem of determining which specimen and probe, when fed into the forward model of equation 1, would result in the recorded diffraction intensities. However, the multiplicative approximation used in equation 1 is applicable only when the specimen is suitably thin [3,18,21]. As a rough guide, the supplement of [3] derives the following upper limit on the thickness of sample that can be accommodated within the multiplicative approximation: T
