Label-free imaging with high resolution makes a phase retrieval method adaptable in the laboratory and clinic.
ZIBANG ZHANG, JINAN UNIVERSITY, AND GUOAN ZHENG, UNIVERSITY OF CONNECTICUT
Traditional optical microscopes require constant adjustments to bring a sample into focus. To see a small feature with details, researchers must use a high-resolution objective lens with a reduced field of view. The trade-off between resolution and imaging area is a major inconvenience for life scientists and pathologists who rely on microscopy to analyze and diagnose disease, because prepared tissue samples have dimensions in the centimeter range1.
It turns out, however, that this inherent limitation can be overcome by using lensless equipment. Rather than using lenses to magnify the object image, a thin diffuser can be placed between the specimen and the image sensor in a lensless setting (Figure 1). The diffuser is then randomly scanned to various positions while the sensor acquires the images.
Figure 1. Rather than using lenses to magnify the object image, a thin diffuser can be used to build a lensless microscopy platform for label-free quantitative imaging with high resolution and a large field of view, opening up new applications in both the lab and the clinic. Courtesy of Zibang Zhang and Guoan Zheng.
The images captured during the scanning process contain the encoded complex object information that will later be used to recover a high-resolution, large-field-of-view image of the specimen. This new lensless approach is called near-field blind ptychographic modulation.
The concept behind the method
The principle of near-field blind ptychographic modulation2 is shown in Figure 2. A thin diffuser can be made by spin-coating polystyrene beads on a coverslip. The diffuser is scanned between the specimen and the image sensor for lightwave modulation. The resulting modulated intensity patterns are then captured by the image sensor, and these images are used to recover the specimen in the lensless microscope.
Figure 2. Near-field blind ptychographic diffuser modulation for lensless microscopy. A thin diffuser is placed between the specimen and the image sensor for lightwave modulation (a). By blindly
moving the diffuser to various xy positions, a sequence of images can be captured, and they will be used to recover the complex object with high resolution and a large field of view. The prototype device where the diffuser is randomly moved to various xy positions using a low-cost, do-it-yourself stage (b). Courtesy of Guoan Zheng.
At the heart of the object recovery process is a phase retrieval technique called ptychography3-5. Light detectors such as image sensors and photographic plates measure only intensity variations of the light that hits them. In the process of recording, they lose the phase information, which characterizes how much the light is delayed through propagation.
Ptychography, originally developed for electron imaging, is a phase retrieval approach that recovers the lost phase information from many distinct intensity measurements, wherein the specimen is moved to various xy positions. In a typical implementation, a spatially confined probe beam is used to illuminate the specimen, and the patterns created by the diffracted light are captured by an image sensor at the far field. These measurements are then iteratively reinforced while an initially random phase approximation is allowed to converge to a solution that matches all measurements.
Since it is challenging to make a high-resolution objective lens in the x-ray regime, ptychography is one of few lensless options available for x-ray microscopy. However, the broad implementation of optical ptychography has been hampered largely due to its slow speed, the precise mechanical scanning process, and the requirement of high-dynamic-range detection at the diffraction plane. To recover the entire complex image — such as a tissue sample — for viewing, ptychography often requires thousands of patterns to be recorded while scanning the sample to various positions.
The new lensless setup addresses these issues with two innovations: It brings the sample close to the image sensor with a magnification factor of one, and it modulates the exit wave of the specimen downstream via a thin diffuser. The unit magnification configuration allows a researcher to have the entire sensor area, 6.4 × 4.6 mm, as the imaging field of view. The downstream diffuser modulation process enables efficient ptychographic reconstruction with fewer acquisitions. It also allows for resolution improvement and imaging of 3D specimens regardless of sample thickness6.
In the prototype platform, the distance d between the specimen and the image sensor is ~1 mm, corresponding to a Fresnel number of ~50,000. The Fresnel number, defined as the imaging area divided by d · wavelength, characterizes how a lightwave travels over a distance after passing through an opening, such as a pinhole. The ultrahigh Fresnel number used in this lensless setup indicates that there is very little light diffraction from the object plane to the sensor plane.
The requirement of high-dynamic-range detection in conventional ptychographic experiments is no longer needed in this lensless design. Low levels of diffraction also mean that the motion of the diffuser can be directly tracked from the captured raw images, eliminating the need for a precise motion stage, which is critical for conventional ptychography.
The word “blind” in “near-field blind ptychographic modulation” has twofold implications: First, it implies the recovery of both the high-resolution complex object and the diffuser profile at the same time, similar to the recovery of both the probe beam and the object in blind ptychography. Second, it means no prior information of the positional shift of the diffuser is needed. The shift of the diffuser can be recovered from the captured raw images via cross-correlation analysis.
In the prototype platform shown in Figure 2(b), a low-cost, do-it-yourself stage is used to move the diffuser to various xy positions at a low speed. With a relatively short exposure time, the images can be captured with continuous diffuser motion. No synchronization is needed between the diffuser motion and the image acquisition.
Up-sampling recovery technique
Figure 3 shows the captured raw images and the reconstructions for a USAF (U.S. Air Force) resolution target and a transparent phase target. An up-sampling ptychographic recovery technique is used to bypass the resolution limit set by the pixel size of the image sensor. With
this method, one pixel from the image sensor is modeled as 3 × 3 subpixels, with a resolution improvement factor
of 3. The intensity summation of these 3 × 3 subpixels is enforced to equal the intensity signal of the corresponding large pixel from the captured images. With this subpixel up-sampling technique, it’s possible to resolve the 0.78-µm linewidth from the superresolution reconstruction in Figure 3(a2), although the pixel size is 1.67 µm in the employed image sensor. A more recent development further demonstrates the capability of resolving 0.55-µm linewidth using an image sensor with a 1.85-µm pixel size7.
Figure 3. Bypassing the pixel size limit using an up-sampling ptychographic recovery technique. The raw image and the recovered intensity image of a resolution target, with a 3× resolution
improvement compared to the pixel size limit (a). The raw image and the recovered quantitative phase image of the transparent phase target (b). Courtesy of Guoan Zheng.
The ultimate achievable resolution is determined by the illumination wavelength and the feature size of the diffuser profile. For a threefold resolution improvement, ~400 images are typically needed to recover both the high-resolution unknown diffuser and the object. If the diffuser is known from a precalibration experiment, the number of images needed can be reduced to 100 or fewer.
Labeling not required with QPI
Imaging of biological cells and tissues often relies on fluorescent labels, which offer high contrast with molecular specificity. The use of exogenous labeling agents, however, may alter the normal physiology of the biospecimens. Complementary to established fluorescence microscopy, label-free quantitative phase imaging provides an objective morphological measurement tool for biospecimens and is free of the variability introduced by contrast agents6.
Figure 3(b) shows the recovered quantitative phase using the lensless platform. The quantitative phase imaging capability of this lensless imaging technique offers a label-free solution for many microscopy applications, including diagnostic pathology, cell volume and mass measurement, and pathogen detection8.
Figure 4 demonstrates an application example where the phase image of a U87MG cell culture is recovered over the entire sensor surface.
Figure 4. The recovered quantitative phase image of a U87MG cell culture over a large imaging field of view. Courtesy of Guoan Zheng.
Digital refocusing for 3D samples
A limitation of conventional optical microscopy is the short depth of field of the high-resolution objective lens. The depth of field of a 20×, 0.4 NA objective, for example, is a few microns. Acquiring an image with such an objective requires placing the sample exactly at the focal position; otherwise, the final image won’t resolve any detailed information. Unfortunately, most practical samples are not thin 2D sections on 100% flat substrates. This produces a challenge for wide-field, high-resolution imaging: Users need to constantly adjust the stage to bring the sample into focus when moving across various lateral positions for creating a wide-field-of-view image, adding considerable time and effort to the process.
The lensless imaging platform tackles this challenge by performing digital refocusing after the object exit wavefront has been recovered. Different from illumination-based superresolution approaches1, the recovered object wavefront of this lensless setup only depends upon how the complex wavefront exits the sample, not how it enters it. Therefore, the sample thickness becomes irrelevant during reconstruction.
After recovery, the complex object wavefront can be digitally propagated to any plane perpendicular to the optical axis. Figure 5(a) shows the recovered object’s exit wavefront of a thick 3D potato sample. Figure 5(b) shows the recovered images after digitally propagating to three axial planes.
Figure 5. The recovered exit wavefront of a thick 3D potato sample (a). The recovered
object amplitude after digitally propagating to the planes of z = 620 µm, z = 650 µm, and
z = 685 µm (b1-b3). The cell walls are in focus in (b1), and the organelles are in focus in (b2)
and (b3). Adapted with permission from Reference 2.
The combination of the wide field of view, high resolution, and the long depth of field of the lensless imaging platform promises substantial gains for digital pathology. In a conventional digital pathology system, the tissue slide is mechanically scanned to various xy positions and the digital images are acquired using a high-resolution objective lens. It is challenging to perform autofocusing during the rapid scanning process. Many existing digital pathology systems create a focus map prior to the scanning process. For each point on the map, the system needs to scan the sample to various axial positions and acquire a z-stack in a time-consuming process. The best focus position is then inferred based on the image with the highest contrast.
The lensless imaging platform, however, allows users to digitally refocus to any axial plane after the data has been acquired. Its large imaging field of view and high resolution also facilitate the microscopic inspection of tissue slides. For example, Figure 6 shows the recovered image of a stained esophagus cancer slide using the lensless platform. Fine details can be resolved over a large field of view. For this image, only a green laser was used as light source. A color image can be obtained by turning on red, green, and blue light sources at the same time7,9. Since no optical lens is needed in the platform, imaging performance can be maintained over the entire field of view without aberration issues1. Multiple image sensors can also be used to increase the imaging throughput.
Figure 6. Wide-field-of-view, high-resolution imaging of a stained esophagus cancer slide. Adapted with permission from Reference 2.
This lensless imaging technique may potentially free pathologists from hours spent bent over the microscope, manually focusing the objective lens. Introducing lensless digital imaging into clinical workflows could allow this platform to be further integrated with other machine learning routines for computer-aided diagnostics.
For example, a blood film can be imaged using the lensless platform. The recovered intensity and phase information can be passed through a pretrained neural network for differential white blood cell counting. The new generation of pathologists trained on digital imaging systems, and the emergence of artificial intelligence in medical diagnosis promises further growth of lensless ptychography in coming years.
Ptychography has become an indispensable imaging modality in most x-ray synchrotron laboratories worldwide. Implementation of ptychography for high-throughput on-chip microscopy in the visible regime, currently in its early stage, will continue to grow and expand. The prototype platform discussed here is among the first steps in this direction.
Meet the authors
Zibang Zhang is an associate professor in the Department of Optoelectronic Engineering at Jinan University, China. His research focuses on computational optics, single-pixel imaging, and microscopy; email: [email protected].
Guoan Zheng is the United Technologies Associate Professor in the Department of Biomedical Engineering at the University of Connecticut. His research focuses on the development of novel imaging tools to tackle measurement problems in biology and medicine; email: [email protected].
References
1. G. Zheng et al. (2013). Wide-field, high-resolution Fourier ptychographic microscopy. Nat Photonics, Vol. 7, p. 739.
2. S. Jiang et al. (2020). Wide-field, high-resolution lensless on-chip microscopy via near-field blind ptychographic modulation. Lab Chip, Vol. 20, p. 1058.
3. H.M.L. Faulkner and J. Rodenburg (2004). Movable aperture lensless transmission microscopy: a novel phase retrieval algorithm. Phys Rev Lett, Vol. 93, p. 02390.
4. M. Stockmar et al. (2013). Near-field ptychography: phase retrieval for inline holography using a structured illumination. Sci Rep, Vol. 3, p. 1927.
5. F. Zhang and J. Rodenburg. (2010). Phase retrieval based on wave-front relay and modulation. Phys Rev B, Vol. 82, p. 121104.
6. Z. Bian et al. (2019). Ptychographic modulation engine: a low-cost DIY microscope add-on for coherent super-resolution imaging. J Phys D Appl Phys, Vol. 53, p. 014005.
7. P. Song et al. (2020). Super-resolved multispectral lensless microscopy via angle-
tilted, wavelength-multiplexed ptychographic modulation. Opt Lett, www.doi.org/10.1364/OL.394923.
8. Y. Park et al. (2018). Quantitative phase imaging in biomedicine. Nat Photonics,
Vol. 12, p. 578.
9. S. Dong et al. (2014). Spectral multiplexing and coherent-state decomposition in Fourier ptychographic imaging. Biomed Opt Express, Vol. 5, p. 1757.