Advancements in engineered point spread function (E-PSF) technology, in the form of optical phase plates, will enable manufacturers to meet rising demands for ultraprecise object imaging.
LESLIE KIMERLING, WARREN COLOMB, and ANURAG AGRAWAL, DOUBLE HELIX OPTICS
As robotics and automation change the face of manufacturing, demands on industrial inspection have increased. Advancements in engineered point spread function (E-PSF) technology are now allowing manufacturers to incorporate high-resolution 3D imaging for improved object and feature inspection. These E-PSFs can be realized in the form of optical phase plates that can be incorporated into existing imaging systems.
Once integrated, imaging systems can perform close-range component inspection with improved depth of field and detail, allowing for the imaging and defect detection of smaller and smaller objects, down to the submicron scale.
An optical phase plate in its holder, with point spread function (PSF) designs that can be etched on the phase plate. Courtesy of Double Helix Optics.
Beyond inspection, 3D imaging holds promise for machine vision, object recognition, and navigation in high-growth applications such as drones, robotics, and haptics. It is now possible to integrate this novel optical technology into existing system designs with minimal impact on system footprints.
A look at limitations
While we live and move in a 3D world, the majority of imaging systems, historically, have been capable of
capturing only 2D information. Methods to obtain and use 3D information in
various settings, including manufacturing and robotics, have been studied for decades but remain a challenging problem to implement, particularly in unconstrained environments that may include variable lighting, specular and deforming scene surfaces, and occluded objects. These environments reduce spatial awareness, which in turn means systems struggle to perform tasks that are easy for humans, such as selecting components from a bin. Unconstrained environments also create challenges for the inspection of submicron-size objects. However, as additive manufacturing technologies enable the creation of more components with features at the micron level, advanced 3D inspection and metrology have become crucial.
The past decade has seen a mini-explosion in the number of depth-sensing devices, or 3D cameras. These systems either use a stereo-vision setup or more recent innovations — such as structured illumination, time of flight, and light field — and are very
useful for large-scale (centimeter to kilometer) measurement of the 3D shape and position of objects.
PSF of a standard optical system compared to the PSF of the double helix phase mask. Courtesy of Double Helix Optics.
These imaging modalities, however,
face various limitations when inspecting
close-proximity objects in the centimeter-to-millimeter range. Stereo-vision systems, for example, become “cross-eyed” at short working distances. Structured light methods are limited by the ratio of the spatial frequency of the projected light and the features of the object, as well as the need for nonobscured illumination, which often requires steep illumination angles. Time-of-flight methods are limited by the timing rates of the sensors. Additionally, light-field methods are constrained by limited resolution and the size of the lenslet arrays. Each of these techniques may be additionally constrained by hardware complexity, size, power consumption, or cost. Compounding these limitations is the increased expectation that manufacturers can accurately measure 3D features of minute structures.
In the face of these challenges, a new approach to 3D object capture is inevitable and will extend capabilities and enable improvements in both precision and depth resolution in areas such as 3D machine vision, gesture recognition, and robotics.
The essence of E-PSF technology
The essence of E-PSF technology
is a simple alteration of the optical
response of an imaging system, by
precisely matching the size and design of a mask to the optical system and imaging conditions. In the case of
microscopy, the phase mask is matched to the specifications of the
microscope objective and the depth
of focus requirements of the experiment.
The specially designed phase masks morph the optical response by introducing phase delays to certain portions of the wavefront, making it possible to change the shape of the PSF of in-focus and out-of-focus object points. The double-helix PSF (DH-PSF)1 design is one example, where the image of a single point is altered from the focused spot of light generated by the lens’ circular aperture (known as the Airy disc) to two well-separated spots.
The midpoint of these two well-separated spots corresponds to the lateral position of the object, and the angle between the two spots corresponds to the object’s axial position. Because the spots can stay in focus
while rotating up to 180°, extended
depth information can be captured with
high precision.
The data collected using the DH-PSF
consists of a number of these well-separated spots at different orientations
corresponding to the object’s lateral (X-Y) and axial (Z) positions. Creating a
sharp 3D image from this detailed data
set of object points is a complex but
solvable matter of image reconstruction. After processing, the result is a
sharp, 3D construct of the original
object.
Several types of E-PSFs have been designed for different applications based on the depth and precision requirements of the application and the signal-to-noise ratio (SNR) of the object. In addition to the DH-PSF, designs include single-helix PSF, tetrapod PSF2,
and multicolor PSF designs3.
The first commercial application of E-PSF technology is an upgrade to existing wide-field microscopes for 3D superresolution imaging and tracking.
The SPINDLE, which is a proprietary and registered product name, enables
imaging and tracking down to the level of the individual molecule or nano-
particle. The SPINDLE installs seamlessly between any wide-field microscope and electron multiplying CCD (EMCCD) or scientific CMOS (sCMOS) cameras using standard C-mounts. An interchangeable library of phase masks enables optimization of the PSF to suit the application of the user. Applications include imaging of cellular structures with a precision of 20 to 25 nm and up to 20 µm of depth range, without the need to alter the user’s existing imaging system setup.
From microscopic to macroscopic
Although the initial applications of
the E-PSF technology have been in
superresolution microscopy, the physics of E-PSF can be broadly applied
to any imaging system by scaling the size of the phase mask to the imaging
system. When applied to the field of machine vision, for example, E-PSF
overcomes many of the earlier mentioned challenges faced by other 3D-
imaging technologies4,5,6.
Double Helix Optics’ library of phase masks. Courtesy of Double Helix Optics.
Furthermore, E-PSF phase plates can be integrated into many existing 2D imaging systems, either by direct integration or by way of a passive optical relay between the camera lens and the sensor. If a 2D camera can resolve the region of inspection, then E-PSFs give that system depth perception.
The first commercial 3D machine vision system incorporating E-PSF technology is now under development. The implementation of this technology will simultaneously provide a brightness map (a 2D image) as well as distance information (a depth map) so each object feature within a scene is associated with its precise location in 3D space.
The E-PSF approach to machine
vision provides several advantages over
existing methods:
-
Incorporating a phase plate reshapes the focal point to enable depth capture with limited impact
on 2D system performance and with minimal shadowing.
- The depth resolution and the depth of field of an E-PSF 3D-imaging
system can be optimized by
designing the E-PSF to match the
2D lens in use.
Image of a credit card with embossed letters (a). Recovered depth map with depth (in µm) encoded in color (b). 3D view of the depth map overlaid with the brightness map (c). Courtesy of Double Helix Optics.
-
The E-PSF technology can be implemented as an add-on to existing 2D imaging systems or as
phase plates that can be incorporated into an existing 2D lens system. For 3D machine vision systems designed explicitly for E-PSF, the phase plate adds
virtually nothing to the volume
or weight. For systems employing OEM or after-market imaging
components, no second camera
or additional light sources are needed in most instances.
E-PSF technology opens up numerous possibilities for industrial inspection, materials science, and other
commercial applications by enabling conventional 2D imaging systems to simultaneously capture high-resolution
depth and intensity information. More-
over, the sensor is amenable to mass production at low cost, enabling applications in areas such as robotics,
3D scanners, advanced manufacturing, and human-machine interfaces. Imaging sensors are now widespread and inexpensive, as is computing power — already an integral part of most cameras — creating the opportunity
to add 3D capabilities at an additional
but limited cost.
Meet the authors
Leslie Kimerling is co-founder and
CEO of Double Helix Optics, a 3D-imaging company headquartered in Boulder, Colo. A serial entrepreneur, she has led multiple technology startups from launch through growth. She has a master’s degree in economics from Stanford University and an MBA from the University of California, Los Angeles (UCLA) Anderson School of Management; email: leslie@double
helixoptics.com.
Warren Colomb is an optical systems engineer at Double Helix Optics. He has a doctorate in applied physics
from the Colorado School of Mines; email: [email protected].
Anurag Agrawal is the principal optics scientist at Double Helix Optics. He has a doctorate in electrical engineering (computational optical imaging) from the University of Colorado, Boulder; email: [email protected].
Acknowledgments
Some of this material is based upon work supported by the NSF SBIR Grant IIP-1059286, Grant IIP 1534745, and Grant IIP 1353638. The imaging work was performed at the BioFrontiers
Institute Advanced Light Microscopy Core.
References
1. S.R.P. Pavani and R. Piestun (2008). High-efficiency rotating point spread functions. Opt Express, Vol. 16,
p. 3484.
2. Y. Shechtman et al. (2015). Precise three-dimensional scan-free multiple-particle tracking over large axial ranges with tetrapod point spread functions. Nano Lett, Vol. 15, Issue 6, pp. 4194–4199.
3. Y. Shechtman et al. (2016). Multi-
colour localization microscopy by point-spread-function engineering. Nat Photonics, Vol. 10, pp. 590-594.
4. A. Greengard et al. (2006). Depth from diffracted rotation. Opt Lett,
Vol. 31, p. 181.
5. S. Quirin and R. Piestun (2013). Depth estimation and image recovery using broadband, incoherent illumination with engineered point spread functions. Appl Opt, Vol. 52, pp. A367-376.
6. R. Berlich et al. (2016). Single shot three-dimensional imaging using an engineered point spread function. Opt Express, Vol. 24, p. 5946.