Deep Learning Enables Structured Light 3D Polarimetric Imaging

Researchers at Nanjing University of Science and Technology demonstrated a dual-frequency multiplexing fringe projection profilometry (FPP) technique that is enabled by deep learning. The researchers said their approach to FPP, which is a noncontact measurement technique for 3D imaging, enabled single-shot, unambiguous, high-precision, structured light 3D imaging.

As part of the demonstration, the researchers showed a deep neural network that was trained to directly recover the absolute phase from a unique fringe image that involves spatially multiplexed fringe patterns of different frequencies. They performed experiments on both static and dynamic scenes, and they showed that the approach is robust to object motion and could be used to obtain high-quality 3D reconstructions of isolated objects within a single fringe image.

The researchers’ technique specifically supports the ability to perform 3D shape reconstruction, filling a so-called gap that exists between 3D imaging and 2D sensing to deliver high-precision 3D image reconstruction using only one pattern as part of a structured light 3D imaging system. In FPP, a projector projects a series of fringe patterns onto a target, and the camera captures images that are modulated and deformed by the object. From the captured fringe patterns, phase information of the object can be algorithmically extracted via Fourier transform and phase shifting methods, for example.

Hamamatsu Corp. - Mid-Infrared LED 11/24 MR

FPP is widely used in optical metrology due to its noncontact, high-resolution, high-speed, and full-field measurement capabilities. These features are advantageous in intelligent manufacturing, reverse engineering, industrial inspection, and heritage preservation.

The researchers constructed two parallel U-shaped networks with the same structure. One network used a dual-frequency composite fringe image as the network input, which is combined with the traditional phase-shifting physical model. This combination is used to predict the sine and cosine terms to calculate the wrapped phase map.

The researchers designed the other network to predict the fringe order information from the input dual-frequency composite fringe image.

A neural network trained on data sets was then used to “de-multiplex” high-resolution, spectrum-crosstalk-free phases from the composite fringe to reconstruct a high-accuracy absolute phase map that supports the imaging process.

Traditional multifrequency composite methods cannot guarantee single-frame high-accuracy 3D imaging, and the researchers said that the work addresses this bottleneck. Additionally, it opens opportunities for single-shot, instantaneous 3D shape measurement of discontinuous and/or mutually isolated objects in fast motion.

The team plans to explore more advanced network structures and integrate more suitable physical models into deep learning networks.

The work was supported by the National Natural Science Foundation of China and received additional funding.

The research was published in Opto-Electronic Advances (www.doi.org/10.29026/oea.2022.210021).

Published: June 2022

Glossary

machine vision: Machine vision, also known as computer vision or computer sight, refers to the technology that enables machines, typically computers, to interpret and understand visual information from the world, much like the human visual system. It involves the development and application of algorithms and systems that allow machines to acquire, process, analyze, and make decisions based on visual data. Key aspects of machine vision include: Image acquisition: Machine vision systems use various...
profilometry: Measurement of surface roughness or quality through the use of a diamond-pointed stylus connected to a coil in an electric field. As the stylus is traced across the surface, a current is created that corresponds to the roughness.
metrology: Metrology is the science and practice of measurement. It encompasses the theoretical and practical aspects of measurement, including the development of measurement standards, techniques, and instruments, as well as the application of measurement principles in various fields. The primary objectives of metrology are to ensure accuracy, reliability, and consistency in measurements and to establish traceability to recognized standards. Metrology plays a crucial role in science, industry,...
neural network: A computing paradigm that attempts to process information in a manner similar to that of the brain; it differs from artificial intelligence in that it relies not on pre-programming but on the acquisition and evolution of interconnections between nodes. These computational models have shown extensive usage in applications that involve pattern recognition as well as machine learning as the interconnections between nodes continue to compute updated values from previous inputs.
deep learning: Deep learning is a subset of machine learning that involves the use of artificial neural networks to model and solve complex problems. The term "deep" in deep learning refers to the use of deep neural networks, which are neural networks with multiple layers (deep architectures). These networks, often called deep neural networks or deep neural architectures, have the ability to automatically learn hierarchical representations of data. Key concepts and components of deep learning include: ...
phase: In optics and photonics, "phase" refers to a property of electromagnetic waves, such as light, that describes the position of a wave at a given point in time within its oscillation cycle. More specifically, it indicates the position of a wave relative to a reference point, typically the starting point of a cycle. When discussing phase in optics, it's often described in terms of the phase difference between two waves or the phase of a single wave. The phase difference between two waves is the...
fourier transform: Any of the various methods of decomposing a signal into a set of coefficients of orthogonal waveforms (trigonometric functions).
structured light: The projection of a plane or grid pattern of light onto an object. It can be used for the determination of three-dimensional characteristics of the object from the observed deflections that result.
multiplexing: The combination of two or more signals for transmission along a single wire, path or carrier. In most optical communication systems this is referred to as wavelength division multiplexing, in which the combination of different signals for transmission are imbedded in multiple wavelengths over a single optical channel. The optical channel is a fiber optic cable or any other standard optical waveguide.

Browse Cameras & Imaging, Lasers, Optical Components, Test & Measurement, and more.