University of Wisconsin-Madison researchers have demonstrated non-line-of-sight (NLOS) imaging by using a method that applies the same math that is used to interpret images taken with conventional line-of-sight (LOS) imaging systems. The new method resolves the challenge of imaging a hidden scene by reformulating the NLOS imaging problem as a wave diffraction issue and then using well-known mathematical transforms from other imaging systems to interpret the lightwaves and reconstruct an image from a hidden scene. “What we did was express the problem using waves,” professor Andreas Velten said. A University of Wisconsin-Madison team is working on a project designed to create non-line-of-sight images using reflected laser light. Courtesy of Bryce Richter/UW-Madison. The researchers introduced a virtual wave field they called the phasor field. This Phasor Field Virtual Wave Imaging framework allowed the team to apply existing LOS wave imaging techniques to obtain images, videos, and other information from NLOS data. NLOS scenes were imaged from raw time-of-flight data by applying the mathematical operators that model wave propagation in a conventional LOS imaging system. To demonstrate their technique, the researchers created three imaging algorithms, modeled after three different LOS systems: A virtual photography camera that can capture 2D images of a scene without knowledge about the illumination source; a transient camera that can create videos of light transport through a scene; and a time-gated confocal camera that can create robust, 3D reconstructions of a scene. The team used its technique to create a video of light transport in a hidden scene, enabling visualization of light bouncing up to four or five times, which, according to Velden, could be the basis for cameras to see around more than one corner. UW graduate students (from left) Xiaochun Liu, Ji-Hyun Nam, and Toan Le work with assistant professor and principal investigator Andreas Velten in the computational optics lab inside the Medical Sciences Building at the University of Wisconsin-Madison. Courtesy of Bryce Richter/UW-Madison. The team showed that hidden scenes can be imaged despite the challenges of scene complexity, differences in reflector materials, scattered ambient light, and varying depths of field for the objects that make up a scene. Once perfected, the Wisconsin team’s technique could be used in a wide range of applications, from defense and disaster relief to manufacturing and medical imaging. In robotic surgery, for example, the technique developed by Velten’s team could provide a more complete picture of what’s going on around a sensitive procedure. The technology can be made inexpensively and can be compact, Velten said. The technique could be further improved if arrays of sensors could be used to capture the light reflected from a hidden scene. For its current experiments, the team used just one detector. The research was published in Nature (https://doi.org/10.1038/s41586-019-1461-3). This video shows the propagation of light through a hidden scene. Light bounces off a relay wall and is reflected off the surfaces of the objects in the scene. Courtesy of Andreas Velten.