Deep Learning Enables Real-Time Imaging Around Corners

A team of U.S. researchers has created a laser-based system able to image around corners in real time. With further development, the system may allow self-driving cars to look around parked cars or busy intersections to see hazards or pedestrians. The technology could also be installed on satellites and spacecraft for tasks such as capturing images inside a cave on an asteroid.

The system uses a commercially available camera sensor and a powerful laser source. The laser beam bounces off a visible wall onto the hidden object and then back onto the wall, creating an interference pattern known as a speckle pattern, which encodes the shape of the hidden object.

Researchers used deep learning to create a new laser-based system that can image around corners in real time. The system may one day enable self-driving cars to 'look' around parked cars or busy intersections to see cars and even read license plates. Courtesy of Felix Heide/Princeton University.

Reconstructing the hidden object from the speckle pattern requires solving a challenging computational problem. Short exposure times are necessary for real-time imaging but produce too much noise for existing algorithms to work. To solve this problem, the researchers turned to deep learning.

“Compared to other approaches, our non-line-of-sight imaging system provides uniquely high resolutions and imaging speeds,” said research team leader Christopher Metzler from Stanford and Rice Universities. “These attributes enable applications that wouldn’t otherwise be possible, such as reading the license plate of a hidden car as it is driving, or reading a badge worn by someone walking on the other side of a corner.”

The system, constructed by Metzler and colleagues from Princeton, Southern Methodist, and Rice Universities, can distinguish submillimeter details of a hidden object from 1 m away. Designed to image small objects at very high resolutions, it can also be combined with other imaging systems that produce low-resolution room-size reconstructions.

“Non-line-of-sight imaging has important applications in medical imaging, navigation, robotics, and defense,” said co-author Felix Heide from Princeton University. “Our work takes a step toward enabling its use in a variety of such applications.”

The researchers tested the technique by reconstructing images of 1-cm-tall letters and numbers hidden behind a corner, using an imaging setup about 1 m from the wall. Using an exposure length of a quarter of a second, the approach was able to produce reconstructions with a resolution of 300 µm.

The research is part of DARPA’s Revolutionary Enhancement of Visibility by Exploiting Active Light-fields (REVEAL) program, which is developing a variety of techniques to image hidden objects around corners.

The research was published in Optica (www.doi.org/10.1364/OPTICA.374026).