Lensless Neural Network Opens Up Machine Vision Possibilities

Researchers led by Hongwei Chen of the Beijing National Research Center for Information Science and Technology and Tsinghua University have developed a lensless optoelectronic neural network architecture for computer vision tasks. The system uses a passive mask inserted in the imaging light path to perform convolution operations in the optical field.

The system addresses the challenge of processing incoherent and broadband light signals in natural scenes, and it is designed to overcome the bandwidth bottlenecks encountered by electrical convolutional neural networks while at the same time avoiding problems encountered by optical neural networks (ONNs).

ONNs require a coherent laser as the light source for computation, and there is a need for an alternative for ONNs to work in combination with a mature machine vision systems in natural light scenes. This has led to interest in the development of optoelectronic hybrid neural networks in which the front end is optical and the back end is electrical. These lens-based systems increase the difficulty of use in edge devices, such as autonomous vehicles.

Schematic of the optical mask replacing the convolutional layer of the network. Courtesy of Tsinghua University.

Compared to the hardware architecture in conventional machine vision systems, the researchers proposed an optical mask positioned close to the image sensor to replace the lenses. According to geometrical optics theory, that light propagates in a straight line, the scenes can be regarded as sets of point light sources, and the optical signal is spatially modulated by the mask to realize the convolution operation of shift and superposition on the image sensor.

To perform object classification tasks like handwritten digit recognition, the team built a lightweight network for real-time recognition to verify the performance of the optical convolution in the architecture. While using a single convolution kernel, the recognition accuracy reached 93.47%. When the multichannel convolution operation is implemented by arranging multiple kernels in parallel on the mask, the classification accuracy was improved to 97.21%. Compared with traditional machine vision links, the systems was shown to save about 50% of energy consumption.

Further, by expanding the dimension of the optical mask, the image is convolved in the optical domain. The sensor then captured an image that is unrecognizable to the human eye, which enabled natural encryption of private information without computational consumption.

The team confirmed the performance of the optical encryption with a facial recognition task. Compared with the random maximum length sequence pattern, the recognition accuracy of the mask jointly optimized by an end-to-end network was improved by more than 6%.

The researchers envision the technology having application in autonomous driving, smart homes, and smart security.

The research was published in Light: Science & Applications (www.doi.org/10.1038/s41377-022-00809-5).