Researchers at UCLA have developed a focal plane array (FPA) capable of supporting real-time multispectral terahertz (THz) video in 3D. According to the team, led by UCLA professors Mona Jarrahi and Aydogan Ozcan, this is the first time a terahertz imaging system has been able to achieve real-time multispectral imaging with video capability while maintaining a high signal-to-noise ratio (SNR). The terahertz spectrum is desirable due to its ability to penetrate opaque materials and provide unique spectral information on various chemicals, but practical implementation of this spectral region has been limited due to the slow speeds, high costs, and general complexity of current imaging systems. The problem, the researchers said, is the lack of suitable FPA detectors — components that contain radiation detectors used by the imaging system. Current single pixel systems use raster scanning, which builds images pixel by pixel. Previous attempts to bypass raster scanning have employed nonlinear crystals, which also require bulky complicated setups and provide very low SNR levels. Another method used a time-varying spatial modulator to enable image reconstruction using acquired spatial information, but the technique proved to be slow, bulky, and costly. A focal-plane array (FPA) developed at UCLA captures multispectral terahertz (THz) images in real time, aided by a trained neural network to enhance the resolution. The system is video-capable and is being commercialized by UCLA spinoff Lookin Inc. Courtesy of Terahertz Electronics Laboratory/UCLA. In contrast, the THz-FPA developed by the UCLA team bypasses the need for raster scanning by directly providing spatial amplitude and phase distributions, as well as an object’s temporal and spectral data. Additionally, the resolution of the captured images is enhanced in real time by a machine learning-trained neural network. The THz-FPA consists of a 2D array of 283,500 plasmonic nanoantennas, engineered to detect broadband terahertz radiation with a high SNR when used in a terahertz time-domain spectroscopy (THz-TDS) system. To simplify data readout from the THz-FPA, the plasmonic nanoantennas are grouped into 7 × 9 clusters, and the collective response of all the nanoantenna clusters is electronically captured at each temporal point to simultaneously resolve their time-domain response. The system extracts the amplitude and phase responses of the THz-FPA outputs from the time-domain data over a 3-THz bandwidth. It captures the THz-FPA outputs at each temporal point in 164 μs using an electronic readout, enabling time-domain THz video capture at 16 fps. The researchers demonstrated the multispectral nature of the data captured by the plasmonic nanoantennas by using it to image different objects, including super-resolved etched patterns in a silicon substrate and defects in battery electrodes. By eliminating the need for raster scanning and spatial terahertz modulation, the THz-FPA provides a more than 1000-fold increase in imaging speed compared with current state-of-the-art terahertz imagers. The team used a deep learning-trained convolutional neural network to enhance the resolution of the captured images in real time and achieve pixel superresolution (PSR). The PSR-enhanced THz-FPA increases the spatial resolution and the effective number of pixels in the final image reconstructions, compared with the physical pixel size of the FPA. The deep learning-driven terahertz imaging system uses the amplitude and phase information of the spectral components detected in an imaged object to enhance the spatial resolution by four-fold in each lateral direction, increasing the space-bandwidth product of the FPA by 16-fold, with more than 1 kilopixel in each reconstructed image. In addition, the THz-FPA can quantify subwavelength material thickness variations as small as 10 µm, with a mean thickness accuracy of ~1 µm. To demonstrate the significance of the multispectral operation of the THz-FPA, the team compared the performance of its PSR framework with multispectral input data against the use of single-frequency input data. The researchers trained six image reconstruction deep neural networks that used the amplitude and phase outputs of the THz-FPA at a single frequency of 0.50, 0.75, 1, 1.25, 1.60, and 2 THz. Although the single-frequency systems showed the images of the input objects to some extent, the reconstructed images exhibited severe artifacts and distortions compared with the much clearer multispectral PSR results of the THz-FPA. The high-speed acquisition of spatial, ultrafast temporal, spectral, amplitude, and phase information presents a variety of implementation opportunities, such as reconstructing 3D images of multilayered objects and chemical identification. Additionally, the system could be integrated with diffractive optical networks for feature detection and object classification. Potential applications include nondestructive biomedical imaging, security screening, quality control of pharmaceutical, industrial, and agricultural products, and cultural heritage conservation. The technology is being commercialized by Lookin Inc., a startup spun off from Jarrahi’s research group. The research was published in Nature Photonics (www.doi.org/10.1038/s41566-023-01346-2).