Single-Pixel Camera Streams in Real Time at 100 fps

The speed of most single-pixel imaging (SPI) systems is hampered by the refresh rates of digital micromirror devices (DMDs) and the time-consuming iterations of compressed-sensing (CS)-based reconstruction. The data acquisition rate of most SPI systems is limited to below the megahertz bandwidths of typical photodetectors, due to the kilohertz pattern refresh rates of DMDs.

Consequently, the 2D imaging speed in most SPI systems is ultimately restricted to under 100 frames per second (fps).

Researchers at the Énergie Matériaux Télécommunications Research Centre of the Institut national de la recherche scientifique (INRS, Quebec) have developed an SPI system that overcomes these limitations to speed to stream real-time video at 100 fps, and up to 12,000 fps offline. The researchers’ Single-Pixel Imaging Accelerated via Swept Aggregate Patterns (SPI-ASAP) system could be used to provide insights into industrial processes, as well as for applications that rely on self-luminescent events, such as optical property characterization of nanomaterials or the monitoring of neural activities.

Previous efforts to overcome the speed limitations of SPI, such as the use of fast-moving physical encoding masks for light modulation, suffer from low reconfigurability and reduced accuracy. The researchers addressed these challenges by linking a DMD with laser scanning hardware to achieve a highly flexible prototype camera. Laser scanning allowed the researchers to use individual encoding masks as optically selected subregions of larger aggregate patterns that are displayed via the DMD. Pattern aggregation enhanced SPI data acquisition rates by more than two orders of magnitude above the limitations of DMD-only modulation, while retaining a high pixel count and flexible pattern display.

SPI-ASAP operates in real time and can be adapted for different scenes, frame resolutions, and frame rates. Courtesy of professor Jinyang Liang and researcher Patrick Kilcullen/Institut national de la recherche scientifique.

The researchers also developed a fast CS reconstruction algorithm that is tightly integrated with the pattern deployment architecture and fully compatible with parallel computing. As a result, SPI-ASAP would not be restricted by the iterative optimization approach used in conventional CS-based image reconstruction, according to the researchers.

SPI-ASAP showed the flexibility to operate at different spatial resolutions, different imaging speeds, and different modes. The researchers demonstrated the high-speed, real-time imaging capabilities of SPI by recording various dynamic scenes in both reflection and transmission modes across various frame sizes. In both the reflection and transmission modes, SPI-ASAP exhibited flexibility regarding frame size and frame rate, resilience to strong ambient light, and the capability to perform ultrahigh-speed imaging at up to 12 kfps.

Further, SPI-ASAP retains the practical advantages of DMDs, including flexible and reconfigurable pattern projection, while extending the rate of pattern deployment. It could have broad application, especially in the nonvisible spectrum — a range that, according to the researchers, currently lacks suitable imaging options.

With its ability to provide dynamic imaging under strong ambient light conditions, additional applications for SPI-ASAP could include the study of combustion phenomena, and SPI-ASAP combined with 3D profilometry could provide scene relighting on high-speed 3D objects. SPI-ASAP could also be used to induce patterned regions of high conductivity in semiconductor materials for high-speed 2D terahertz imaging.

The team has patented the technique used for SPI-ASAP and is seeking collaborators to work on commercialization of the camera.

The research was published in Nature Communications (www.doi.org/10.1038/s41467-022-35585-8).