To study brain functions like memory, neuroscientists track the electrical communications of neurons. These voltage changes can be subtle and happen on a millisecond timescale. A programmable image sensor from MIT could improve the ability to measure these signals, potentially allowing greater insight into how they affect brain function. In the new sensor, the sampling speed and phase of each pixel can be individually programmed, and pixels can be arranged to simultaneously sample at high speed and with a high signal-to-noise ratio (SNR). The gain in SNR could enable the detection of subtle voltage changes associated with spiking events and subthreshold activity in neurons, which standard CMOS sensors frequently miss. Biological processes can be captured with high resolution in high-speed, wide-field fluorescence microscopy, but tradeoffs exist between the pixels’ sampling speed and SNR. Fast sampling speeds lead to high readout noise, shortened exposure time, and fewer collected photons, which lower the SNR. Low frame rates with long pixel exposure can lead to signal aliasing. The key trade-offs between speed and SNR are linked to how a standard CMOS image sensor uniformly exposes and samples an array of pixels, causing all the pixels to turn on and off at the same time. The new sensor design, from MIT researcher Jie Zhang and professor Matt Wilson, enables the timing for each pixel to be controlled separately. To improve the signal that could be gathered by imaging an optical readout of the voltage of neurons, researchers at MIT invented an image sensor in which each pixel’s on-and-off timing and duration can be individually programmed. Each new pixel circuit uses only two additional transistors compared to a conventional CMOS pixel. Courtesy of Jie Zhang/Picower Institute. The programmable image sensor invented by Zhang allows neighboring pixels to each function according to their own timing and to function in complementary ways. Faster pixels can capture rapid changes and slower pixels can gather light, with fast and slow pixels working together so that no photon or electrical activity are overlooked by the sensor. The sensor’s control electronics are designed to occupy a minimum of the space that is available for the light-sensitive elements on a pixel. This ensures the sensor remains highly sensitive under low light conditions, Zhang said. By permitting flexible exposure at each pixel, the image sensor with pixel-wise programmable exposures, known as PE-CMOS, allows versatile pixel configurations that increase temporal resolution without sacrificing SNR. PE-CMOS is designed to enable pixel-wise programmability without compromising pixel sensitivity. The researchers demonstrated how the PE-CMOS sensor can improve imaging of voltage activity of mouse hippocampus neurons cultured in a dish. In high-speed voltage imaging experiments, the PE-CMOS image sensor increased the output SNR by about 2 to 3 fold, compared to a low-noise, standard, scientific CMOS image sensor. In the first set of experiments, the team imaged the fast dynamics of neural voltage. Each pixel on the standard CMOS chip had a 1.25-millisecond (ms) exposure time. On the PE-CMOS sensor, each pixel in neighboring groups of four had an exposure time of 5 ms. The researchers staggered the start times between each pixel on PE-CMOS, so that each pixel turned on and off at 1.25-ms intervals. Because the pixels on the PE-CMOS sensor were exposed for a longer period, they collected more light than the pixels on the conventional CMOS. At the same time, each pixel on the PE-CMOS sensor captured a new view every 1.25 ms, which resulted in fast temporal resolution. The SNR for the PE-CMOS chip was doubled, and this led to high temporal resolution at a fraction of the sampling rate, compared to conventional CMOS chips, Zhang said. The PE-CMOS chip detected neural spiking activities that the conventional sensor missed. When the researchers compared the performance of each sensor to the electrical readings made with a traditional patch clamp electrode, they found that the staggered measurements from the PE-CMOS sensor more closely matched those of the patch clamp. In a second set of experiments, the researchers demonstrated that the PE-CMOS chip could capture both the fast dynamics and the slow, subthreshold voltage variances that neurons can exhibit. The team varied the exposure durations of neighboring pixels on the PE-CMOS chip from 15.4 ms down to 1.9 ms. The fast pixels sampled every quick change, albeit faintly, while the slower pixels integrated enough light over time to track even slight, slow fluctuations. By integrating the data from each pixel, the PE-CMOS sensor was able to capture both fast spiking and slow, subthreshold changes. The experiments demonstrated how the programmability of the PE-CMOS sensor improves the ability to visualize neural voltage spikes — the signals neurons use to communicate with each other — and the faint fluctuations in voltage that occur between spiking events. “Measuring with single-spike resolution is really important as part of our research approach,” Wilson said. “Thinking about the encoding processes within the brain, single spikes and the timing of those spikes are important in understanding how the brain processes information.” The team’s goal is to conduct brain-wide, real-time measurements of activity in distinct types of neurons in freely moving animals while the animals learn how to navigate mazes. The ability to identify which neurons are active at any given moment will help scientists learn what types of neurons participate in memory processes and provide further clues to how the brain’s circuits work. To better track the voltage of neurons, a team of neuroscientists including MIT professor Ed Boyden, invented genetically encoded voltage indicators (GEVIs), which cause cells to glow as their voltage changes in real time. The development of GEVIs and image sensors like the PE-CMOS sensor make the detection of subtle changes in neuronal voltage feasible. “That’s the idea of everything we want to put together — large-scale voltage imaging of genetically tagged neurons in freely behaving animals,” Wilson said. Advancements in sensor technology will complement ongoing innovations in optics and denoising algorithms. Together, these advances will enhance fluorescence imaging technology to enable the tracking of neural activity at milisecond resolution across large numbers of neurons, over long experimental durations. “We are already working on the next iteration of chips with lower noise, higher pixel counts, time-resolution of multiple kHz, and small form factors for imaging in freely behaving animals,” Zhang said. The research was published in Nature Communications (www.doi.org/10.1038/s41467-024-48765-5).