Researchers at the University of Wisconsin-Madison, Washington University in St. Louis, and OmniVision described approaches to image sensor production that used integrated nanostructured components to improve multimodal imaging. The developments could allow autonomous vehicles to see around corners, biomedical imaging to detect abnormalities at different tissue depths, and telescopes to see through interstellar dust. It could spur further advancements in lensless cameras, augmented reality, and robotic vision. The schematics of a conventional sensor that can detect only light intensity (a) and a nanostructured multimodal sensor, which can detect various qualities of light through the light-matter interactions at subwavelength scale (b). Courtesy of Yurui Qu and Soongyu Yi. The researchers described an approach to enable images sensors to detect multiple-band spectra by fabricating an on-chip spectrometer. The team deposited photonic crystal filters made of silicon directly on top of the pixels to create complex interactions between incident light and the sensor. The pixels beneath the films recorded the distribution of light energy, from which light spectral information could be inferred. The device, less than a hundredth of a square inch in size, is programmable to meet various dynamic ranges, resolution levels, and almost any spectral regime from visible to infrared. The component built by the researchers detected angular information to measure depth and construct 3D shapes at subcellular scales. Directional hearing sensors found in animals such as geckos inspired the work. These animals’ heads are too small to determine where sound is coming from in the same way as humans and other animals. Rather, they use coupled eardrums to measure the direction of sound within a size that is orders of magnitude smaller than the corresponding acoustic wavelength. In a similar fashion, the scientists constructed pairs of silicon nanowires as resonators to support optical resonance. The optical energy stored in two resonators was demonstrated to be sensitive to the incident angle. The wire closest to the light sent the strongest current, and by comparing the strongest and weakest currents from both wires, the scientists determined the angle of the incoming lightwaves. The researchers said that millions of the type of nanowires used in the work could be placed on a 1-mm2 chip. The research was published in Applied Physics Letters (www.doi.org/10.1063/5.0090138).