Researchers from the Chinese Academy of Science developed an endoscope capable of acquiring 3D visible light and near-infrared fluorescence images at the same time. The optical design combined high-resolution 3D imaging capabilities with the ability of the mantis shrimp to simultaneously detect multiple wavelengths of light. The endoscope uses a stereoscopic design to achieve 3D imaging while a sensor, inspired by the compound eyes of mantis shrimp, not only detects multispectral information but also recognizes polarized light. The sensor features pixels with different spectral and polarization responses, detecting multiple parts of the electromagnetic spectrum. Different concentrations of the fluorescent reagent indocyanine green. Images: (a) a near-infrared fluorescence image, (b) the intensity distribution of the near-infrared fluorescence image, (c) color image of the samples under visible illumination, and (d) a fusion image combining the color image and the near-infrared fluorescence image. Courtesy of Chenyoung Shi, Chinese Academy of Sciences. For this to occur, both individual systems of the binocular optical design must have exactly the same parameters. “This places stringent requirements on the processing accuracy of optical components,” said Chenyoung Shi, a researcher at the Chinese Academy of Sciences. The researchers tested the endoscope, analyzing resolution, fluorescence imaging capability, and the ability to simultaneously image with near infrared and visible color information. The endoscope achieved resolution as high as seven line pairs per millimeter with visible light, which matches the resolution of endoscopes in use today, and four lines per millimeter under near-infrared illumination. The researchers used the device to obtain visible color and near-infrared fluorescence images of three concentrations of indocyanine green, a near-infrared fluorescence label used to label tumor tissues. The three samples are indistinguishable to the human eye. Using the multimodal endoscope, though, the samples were easily distinguishable. By imaging a complex toy with many crisscrossed parts, the researchers tested the endoscope’s 3D imaging performance. The researchers envision their device replacing existing endoscopes without clinicians needing to be trained on how to use the new instrument. “Existing fluorescence 3D endoscopes require surgeons to switch working modes during operation to see the fluorescence images,” Shi said. “Because our 3D endoscope can acquire visible and fluorescent 3D images simultaneously, it not only provides more visual information, but can also greatly shorten the operation time and reduce risks during surgery.” In robotic surgery, the enhanced visual information the endoscope delivered in testing could help surgeons distinguish various types of tissue. “Although today’s robotic surgical systems require the surgeon to be close by, robotic surgery based on this multimodal 3D endoscope might one day allow surgeons to remotely perform procedures in faraway locations,” Shi said. “This could help solve the problem of uneven distribution of medical resources and benefit people who live in areas with relatively poor medical conditions.” The researchers intend to further develop the system by incorporating more wavelengths, as well as the ability to discern polarization to provide further enhanced information. The research was published in Optics Express (www.doi.org/10.1364/OE.410424).