BioPhotonics spoke with Wouter Charle, program manager for hyperspectral imaging technology at imec, who leads off-the-shelf and evaluation system activities. After starting his career in 3D machine vision, Charle joined imec in 2014 to help grow the hyperspectral imaging business. He discusses how placing hyperspectral filters directly on a normal image sensor developed by imec allows for the technology’s integration into a variety of biomedical imaging systems.

Is the primary benefit of hyperspectral imaging to provide delineation of surgical margins in fields such as neuro-oncology and orthopedic surgery? Why have traditional technologies fallen short in this regard?

Traditional spectral setups are based on pushbroom, liquid crystal tunable filters (LCTFs), or filter wheel technologies, and all include a combination of optical components in a specific system configuration. Thus they come with constraints to integrate them into a larger imaging system. An approach to spectral imaging implemented by imec uses pattern filters at the pixel level with a (wafer-level) CMOS process. This sensor-level integration comes with less impact on the camera’s configuration and, thus, more freedom in terms of integration. There are several benefits to this approach: The process doesn’t alter the sensor electrically, making it directly compatible with existing cameras. Single-pixel patterning can make a Bayer-like pattern, with narrowband filters enabling true snapshot and video-rate spectral imaging. With no need for other system-level components, the setup can be miniaturized. Compared to other spectral imaging technologies, these are key features enabling use in applications such as providing the delineation of surgical margins, blood perfusion monitoring, or other biomarker detection in real time during surgery.

The feature imec co-wrote last year for BioPhotonics discussed a hyperspectral sensor on a chip as the centerpiece of a camera that could be used for diagnostics. Is this also the centerpiece of the hyperspectral technology set to be employed in surgery, and are there additional design details being developed?

Last year, we published results obtained with our snapshot camera, integrated with a fundus microscope (a standard tool for ophthalmologists), that showed that the camera’s spectral data could be used to detect the accumulation of proteins such as amyloid in the retina of Alzheimer’s patients. This is an important step toward an accessible, noninvasive early Alzheimer screening tool. The same technology is now used for surgical applications, and the same features stand out: the sensor’s snapshot and video-rate capability in combination with the miniaturization. The form factor is, of course, different when used in surgery, as the goal is to integrate the technology into exoscopes and endoscopes, as well as laparoscopes, for assisted and robotic surgery.

Our technology’s flexible “tuning” for different use cases is possible because we can pattern filters on individual image sensor pixels. We fabricate Bayer-like patterns with narrowband filters to make video-rate spectral sensors sensitive to blue wavelengths that can detect the protein beta-amyloid (in the case of Alzheimer’s), or to green or near-infrared wavelengths that can detect water, lipids, or other biomarkers in tissues (in the case of intraoperative use). At Photonics West, we will announce and demonstrate a specific device for the development of applications in surgical imaging.

Have sterility and ease of integration into the clinical workflow been the impediments to hyperspectral imaging’s application in mainstream surgery? How are these being overcome?

Because we do not use system-level spectral filtering, but filters on the sensor itself, our hyperspectral sensor functions like any other imaging sensor. This makes it relatively easy to integrate into existing instruments that currently use RGB or panchromatic sensors. Thus, without major changes to configuration of existing instruments, the hyperspectral technology can be put to use without disrupting the normal clinical workflow, and all requirements in terms of sterility and integration can be met. The spectral cameras that are currently being used on exoscopes and laparoscopes — in clinical trial phases by our industry partners — are compact enough to be integrated into a traditional surgical draping. The current off-the-shelf cameras weigh 36 g and are approximately 2 × 2 × 2 cm in size.