Hyperspectral Imaging Characterizes Healthy and Diseased Tissues During Surgery

TEHZEEB GUNJA, OMNIVISION, AND AXEL KULCKE, DIASPECTIVE VISION

Upgrades in medical imaging technology have made possible the accurate diagnosis and successful treatment of ailments ranging from broken bones to cancer. Techniques such as endoscopy keep evolving to image at greater depth and with stronger resolution, and advancements in miniaturization, electronic sensing, and computing capabilities inform the decision-making of clinicians. Physicians have an ever-growing array of imaging modalities available to them — such as hyperspectral techniques — that enhance their ability to treat injury and disease with greater clarity and speed and with less patient discomfort.

Intraoperative diagnosis, interventional therapy, and minimally invasive surgery are possible because of advancements in endoscopy. The instrumentation and use of endoscopy have evolved over the decades as high-resolution imaging has become more adaptable. In modern medicine, the technique can do much more than visualize areas of the body that are easy to access through natural orifices. The range of applications in which endoscopy has proved useful include gastroenterology, airway management, gynecology, arthroscopy, and cardiology.

Extremely small image sensors using complementary metal oxide semiconductor (CMOS) technology have been embedded on the tips of endoscopes, allowing cameras to enter the body through ever smaller incisions¹. State-of-the-art software and hardware for image processing enable high-speed video that serves as the “eyes” of the surgeon as he or she works inside the body (Figure 1). Surgeons can then see where to cut in real time.

Figure 1. An endoscope and an image it provided. Courtesy of OMNIVISION.

Common imaging techniques

Diagnostic imaging has traditionally been grouped into four categories of tools: x-ray, including both single-image and computed tomography (CT) scans; magnetic resonance imaging (MRI); nuclear medicine with radioactive tracers, including positron emission tomography (PET); and ultrasound (Figure 2). These techniques are relatively noninvasive and are valuable tools for helping physicians decide on a course of treatment. However, the tools carry risks from exposure to ionizing radiation, and some patients experience allergic reactions to injected tracers. Also, most of these imaging techniques cannot visualize the area of interest intraoperatively. Physicians therefore use the images as more of a reference to guide treatment, not as a real-time aid.

Figure 2. An example image from each of the four traditional categories of diagnostic imaging: CT scan (a), MRI (b), PET (c), and ultrasound (d). Courtesy of OMNIVISION.

In this context, endoscopy is a wide-ranging technique with many advantages over traditional imaging technologies employed in conjunction with surgery. Robotic surgical instruments have advanced the uses of endoscopy even further. Available tools are much more flexible than a surgeon’s hand and can be directed precisely at the point of interest.

But despite the advancements in endoscopy, limitations still exist in the information it captures. The endoscope does not necessarily offer the surgeon a full 360º view of the area of interest. In most cases, imaging covers only the visible portion of the spectrum and, as a result, distinguishing between different types of tissue can be difficult.

Overall, the benefits of procedures guided by endoscopy outweigh the risks when compared to traditional surgery, which often requires large incisions. Patients with less invasive operations spend less time in surgery and recover faster. But risks remain even if endoscopy is used. Wounds can heal less quickly when oxygenation is cut off from remaining tissue. Recognizing underperfused tissue — where circulation is poor — requires a judgment call based on a color change in the tissue. Additionally, in an effort to avoid resecting healthy tissue, endoscopic surgery sometimes leaves behind some of the cancerous tissue not visible to the human eye. The only way to concretely confirm that the cancer is gone is through follow-up medical imaging after surgery.

Enhanced vision

Fortunately for clinicians and their patients, a new imaging solution could potentially transform endoscopy and expand its capabilities. Hyperspectral imaging utilized in state-of-the-art endoscopic cameras holds the promise of higher productivity of detailed images, enabling the thoroughness of surgical intervention along with lower risk of damage to a patient’s collateral structures. Physicians use endoscopes of various types to see clear images of tiny structures inside the human body. With this guidance, clinicians can accurately diagnose disease and can often treat it immediately using endoscopic surgical tools. But what if these tools could help doctors distinguish between nerves, veins, and muscle tissue? Or what if a doctor could see the level of perfusion and, along with it, the viability of the tissues’ healing capability?

From the viewpoint of a standard endoscopic camera, organ tissue looks red and has a distinct texture. But there is no way to specify embedded structures and blood circulation in the resulting images.

One way to visualize perfusion is by injecting the patient with indocyanine green (ICG). When the use of ICG is combined with the insertion of an endoscope, perfusion visualization is possible to a limited degree within a subject’s systems. The color agent allows the physician to better assess the location of the tissue that needs to be removed and which areas to avoid in the subsequent surgery. The timing of ICG injection is tricky, however, as fluorescence is present only for a limited time, making the method less effective for lengthy surgeries. At the same time, repeated color injections are likely to result in poor contrast because of the accumulation of residual tracer left in the body.

It is possible to imagine even greater enhancements to endoscopy as research and development continue. What if tumors showed up as distinct from healthy tissue in images, so that oncologists could literally see cancer and remove it on the spot? The creation of a such a clear picture is not yet possible in medical procedures, but it is also not science fiction.

What is spectral imaging?

Medical spectral imaging, an umbrella term for hyperspectral imaging, is a new application of an imaging technique that has existed for decades in other industries. The technology combines absorption spectroscopy and digital imaging. It is applicable for use in a wide range of fields, including mining, agriculture, military combat, and medicine.

Within the field of view, hyperspectral imaging scans multiple wavelengths of light over a range of 400 to 1000 nm, thus extending from visible light into the near-infrared (NIR) region of the electromagnetic spectrum. The longer the wavelength, the deeper into the sample the light can penetrate.

Each wavelength of light will interact differently with the particular material that the light is directed toward, depending on the material’s chemical composition. The amounts of oxygen, water, or other specific molecules in the sample affect the degree to which light can be absorbed by the materials. Converting the absorbance value into a color creates a visual representation of the results (contrast) in image form.

A high-resolution camera processes data from millions of individual pixels in a sensor. The spectrometer captures the spectral data from each pixel, resulting in a cube of images that represents two spatial dimensions (x and y) plus wavelength (λ)². Imaging software enables the user to see physiological differences in the tissue in the camera’s field of view.

A new type of camera

The capabilities of medical spectral imaging are well suited to clinical applications. The various tissues in the human body have a unique spectral signature that traditional imaging technologies such as ultrasound cannot capture. But combining hyperspectral imaging with other imaging modalities expands their efficacy.

Diaspective Vision has taken advantage of this synergy in technical capability to incorporate medical spectral imaging into a new type of endoscopic camera, the MALYNA system. When applied to high-resolution medical imaging applications, this unique combination of technologies can expand the benefits of endoscopic procedures (Figure 3). The camera is applicable in laparoscopy, where it has seen clinical use, and it is also under development for robotic surgery and diagnostic endoscopy. The system augments 4K live video streaming along with physiological information to provide surgeons with visual confirmation that can inform their decision-making in real time.

Figure 3. The MALYNA system incorporates medical spectral imaging into a new type of endoscopic camera. Courtesy of Diaspective Vision.

The benefits of using this advanced imaging technology include higher productivity, reduced patient risk, and better post-surgery outcomes because the tissue affected by a particular condition or procedure is easily identifiable. Hyperspectral imaging capabilities enhance the performance of even the most advanced standalone endoscopic cameras. The success of medical spectral imaging requires leveraging the most advanced components available for high-end endoscopes. And it requires high-performance CMOS image sensors with 4K resolution and low latency for streaming video at 30 or 60 fps.

Most endoscopes incorporate sensors that operate in the visible spectrum, but these sensors are not sufficient for medical spectral imaging. NIR light-sensitive image sensors are necessary to take advantage of the expanding imaging power. The sensors must be medical grade, with high quantum efficiency covering the extended spectral range of the system.

While image sensors that detect NIR light have been commercially available for some time, they are usually associated with applications such as security that need to record images in low-light conditions. The same technology that allows security cameras to see intruders at night, however, can be applied to medical applications. Medical-grade sensors with OMNIVISION’s Nyxel technology, which increases sensitivity in the NIR region, meet the necessary requirements. Medical imaging cameras are now capable of distinguishing between tissues with different spectral signatures in the NIR range.

Using medical spectral imaging

Perfusion visualization is a key application for which medical spectral imaging offers significant benefits. The new camera technology supports ICG-based perfusion visualization, but it goes further, offering quantified perfusion data without the need for any color agents. The system can convert real-time measurements of oxygen content in the blood into a live video stream that shows the area of interest in full color.

This increased technological capacity enables physicians to examine perfusion at any time, regardless of the duration of the procedure. There is no need to delay the next step while waiting for ICG to dissipate in a person’s system.

When a surgeon is repairing a wound, real-time perfusion visualization allows him to optimize the repair to promote self-healing. Medical spectral imaging can also show the location of nerves, veins, and arteries that need to be avoided during surgery. As a result, surgeons avoid the risk of accidentally cutting a nerve and causing the patient permanent damage. NIR imaging goes deeper into tissue than imaging in the visible spectrum does. With NIR sensors, images can be captured from up to 6 mm under the surface, allowing physicians to see underlying structures before making any incisions, which improves the accuracy of endoscopic surgery by limiting incisions to an appropriate area.

Medical spectral imaging can quantify oxygenation and provides indexing of hemoglobin water and lipid content. Surgeons can use the data to identify various types of tissue as they move the endoscope (Figure 4)³.

Figure 4. The visualization modes of the MALYNA endoscopic camera system. Courtesy of Diaspective Vision.

Potential future applications

Each type of tissue and each organ in the human body has a unique spectral footprint. With advancements in deep learning software and image mapping, it should be possible to distinguish even more types of tissue, including nerves and head and neck tumors. Embedded features of specific tissues that were previously invisible to a clinician for diagnostic purposes will become visible and inform a growing number of procedures.

If the medical spectral imaging system can identify particular organs, robotic surgery will be an option for a greater variety of procedures. In the near future, it may even be possible to distinguish clusters of cancer cells using hyperspectral imaging technology in conjunction with deep learning methods. This capability could potentially make tumor removal more precise, protecting noncancerous areas. Scans after surgery could then verify whether the cancer was completely removed before the end of the procedure.

Medical spectral imaging, in combination with state-of-the-art endoscopic cameras, is set to become a standard diagnostic tool, alongside contemporary modalities such as x-ray, MRI, nuclear medicine, and ultrasound. Unlike the legacy tools, however, hyperspectral imaging can support both diagnosis and treatment of injuries and diseases, leading to better outcomes for the patient and quicker recovery times.

Meet the authors

Tehzeeb Gunja is director of medical marketing at OMNIVISION. He holds a Bachelor of Science degree in electronics from the University of Mumbai and a Master of Science degree in electrical engineering from Wayne State University; email: medical.marketing@ovt.com.

Axel Kulcke, Ph.D., founded Diaspective Vision GmbH in 2015 after studying physics and chemistry at Georg August University of Göttingen, followed by earning a doctorate at the same university and working in various positions in industries adopting spectral imaging technologies; email: office@diaspectivevision.com.

References

1. R. Yang (2021). Chip-on-tip technology expands endoscopy’s use in localized procedures. BioPhotonics, Vol. 28, No. 1, pp. 30-36, www.photonics.com/articles/chip-on-tip_technology_expands_endoscopys_use_i/p1/vo201/i1267/a66501.

2. P. Heney (2020). What is hyperspectral image analysis? R&D World, www.rdworldonline.com/what-is-hyperspectralimage-analysis.

3. B. Jansen-Winkeln et al. (2019). Determination of the transection margin during colorectal resection with hyperspectral imaging (HSI). Int J Colorectal Dis, Vol. 34, pp. 731-739, www.doi.org/10.1007/s00384-019-03250-0.