Noninvasive, nondestructive, and precise multispectral imaging nudges medicine away from one-size-fits-all toward personalization and accessibility.
MARIE FREEBODY, CONTRIBUTING EDITOR
From airborne mapping to astronomical imaging to manufacturing inspection, multispectral imaging (MSI) has long been the go-to tool for extracting crucial details, often from distant views. Today, MSI is making promising inroads in biomedical applications, where its capacity for noninvasive, cost-effective imaging delivers customizable information in real time.
“Spectral sensing is extremely common. Chances are you have used spectral sensing either at the doctor’s office or hospital without even knowing it,” said Heather Lovelady, medical marketing manager at spectroscopy specialist Ocean Optics Inc., based in Largo, Fla. “Pulse oximeters — the little finger clips that measure oxygen levels in blood — are spectral sensors. From the simplest pulse ox to complex PCR (polymerase chain reaction) diagnostics, spectral sensing has become indispensable in medicine.”
Multiplex immunofluorescence for evaluation of the tumor immune microenvironment (TIME). Seven-color TSA (tyramide signal amplification) staining of breast cancer showing the original multispectral ‘mixed’ image (left) and the unmixed pseudocolor image (right) generated using Vectra 3.0 multispectral microscope and inForm analysis. Pseudocolor key (right): blue: DNA; yellow: keratin; green: CD3; orange: CD68; white: PD1; red: PDL1; magenta: CD8. Courtesy of Hidetoshi Mori/Borowsky Lab/UC Davis and the Center for Genomic Pathology.
As its name suggests, MSI involves a relatively limited number of spectral bands — typically between three and 10 — combining spectral information with spatial information to provide spatiovisual data. With the resulting manageable data volume, MSI permits a crucial advantage when dealing with living
tissue: real-time imaging.
“Because we are wired to process spatiovisual data, this becomes a rich way of conveying information that is quite intuitive,” Lovelady said. “From the user’s point of view, there are few barriers to adoption. Most of the challenge is in developing the right product to fit a particular application.”
Smart imaging of burns
With dentists and dermatologists already wise to the benefits of MSI, the technique is beginning to find its way into new areas of biomedicine. Emerging applications include wound care and management, digital pathology, neonatal care, and precise tumor boundary imaging.
Human tonsil stained with DAPI (4′,6-diamidino-2-phenylindole) and five spectrally distinct quantum-
dot-labeled primary antibodies against Ki67 (pseudocolored blue), CD3 (yellow), CD20 (red),
IgD-positive (green), and CD68 (cyan) (a). The stained specimen was imaged with a Nuance
multispectral camera and individual channels were unmixed with optimized spectral endmembers (b). The cortex and mantle zone (upper left corner, b) is visualized and indicated by the yellow and green cells, respectively. Sample courtesy of Dr. Stefania Pittaluga/National Cancer Institute. Spectral analysis courtesy of Dr. Richard Levenson/UC Davis Health.
By combining a fluorescence tumor image with traditional visible color imaging in real time, surgeons are able to identify tumor boundaries with greater precision. During surgery, MSI and sensing can also be used to check the health and perfusion of tissues prior to resectioning, which mitigates the need for additional surgeries later.
Medical device company Spectral MD Inc., based in Dallas, is currently partnering with Ocean Optics to create a smart wound-imaging system for burn victims in triage and other wound management scenarios. The system incorporates artificial intelligence (AI) to see deep within a burn wound, characterize the tissue damage, and provide intelligent insights into healing.
“There is a limited number of burn wound experts in the country, and having an AI-driven system to analyze and triage burn wounds gets patients the accurate care they need faster,” Lovelady said. “In the surgery suite, the burn surgeon uses this information to know what tissue will heal on its own and what has to be removed.”
Other imaging techniques, such as broadband NIR imaging, only provide one piece of the puzzle, whereas MSI gives you the NIR image plus images at any other wavelengths needed to make an informed decision.
According to Lovelady, “When imaging ICG (indocyanine green) fluorescence [a commonly used dye meant to provide additional information] during surgery, you can see it with an NIR camera, but you lose the visual-range color image. MSI allows for imaging both at the same time with a single camera.” Eliminating the guesswork with accurate analysis not only reduces scarring but also speeds recovery, contributing to an overall better healing experience for the patient.
“Even outside of the operating room,” she said, “sensing and imaging are hot topics for medical device cleanliness verification, pharmaceutical process control, and noninvasive testing.”
The need for speed
One of the key advantages of limited-spectral-band imaging systems is their potential to provide video-rate imaging capabilities — a capacity that other high-information-bearing systems, such as Raman and hyperspectral imaging (HSI), don’t typically share. While HSI offers greater flexibility than its multispectral imaging counterparts, it targets around 32 to 256 spectral bands, generating huge data “cubes” that place enormous demands on storage and computation.
Blood analysis without pain for neonates. Optical spectroscopy could be a faster and less painful method for testing bilirubin concentration in
newborns. Courtesy of Ocean Optics.
Today’s cameras can produce and transmit 5 gigapixels per second, comfortably supporting multispectral imaging. But as Matthias Sonder — advanced development leader of scanning at imaging specialist Teledyne DALSA in Waterloo, Canada — points out, producing hyperspectral data at typical line-scan or frame rates requires over 100× more bandwidth and hyperline/hypercube processing capability that simply does not exist, at any cost.
“Technology will have to continue advancing to either reduce or enable such data amounts,” Sonder said.
Lovelady concurs. “Where we used to have hyperspectral cameras with unmanageable amounts of data to sift through, we can now tailor MSI to a completely custom solution to make the data you want stand out amongst the noise,” she said. “As the field grows and the benefits of MSI become more widely recognized, there is no limit to the applications where it can be a game changer.”
Enhancing fluorescence imaging
MSI is one of the default methods used in fluorescence imaging, commonly applied in conjunction with rastering confocal microscopy and spectrally resolved avalanche photodiodes. The technique involves tagging cells of interest with fluorophores that excite and/or emit in distinct spectral bands. As the fluorescence microscope acquires the same image in several wavelength regions, rastering spatially (and in time) reveals subcellular details on cells.
Sonder highlights one of the key benefits of multispectral imaging, namely the ability to simultaneously separate different fluorophores in an investigation of in-cell interactions. “Typically, two to four different spectra may be emitted by a prepared sample under illumination at any given time,” he said. “Today, filter cubes in [lower-cost] microscopes are used to separate these spectra apart in time, via filter switching.”
While leading microscopy companies offer high-end systems with simultaneous spectral acquisition, high prices can often be prohibitive and restrict individual researchers or midsize companies from maintaining their own systems.
A demonstration of 5+ DAPI-plex imaging, unmixing, and display. The individual channels can be separately analyzed and spatial statistics computed as desired. Sample courtesy of Dr. Stefania Pittaluga/National Cancer Institute. Spectral analysis courtesy of Dr. Richard Levenson/UC Davis Health.
“This limits quick-access evaluations and makes experimental design complicated,” he said. “However, researchers see this technique (rastering, multispectral confocal microscopy) as the ‘gold standard’ and any new technology would need to measure itself against this.”
Characterizing the TME
For the last two decades, MSI has established itself as the standard in translational research efforts to characterize the unique area immediately around a tumor. Known as the tumor microenvironment (TME), scientists have found MSI to provide clear gains over the traditional single-color approach.
“We and others have found that tissue-based multispectral imaging is the ideal method for this evaluation, given the importance of not only the immune cell types but their location in and around tumors, and in many cases the proximity and contact of two or more different cell types,” said Alexander (Sandy) Borowsky, professor of pathology and laboratory medicine at the University of California, Davis Health.
Borowsky’s laboratory proposes that every cancer, at the time of diagnosis, should have an initial evaluation of the TME, which includes a wider array of checkpoint- and immune-suppressive mechanisms to guide initial therapy choices, followed by reevaluation of treatments. As professor Richard Levenson (Borowsky’s colleague in the same department) points out, MSI alleviates two major challenges associated with immunofluorescence (IF) assay development: reduction of the impact of autofluorescence and proper separation of fluorophores that have overlapping spectral signatures.
A euro banknote image, captured with a multispectral camera that features three different outputs: color, monochrome, and NIR. Light sources were white LEDs for color and monochrome, and 850-nm LEDs for NIR. Half of the architectural window image on the banknote was printed with IR security inks, which are invisible to the naked eye (shown in the color and monochrome images). With the light source at 850 nm, the NIR channel unveils the invisible features. Courtesy of Teledyne DALSA.
IF (and the related chromogenic technique, immunohistochemistry, or IHC) involves staining certain proteins in cells and tissues to identify specific antigens. By labeling the antibody with a fluorescent marker or enzyme, antigen/antibody interactions can be visualized by fluorescence or colorimetric detection. The technique is currently limited by the use of fluorophores that must be spectrally distinct. However, by applying MSI imaging and software post-staining, spectrally overlapping fluorophores can effectively be “unmixed.”
It does take some up-front work to customize the multispectral bands to be measured for each application. But Levenson has shown that once suitable spectral libraries are generated, linear unmixing can be accomplished. And this powerful technique has already significantly improved the ability of IF stains to be quantified with image analysis.
The Vectra Polaris, developed by the American life science and diagnostics giant PerkinElmer Inc., capitalizes on the approach, which, paired with the company’s inForm software, performs the unmixing of seven, and soon nine, colors. The upshot is scientists can explore biological tissue at multiple scales, from cell-to-cell interactions to the macroscopic tissue architecture.
Unfortunately, many barriers exist for clinical application and adoption of new technologies in pathology practice, and Borowsky concludes that remaining hurdles include incorporating MSI into training, clear applications to drive adoption and investment in the technology, and coevolution and adoption of MSI for routine everyday diagnostics.
But Ocean Optics’ Lovelady notes that
as the world moves away from one-size-fits-all medicine and toward personalization and accessibility, the need for novel technologies to analyze results will continue to develop. Multispectral imaging and sensing certainly ticks several boxes crucial to biomedical diagnosis, treatment, and therapy, and as costs come down and its potential benefits are recognized, it is expected to become an enabling technology for an increasing number of applications in the years to come.