The Hyperspectral Camera Looks Alzheimer’s in the Eyes

ELS PARTON, IMEC, AND ELENA BELETKAIA, EUROPEAN PHOTONICS INDUSTRY CONSORTIUM

Alzheimer’s disease is one of three so-called trillion-dollar diseases, along with cancer and diabetes — meaning that financial and societal costs are approaching $1 trillion worldwide. Diagnosis has historically been a complex task, requiring several specialists who perform many tests that cover an individual’s history, physical health, and cognitive performance. Following this initial analysis, more invasive and expensive tests are usually needed to make a definitive diagnosis. Recent technical developments may make this process far simpler — within the context of a simple eye examination — thanks to a combination of hyperspectral imaging, OCT, and machine learning.

Following the initial cognitive evaluation, the next round of tests typically focuses on the biochemical markers of Alzheimer’s, such as the amyloid beta and tau proteins. In this context, positron emission tomography (PET) scans are essential because they visualize protein plaques present in the brain (Figure 1) — the prime suspects in damaging and killing nerve cells in an Alzheimer’s patient. Before imaging technologies advanced, these plaques could only be detected by examining the brain during autopsy.

Figure 1. It is possible to detect amyloid beta protein accumulations in the brain of patients with Alzheimer’s disease, but the process can be invasive and expensive when using traditional methods. Courtesy of iStock.com/selvanegra.

Another traditional diagnostic technique uses cerebrospinal fluid, a clear fluid surrounding the brain and spinal cord. This fluid can be collected from the patient through a lumbar puncture and provides a snapshot of what is going on in the brain. The procedure is invasive and cumbersome for the patient, requiring specialized personnel to execute it, and the patient must lie flat for up to two hours to recover. Such tests are therefore not routinely performed, as they are reserved for a small group of patients who are faced with an uncertain clinical diagnosis or who are experiencing an unusual progression of symptoms. Even then, the tests remain inconclusive in about 10% to 20% of patients¹, leading doctors to take a case-by-case approach to determine which traditional test result will lead to a correct diagnosis. Cerebrospinal fluid analysis and amyloid beta PET scans could contribute to a diagnosis when a patient is in a stage of mild cognitive impairment, but these tests are not well suited for screening purposes among healthy patients or those in the advanced stages of Alzheimer’s.

Filling a need

Hyperspectral imaging is an emerging alternative solution for Alzheimer’s diagnostics and screening, specifically by examining the eye’s retina. The method has the potential to catch the disease at early stages, well before any symptoms are experienced².

Hyperspectral cameras were once complex pieces of lab equipment, with many components attached to large computers that processed the data. Thanks to modern system-on-a-chip technology, it is possible to fit a hyperspectral sensor on a chip and make a camera that is the size of a 10- × 10- × 10-cm cube. The compact design opens up tremendous possibilities for integrating these lab tools into medical instruments such as microscopes, clinical examination tools, endoscopes, and surgical instruments.

The working principle behind a hyperspectral camera is simple: It captures the light that is reflected by an object (tissue, skin, or a retina). Instead of measuring the amount of red, blue, and green light in the reflection as regular cameras do, a hyperspectral camera divides and analyzes the reflected light into tens or hundreds of different colors or narrow spectral bands. And these spectral bands can cover a larger spectrum than simply the visible region.

The research center imec in Leuven, Belgium, does R&D in adaptive optics-advanced chip technology and post-processing techniques and uses its expertise to produce compact hyperspectral cameras. Imec replaced the typical diffraction grating optics with more compact components that split the light into various wavelengths via a hyperspectral filter, and then attached the components and the filter directly on top of an existing commercial image sensor (Figure 2b)³.

The hyperspectral filter is a thin material layer and is based on the concept of Fabry-Pérot filters (Figure 2a). One such filter consists of two tightly spaced parallel mirror surfaces between which incoming light reflects and interferes. This interference, both constructive and destructive, allows some wavelengths to pass through while others are filtered out. By carefully controlling the distance between the reflecting surfaces, it is possible to make a filter that allows only a specific narrow wavelength band to pass. To cover a whole spectrum with small adjacent bands, for example, a staircase array of Fabry-Pérot filters could be designed, with each filter covering one wavelength band.

Figure 2. A Fabry-Pérot structure selecting only one wavelength out of a broad light spectrum (a). By covering image sensors with a special kind of thin material layer, the optics of a traditional hyperspectral camera system can be replaced so that the camera is much more compact, eliminating the need for regular calibration. CMOS technology enables the hyperspectral image sensors to be mass-produced at a low cost (b). Adapted with permission from Reference 3.

When fabricated with CMOS technology, these filters can be manufactured at high volume, up to millions of pieces. Hyperspectral filters can be fabricated on top of commercial image sensors because these filters can be patterned very accurately at pixel level, and the filter layout — the pattern in which the filters are deposited onto the sensor — can be easily tuned to fit an application’s needs (in terms of spectral bands or resolution, for example). Imec has developed two types of hyperspectral filters: line scan and snapshot. The company has also partnered with several camera builders, such as XIMEA, to integrate the chips into compact cameras.

In a line-scan hyperspectral sensor, the filters are arranged in a staircase-like structure over the entire width of the sensor (the pixel array) (Figure 3a). The object is scanned line by line to reconstruct the image.

In the filter layout of a snapshot sensor, the Fabry-Pérot filters are deposited on the image sensor in a mosaic configuration, instead of in a staircase format, with one filter placed onto each pixel (Figure 3b). Each filter is responsible for sensing only one narrow band of the spectrum for the whole scene. Such a mosaic layout is also used in traditional digital RGB cameras, in which the pixels are grouped by four, and each group of four pixels gets a separate color filter — a red and blue filter and two green filters.

Figure 3. The line-scan and snapshot filter concept designed by imec to be placed on top of a commercial image sensor in a staircase (a) or mosaic (b) configuration. Courtesy of imec.

Eye examination

The eye is often referred to as a “window into the brain,” and the images captured during an eye examination may provide insight into what’s happening in the brain of Alzheimer’s patients. Structural data from the eye can be linked to the brain and spinal cord. Researchers are considering the possibility that the typical Alzheimer’s protein accumulations, such as amyloid, could be detected in the eye even before they accumulate in the brain.

Lies De Groef — principal investigator at the Animal Physiology and Neurobiology division at KU Leuven, a university in Belgium — performs fundamental research on Alzheimer’s using transgenic mouse models. She performed mouse retina tissue studies using a microscope with an attached high-resolution XIMEA SNAPSCAN hyperspectral camera with an imec Linescan sensor inside.

The SNAPSCAN camera exploits the high spectral and spatial resolution of the hyperspectral line-scan sensor, without the need for any external scanning movement. Scanning functions are completed internally, using a miniaturized scanning stage. Full high-resolution hyperspectral images can be acquired in a few seconds, a speed that is suitable for fundamental research. De Groef was able to detect amyloid beta accumulations in the retinas of mice with Alzheimer’s disease that was already in a very early stage, several months before the first signs of neuronal dysfunction or inflammation became apparent⁴.

Ingeborg Stalmans and Sophie Lemmens from the Research Group Ophthalmology at KU Leuven also completed a study, examining 39 patients (some healthy and some diagnosed with Alzheimer’s) using a fundus camera (a specialized low-power microscope) equipped with a XIMEA Snapshot hyperspectral camera. This Snapshot camera houses a special mosaic hyperspectral snapshot sensor designed by imec engineers (Figure 4). They grouped the pixels of a standard CMOS 1088- × 2048-pixel image sensor into 4 × 4 arrays. On top of each group of 16 pixels, 16 narrowband filters were processed, effectively making the filters the size of one imager pixel. This dedicated design allows spatial and spectral data to be acquired from 16 individual spectral bands with a 10-nm bandwidth and wavelengths between 460 and 620 nm. The researchers detected spectral changes caused by the presence of amyloid beta accumulations on the retina (Figure 5). Such detection would not have been possible if fewer spectral bands were acquired.

Figure 4. The eye examination setup in the Alzheimer’s disease study conducted at the university hospital UZ Leuven using a fundus camera (a) with an integrated hyperspectral camera (b). Courtesy of UZ Leuven.

Figure 5. Four regions of interest are defined — superior 1 (S1), superior 2 (S2), inferior 1 (I1), and inferior 2 (I2) — relative to a line going through the center of the optic disc (OD). The areas used for the analysis (green) show the retinal blood vessels subtracted from the image (a). The mean spectra in the regions of interest (b). The mean (shaded), plus or minus the standard error of the mean. AD: Alzheimer’s disease. Courtesy of UZ Leuven.

The main reason for using the Snapshot camera for the patient research is its speed. With an exposure time of 0.2 ms, the camera acquires images in real time. This helps to avoid capturing the blinking of the eye, which would inevitably be captured if longer exposure times were used, leading to blurring and motion artifacts.

To complement the hyperspectral data and verify the presence of Alzheimer’s disease, optical coherence tomography (OCT) — a tool commonly used by ophthalmologists — was used to noninvasively produce high-resolution cross sections of the retina. The researchers measured the thinning of the retinal nerve fiber layer, which is a well-known neurodegeneration biomarker.

Machine learning

The Health unit of VITO, the Flemish Institute of Technological Research in Belgium, became involved early on in the project. In a proof-of-concept study, the institute added the essential ingredient of machine learning to the already-compiled mix of biological knowledge and expertise involving the hyperspectral and OCT imaging technologies. VITO developed a dedicated analysis pipeline to clean (remove unwanted information) and process the raw hyperspectral imaging data. It used the information to construct classification models based on linear discriminant analysis. The models were trained using a specialized algorithm. Two input configurations for the classifier were evaluated for each region of interest: one that comprised normalized hyperspectral data and one that combined normalized hyperspectral data and OCT data.

The resulting models showed that this method could discriminate between Alzheimer’s patients and healthy individuals with an accuracy of approximately 75% in a nested leave-one-out cross validation. The hyperspectral information present in the images was the primary driver for achieving this result. Classification accuracy improved by including OCT data⁵.

Future focus

Advancements in photonics and the development of on-chip technologies have made hyperspectral imaging significantly more accessible to researchers and clinicians investigating specific conditions. The technology holds great potential for early detection of Alzheimer’s disease through detection of amyloid beta changes in the retina. The 75% accuracy rate that was reached in the proof-of-concept study justifies further development of the technology.

Hyperspectral imaging is a noninvasive, low-cost, faster alternative to today’s diagnostic tests for Alzheimer’s that could therefore become more widely used. The Snapshot camera is a key enabler for this application because it allows spatial and spectral information to be obtained in one image, enabling real-time data acquisition at a speed that is fast enough to compensate for eye movements.

Before the method can be widely used and adopted, hyperspectral and multimodal retinal imaging needs to be further improved by focusing on optimizing the technical setup, by using a nonmydriatic camera (making use of the retina’s reflective properties to show detail), or by utilizing an improved hyperspectral sensor to improve spectral quality. Parameters will be added to further refine diagnostic accuracy. Additionally, with the expansion of parameters, testing has included biomarkers in other related proteinopathies (e.g., Parkinson’s disease and Lewy body dementia) and eye diseases (e.g., glaucoma and age-related macular degeneration) to enable proper diagnosis, though these results have not yet been fully analyzed.

Meet the authors

Els Parton, Ph.D., is a scientific editor at the R&D hub imec in Belgium. She works in the external communications department and has written numerous articles on a broad range of topics describing the scientific results of imec researchers and their collaboration projects with industry. She obtained her doctorate from the Faculty of Bioscience Engineering at the University of Leuven in 2001 and has since worked at imec; email: els.parton@imec.be.

Elena Beletkaia, Ph.D., is project leader at the European Photonics Industry Consortium (EPIC). She is responsible for sensing technologies and life-science applications. Beletkaia’s background includes a degree in biophysics from Lomonosov Moscow State University and a doctorate from Leiden University. She completed an extensive work on the biochemical and biophysical mechanisms of cancer metastasis and application of multiple microscopic and spectroscopic techniques for sensing and noninvasive imaging; email: elena.beletkaia@epic-assoc.com.

Acknowledgments

This research was carried out under the framework of the ADMIRE (Alzheimer’s Disease Detection Using Multimodal Imaging of the Retina) project and funded through the Mission Lucidity initiative (www.missionlucidity.com). The doctoral degree of Sophie Lemmens, who took a leading role in the project, was jointly funded by VITO and KU Leuven. Part of this research work has been co-funded in the context of the HERALD project that was granted by the ATTRACT consortium, which received funding from the European Union’s Horizon 2020 Research and Innovation Programme (2014-2020). Research is also funded through a Neurodegenerative Disease Research (JNPD) grant with the University of Melbourne and Umeå University, and through a joint doctoral program at KU Leuven and the University of Melbourne.

References

1. A. De Wilde et al. (2019). Discordant amyloid-β PET and CSF biomarkers and its clinical consequences. Alzheimer’s Res Ther, Vol. 11, No. 1, p. 78, www.doi.org/10.1186/s13195-019-0532-x.

2. S. Lemmens et al. (2020). Hyperspectral imaging and the retina: worth the wave? Transl Vis Sci Technol, Vol. 9, No. 9, p. 9, www.doi.org/10.1167/tvst.9.9.9.

3. A. Lambrechts et al. (2014). CMOS-compatible, integrated approach to hyper- and multispectral imaging. IEEE International Electron Devices Meeting, pp. 10.5.1-10.5.4, www.doi.org/10.1109/iedm.2014.7047025.

4. M. Vandenabeele et al. (2021). The AppNL-G-F mouse retina is a site for preclinical Alzheimer’s disease diagnosis and research. Acta Neuropathol Commun, Vol. 9, No. 6, www.doi.org/10.1186/s40478-020-01102-5.

5. S. Lemmens et al. (2020). Combination of snapshot hyperspectral retinal imaging and optical coherence tomography to identify Alzheimer’s disease patients. Alzheimers Res Ther, Vol. 12, No. 144, www.doi.org/10.1186/s13195-020-00715-1.