Deep Learning Delivers Optoacoustic Images in Real Time

High-quality medical imaging from multispectral optoacoustic tomography (MSOT) could be used to diagnose and evaluate a range of diseases, including breast cancer, Duchenne muscular dystrophy, inflammatory bowel disease, and many more. However, the length of time currently required for MSOT to process high-quality images makes it impractical in clinical settings.

The MSOT scanner, developed by researchers from the Bioengineering Center and the Computational Health Center at Helmholtz Munich and the Technical University of Munich, collects sound waves generated when light is absorbed by tissue. It uses reconstruction algorithms to translate the sound waves into images, which are displayed on a monitor.

Some of the algorithms used by MSOT can reconstruct images quickly enough for the scanner to display in real time, but with low image quality. The more complex algorithms can produce high-quality images but require prohibitively long processing times.

DeepMB provides a deep-learning framework for realizing high-quality optoacoustic imaging in real time, enabling multispectral optoacoustic tomography to be used in clinical settings. Courtesy of iThera Medical GmbH/Guillaume Zahnd.

Deep learning can enable fast reconstruction of optoacoustic images, but a lack of experimental ground-truth training data can lead to reduced image quality for in vivo scans. Model-based reconstruction can produce high-quality optoacoustic images, but it cannot be used for real-time imaging.

To provide high-quality, real-time optoacoustic imaging via MSOT, the researchers developed DeepMB, a deep-learning framework. DeepMB expresses model-based reconstruction with a deep neural network.

The researchers based their training strategy for DeepMB on optoacoustic signals synthesized from real-world images, paired with ground-truth optoacoustic images generated by model-based reconstruction. Using experimental data with DeepMB, they demonstrated accurate optoacoustic image reconstruction in 31 milliseconds per image.

The team further showed that DeepMB could reconstruct images approximately 1000 times faster than a state-of-the-art algorithm, with virtually no loss in image quality, based on qualitative and quantitative evaluations of a diverse data set of in vivo images.

According to the researchers, DeepMB can be used by clinicians to accurately reconstruct any scan acquired from a patient, regardless of the area of the body being scanned or the type of disease being evaluated.

Once deployed in the field, DeepMB will provide clinicians, for the first time, with direct access to optimal image quality from MSOT scans. Accurate, real-time image reconstruction with DeepMB will give medical personnel full access to the high-resolution, multispectral contrast of hand-held optoacoustic tomography, enabling this technology to be adopted into clinical routines.

The researchers believe that DeepMB represents a major step forward for optoacoustic imaging. High-quality, real-time optoacoustic imaging could have a positive impact on clinical studies and ultimately could help patients receive better care.

The core principles of DeepMB are adaptable and can be applied to other reconstruction methods in optoacoustic imaging, including other research efforts at Helmholtz Munich. The DeepMB framework could be applied to other imaging modalities, such as ultrasound, x-ray, or magnetic resonance imaging.

The research was led by professor Vasilis Ntziachristos and is jointly advanced by his research team at Helmholtz Munich and the Technical University of Munich and by his spinoff company, iThera Medical GmbH.

The research was published in Nature Machine Intelligence (www.doi.org/10.1038/s42256-023-00724-3).