Researchers from Dartmouth College, Beijing University of Technology, and the University of Birmingham have developed an image reconstruction approach that could contribute to better breast cancer detection. The approach uses a deep learning algorithm to overcome a major hurdle in multimodality imaging by allowing images to be recovered in real time The team’s algorithm, Z-Net, works with an imaging platform that combines optical spectral information with contrast-free MRI. “The near infrared spectral tomography (NIRST) and MRI imaging platform we developed has shown promise, but the time and effort involved in image reconstruction has prevented it from being translated into the day-to-day clinical workflow,” said Keith Paulsen, who led the researchers from Dartmouth College. “Thus, we designed a deep-learning algorithm that incorporates anatomical image data from MRI to guide NIRST image formation without requiring complex modeling of light propagation in tissue.” Researchers developed the Z-Net deep learning algorithm for real-time reconstruction of images that combine spectral and MRI data. This could allow better breast cancer screening and diagnosis. Courtesy of Keith Paulsen, Dartmouth College. Paulsen and his team members reported that the algorithm distinguishes malignant and benign tumors using MRI-guided NIRST imaging data from patient breast exams. “Z-Net could allow NIRST to become an efficient and effective add-on to noncontrast MRI for breast cancer screening and diagnosis because it allows MRI-guided NIRST images to be recovered in nearly real time,” Paulsen said. “It can also be readily adapted for use with other cancers and diseases for which multimodality imaging data are available.” Currently, dynamic contrast-enhanced (DCE) MRI is recognized as the most sensitive breast cancer detection method. This method requires intravenous injection of a contrast agent and has a substantial false positive rate. While noncontrast MRI-guided NIRST offers an alternative that doesn’t require an injection or ionizing radiation, reconstructing the combined images is a complicated, computationally intense, and time-consuming process, requiring light propagation models and time-consuming MRI image analysis. To make that process faster, the team used deep learning, which creates connections between pieces of information in a manner similar to the process of human learning. This allowed the researchers to train their algorithm to recognize patterns and complex relationships. “The Z-Net algorithm reduces the time needed to generate a new image to a few seconds,” said lead author Jinchao Feng of Beijing University of Technology. “Moreover, the machine learning network we developed can be trained with data generated by computer simulations rather than needing images from actual patient exams, which take a long time to collect and process into training information.” After training the algorithm, the researchers used simulated data to confirm that the quality of the reconstructed images was not degraded by eliminating diffuse light propagation modeling or by not segmenting MRI images. They then applied the algorithm prospectively to MRI-guided NIRST data collected from two breast imaging exams — one leading to a biopsy-confirmed cancer diagnosis, the other resulting in a benign abnormality. The algorithm generated images that could tell the difference between the malignant and benign cases. “In addition to showing the potential of our approach, the results also demonstrate that when in vivo data is insufficient or unavailable for training a deep learning algorithm, a large amount of simulation data may work,” said Shudong Jiang, a study co-author and pioneer in developing simultaneous MRI and optical breast imaging technology. The researchers are working to adapt the new image reconstruction technique to work with 3D data, and they plan to test it in a larger clinical trial in the near future. The research was published in Optica (www.doi.org/10.1364/OPTICA.446576).