This imaging technique, when matched with deep learning, helps physicians differentiate among colorectal tissues and quickly plot a course for surgery or treatment.
YIFENG ZENG, SHIQI XU, AND QUING ZHU, WASHINGTON UNIVERSITY IN ST. LOUIS
The origins of optical coherence tomography (OCT), a subsurface imaging technology based on low-
coherence interferometry, can be traced to the first decade of the 1800s, when Thomas Young’s double-slit interference experiment showed that light could move as a wave and interfere either constructively or destructively. Since 1991, researchers have explored the use of this phenomenon for medical imaging1, with the goal of saving lives in clinics and hospitals. Advancements in lasers, optical detectors, and fast electronics have boosted OCT’s imaging resolution, signal dynamic range, and real-time imaging capabilities2.
An OCT image of colon cancer.
Adapted with permission from Reference 3.
Light in an OCT system is guided into two arms — a sample arm and a reference arm. In the sample arm, photons are reflected and backscattered in various layers of the tissue. The combination of reflected light from both arms generates an interference signal, which is used to construct a depth-resolved image with a penetration depth in the range of 1 to
2 mm. Since OCT can provide high-
resolution depth-resolved images, it is suitable for visualizing subsurface
microstructures.
To date, OCT has successfully established diagnoses in ophthalmology, cardiology, and oncology. Most malignant tumors grow from epithelial tissue, where identification of a subsurface structural change is key for early detection. This imaging technology has demonstrated great potential in detection and diagnosis of ovarian, breast, prostate, skin, and lower and upper gastrointestinal cancers. However, due to OCT’s high resolution and fast scanning speed, large volumetric data sets are generated, which make real-time screening and diagnosis a challenging task.
Computer-aided diagnosis (CADx) has been widely used in clinics to help physicians interpret medical images and make diagnostic decisions. Progress made in machine learning, especially deep learning in computer vision, has brought artificial intelligence to CADx. Studies have demonstrated that a well-trained deep convolutional neural network (CNN), composed of multiple layers of linear and nonlinear operations that mimic human brain structures, can be used to analyze various medical images and provide physicians with useful information. Combining deep learning and OCT, researchers can discriminate cancerous tissues from normal tissues in real time with almost 100% accuracy3.
OCT in cancer screening
Colorectal cancer is the second most commonly diagnosed malignancy and the second leading cause of cancer mortality worldwide. According to the American Cancer Society, approximately 145,600 cases of colorectal cancer are diagnosed annually in the U.S4.
This cancer arises on the inner surface of the colon and then penetrates through the deeper layers. If left untreated, it will spread to other organs and thus become fatal. The current standard of care for endoluminal screening for colorectal cancer is by flexible endoscopy. Abnormal appearing areas are then biopsied for histologic analysis, and the results are available in hours or days.
However, there are two main challenges to this approach: First, endoscopy relies on visual identification of abnormal tissue to guide biopsy site selection, and early malignancies are often missed due to small or sessile lesions that are hard to detect. This oversight may cause the best surgery or treatment time period — in the beginning stages — to be missed, and the patient may die from the disease. Second, visual endoscopy can only detect changes on the surface, which limits its efficacy in evaluating chemotherapy and radiotherapy treatment response deeper within tissue. Overtreatment or unnecessary surgeries may be performed, which are costly and reduce the patient’s quality of life.
Research has shown that OCT, which can visualize subsurface microstructures with micrometer resolution, could reliably screen for and monitor colorectal cancer5. With the help of CNN, real-time pattern recognition can provide additional information to assist biopsy site selection and treatment response monitoring. Better and more sensitive CADx OCT tools to effectively evaluate the colorectal tissue can save lives and reduce suffering.
Fast-scanning technology
A pattern-recognition OCT (PR-OCT) technique was developed in a proof-of-concept demonstration of noninvasive, real-time examination of human colorectal specimens. A fiber-based swept-source OCT system was implemented with a center wavelength of 1310 nm. B-scan images could then be recorded and fed into CADx algorithms to capture structural signatures within the human colon mucosal layer. Figure 1 shows the OCT system used for this pilot patient study.
Figure 1. A pattern-recognition OCT (PR-OCT) working procedure. The interference signal from the imaged human specimen is recorded by the data acquisition computer. Real-time B-scan images and pattern recognition results can be presented. Courtesy of Yifeng Zeng and Shiqi Xu.
The OCT system consisted of a swept-source (HSL-2000, Santec Corp.), two fiber couplers, two circulators to deliver light to the sample and reference arms, a galvo-mirror system (GVS002, Thorlabs Inc.) for light scanning, a balanced detector (PDB450C, Thorlabs Inc.) to detect the interference pattern, and a data acquisition board.
Accurate dense object detector
RetinaNet is a CNN structural model that provides state-of-the-art detection of specific objects in a series of natural and medical images. In a single shot, Retina- Net draws a box around features that indicate the health of the colon. The researchers’ modified version of RetinaNet has three functional parts: one backbone CNN that generates feature maps and two subnetworks that predict the class and location of the bounding box. The backbone uses a U-shaped network, where multiresolution features are extracted by combining a residual network (ResNet) and a feature pyramid network, which together identify the features in various stages and scales.
With the feature maps given by the backbone, two subnetworks are then used to predict the location and category of the bounding boxes, as depicted in Figure 2. However, since the number of background objects (i.e., regions that do not belong to objects of diagnostic interest) is significantly higher than the rest, a focal loss is proposed to solve the imbalance and concentrate on the affected area. Moreover, a smooth L1 penalty — a combination of L1 and L2 loss — is used to effectively train the locations of the bounding boxes.
Figure 2. The modified RetinaNet. A feature pyramid network with a ResNet-18 backbone (left), and two subnetworks predicting the classifications and locations (right). Courtesy of Yifeng Zeng and Shiqi Xu.
Pilot study results
The first ex vivo patient study using PR-OCT was reported in Theranostics3. A total of 24 patients with an average age of 69 years were enrolled at the Siteman Cancer Center at Washington University in St. Louis. The patients had colorectal abnormalities that needed surgical resection. Within this data set, there were 20 tumor areas, 16 normal areas, 2 adenomatous polyp areas, 2 treated areas from complete responders with no viable tumor cells left, and 2 treated areas from nonresponders with residual tumors. In total, ~26,000 OCT images were recorded.
The researchers found that there was a distinct “teeth” pattern as a landmark in OCT images of the normal human colon. Because the crypt lumens in a normal mucosa layer increases optical transmission, the teeth pattern can be observed in normal colon OCT images. In contrast, when a neoplasm first grows in the mucosa layer, this pattern will be distorted or disappear.
Figure 3 exhibits forward-facing images of two 3D data sets, one from a normal area and the other from a malignant area. Additionally, cross-section B-scan images and comparative histology images are also shown. The teeth structure is apparent in the normal images and absent in the cancer images.
Figure 3. 3D-OCT images of normal and cancerous human colon specimens. Normal specimen
en face image constructed by axial summation (a). An xz cross section of a normal colon specimen (b).
A yz cross section image (c). Enlarged area of (a) (d). Representative en face histology (e). A photograph of a normal specimen (f). A cancerous specimen en face image constructed by axial summation (g).
An xz cross section of a cancerous colon specimen (h). A yz cross section image (i). Adapted with
permission from Reference 3.
Using the teeth structure as a promising landmark, researchers used supervised learning to accelerate diagnosis. Supervised learning for decision-making often requires a large training data set of tens of thousands of images, which is expensive and time-consuming to obtain in clinics. To address this problem, the researchers took advantage of the repeating dentate structure, and they carefully labeled dentate structures on 838 images of both cancer and normal colon areas from the first four patients they recruited. In this way, about 2000 labeled dentate structures were obtained3. They then used these labeled images to train a modified RetinaNet.
Rather than directly identifying the imaged area, which requires a much more extended training set, the researchers let the trained network find all the useful substructures and quantify the probability of the dentate structure’s correct identification in subsequent images. The clinical performance of the model was tested by the testing image cohort acquired from the remaining patients. Figure 4 shows the representative pattern-recognition results for six OCT images. The teeth pattern was successfully detected in normal colon tissues, and no such pattern was found in the cancerous cases. A sensitivity of 100% and specificity of 99.7% were achieved in identifying normal and cancerous tissue.
Figure 4. PR-OCT dentate pattern detection results. Normal colon images; green boxes show the predicted teeth patterns and the corresponding probabilities
(a-b). A colon cancer image (c). An adenomatous polyp image (d). A treated complete responder colon image (e). A treated nonresponder colon image (f). Adapted with permission from Reference 3.
The researchers extended their method and tested the algorithm on treated, nontreated, and polyp cases. For adenomatous polyps that had malignancy potential, no such pattern was found. Notably, in the complete responders to radiation and chemotherapy, the teeth pattern reappeared while the nonresponder continued to lack such a pattern.
PR-OCT in practice
The initial results on 24 patients are promising. Next, researchers at the university plan to develop a PR-OCT endoscope and optimize the user interface for patient studies, with the expectation that their pilot data will be validated by a larger patient pool.
When fully developed in a PR-OCT endoscope as an optical biopsy tool, this technique may optimize the screening of colorectal tissue. When surgeons or radiologists identify suspicious areas through a colonoscope, they could in theory turn on the PR-OCT system for tissue diagnosis and guidance of the biopsy procedure, if needed.
The potential for treatment monitoring will also be explored using the PR-OCT system. An accurate assessment of treatment response can optimize the clinical management of treated colorectal tissues and avoid unnecessary surgical resection, thereby lowering health care costs and improving the patient’s quality of life.
Meet the authors
Yifeng Zeng received a bachelor’s degree in engineering physics from Tsinghua University in Beijing in 2016. Since then, he has started working toward a doctoral degree in the Department of Biomedical Engineering at Washington University in St. Louis. His current research interests include deep learning applications in oncology, optical coherence tomography, diffuse optical tomography, and spatial frequency domain imaging; email: [email protected].
Shiqi Xu received a bachelor’s degree in electrical and computer engineering from University of Illinois at Urbana-Champaign and a master’s degree in electrical systems engineering from Washington University in St Louis. Currently, he is a doctoral student at Duke University. His research interests are noninvasive imaging deep inside the tissue and high-resolution wide-field gigapixel microscopies; email: [email protected].
Quing Zhu is professor of biomedical engineering at Washington University in St. Louis. She is a fellow of The Optical Society (OSA) and of SPIE, topical editor of Optics Letters (a publication of OSA), and an editorial board member of Photoacoustics and the Journal of Biomedical Optics (an SPIE publication). Zhu’s research focuses on ultrasound-guided NIR imaging for breast cancer diagnosis and cancer treatment assessment and prediction, co-registered photoacoustic/ultrasound tomography for ovarian cancer detection and diagnosis, as well as colorectal cancer treatment prediction, optical coherence tomography, and structured light imaging for cancer detection and diagnosis; email: [email protected].
References
1. D. Huang et al. (1991). Optical coherence tomography. Science, Vol. 254, Issue 5035, pp. 1178-1181.
2. M.J. Gora et al. (2017). Endoscopic optical coherence tomography: technologies and clinical applications. Biomed Opt Exp,
Vol. 8, Issue 5, pp. 2405-2444.
3. Y. Zeng et al. (2020). Real-time colorectal cancer diagnosis using PR-OCT with deep learning. Theranostics, Vol. 10, Issue 6,
p. 2587.
4. R.L. Siegel et al. (2019). Cancer statistics, 2019. CA: Cancer J Clin, Vol. 69, Issue 1, pp. 7-34.
5. Y. Zeng et al. (2019). The angular spectrum of the scattering coefficient map reveals subsurface colorectal cancer. Sci Rep,
Vol. 9, Article no. 2998.