Medical Imaging Learns from Computer Gaming

Dr. J. Ross Mitchell, Sonney Chan and Pierre Lemire, University of Calgary and Calgary Scientific Inc.

Medical imaging systems such as CT, MRI and ultrasound are becoming ubiquitous. As the technology becomes more commonplace, it also continues to mature. Significant imaging applications arise each year to aid in the diagnosis and monitoring of disease. This fuels demand, leading to greater deployment and use.

This cycle of growth has continued unabated over the past decade and has led to dramatic increases in the size and complexity of imaging data. For example, state-of-the-art CT scanners can produce images from 64 slices concurrently. However, 256-slice CT scanners are under development, with deployment planned in the next two years. Similarly, in MRI, new parallel arrays of imaging coils allow collection of ever-expanding amounts of data during the scan process.

Interpreting these large, complex medical image data sets will increasingly require powerful 3-D visualization tools. Diagnostic interpretation is still largely qualitative, subjective and prone to observer variability.

A recent computing and clinical collaboration has resulted in novel software that enables real-time, high-quality volume rendering of medical image data sets on standard desktop and laptop computer systems — taking advantage of the sophisticated hardware required for the latest generation of graphics-intensive computer games. The goal is to provide quantitative, objective and reliable assessments to help improve the accuracy and utility of modern medical imaging.

Volume visualization

Traditional volume visualization uses a mathematical technique called “ray casting” to render a two-dimensional image of a three-dimensional volume. For every pixel in the projected image, a ray is cast along the viewing vector through the 3-D data. Samples of the image data along this ray combine to produce the final intensity, or color, of the pixel on the projected 2-D image.

Altering the formulas for the composition of the sample can model effects such as lighting and opacity. However, this method is computationally expensive, requiring that the computer’s central processing unit perform most of the work to generate the image.

With colleagues in the department of computer science at the University of Calgary in Alberta, Canada, we have developed a technique for rendering 2-D images from 3-D data (Figure 1). Calgary Scientific Inc. plans to commercialize the resulting software, which we call ResolutionRD.

Figure 1. Above, a software technique can quickly render 3-D image data to suit a user’s needs. In this CT angiogram to investigate vasculature, the software provides simple texturing (left), maximum-intensity projection (center) and full-color surface rendering with lighting effects (right).

The technique employs the recent advances in graphical processing unit (GPU) power. Using dedicated, pipelined hardware and massive parallelism, modern graphics devices can render millions of textured polygons per second. Modern GPUs also enable the development of many general-purpose algorithms to take advantage of their tremendous floating-point processing capabilities.

We have used these new capabilities to perform high-quality 3-D visualizations at frame rates that were unthinkable two or three years ago. For example, we can render CT, MR and other medical image data sets with more than 100 million voxels (512 x 512 x 512 resolution) at fully interactive rates, on inexpensive “gaming” graphics cards common in desktop and laptop systems. This allows clinicians and researchers to interactively explore the image data in a full 3-D environment, rather than looking at precalculated 2-D projections.

Seeing data from all angles

However, an efficient implementation of a new algorithm, by itself, is not enough to ensure effective application in the clinical setting. Only very close and sustained integration with the clinical team enables us to understand their most fundamental needs and provide useful solutions.

The result is software with a clean and greatly simplified graphical user interface (Figure 2). Users rotate the data set in three dimensions by clicking in the data window and dragging the mouse. They move planes in and out of the data set by holding down the option key, then selecting and dragging.

Figure 2. The software is designed with a simple, uncluttered user interface.

Clinicians and researchers often need to visualize an image data set from several viewing angles with a high-quality, clear view of specific internal structures and organs. These two requirements can be mutually exclusive: Visualization of the entire data set typically occludes internal details; removing obscuring data can reduce important contextual information.

Medical illustration inspired the solution, a new real-time rendering method. Medical illustrators often provide contextual close-up views showing details of internal anatomy in an enlarged and offset region of interest, while still providing the general context of the close-up with a fully visible rendering of the entire data set. Physicians are familiar with this mode of visualization, which appears extensively in anatomy textbooks.

We have extended this traditional visualization technique to real-time multidimensional volume rendering: a “lens” rendering mode, where an offset region of interest provides a contextual close-up volume visualization (Figure 3). The user can manipulate the lens, moving it throughout the data set in 3-D, while changing its size and the magnification. In addition, the user can manipulate the view within the lens; for example, by rotating the data set.

Figure 3. The visualization software enables close-up views of occluded areas by using contextual “zoom” lenses, as in this example of bronchial/bronchiolar structures of the lungs.

To verify the utility and user-friendliness of the software, we have arranged a study with the stroke team at Foothills Medical Center in Calgary.

Stroke is a rapidly progressing trauma to the central nervous system. Time is of the utmost importance when dealing with this affliction. During treatment, stroke neurologists must rapidly and easily visualize the 3-D structure and function of brain arteries. However, current technical constraints mean that this is often difficult and time-consuming.

Figure 4. Among the possible applications for the software are helping physicians interpret clinical imaging data and enabling researchers to visualize volumetric images, such as this view of a mouse embryo.

Several stroke neurologists and stroke fellows at the center are using ResolutionRD to retrospectively examine CT angiogram data sets acquired from 60 patients who were admitted with acute ischemic stroke. Angiograms are imaging exams used to determine the health and function of blood vessels.

The neurologists will be looking for occluded arteries. Their diagnoses will be compared with the gold standard diagnoses provided by neuroradiologists with extensive experience interpreting stroke image data sets. The results will provide information about the speed and accuracy of diagnosis using the new software.

Figure 5. As well as with clinical applications, the 3-D visualization software can help in the research laboratory, as in this visualization of confocal microscope images of fluorescently tagged kidney cells.

We also are investigating 3-D visualization of other human anatomy, such as the heart, lungs and abdomen, as well as the 3-D structure of the mouse heart. The software also can produce 3-D renderings of seismic data and 3-D data from confocal microscopes (Figure 5).

Meet the authors

J. Ross Mitchell is with the Imaging Informatics Laboratory, Foothills Medical Center and the University of Calgary in Alberta, Canada. He also is founding scientist, a board member and a shareholder in Calgary Scientific Inc.; e-mail: rmitch@imaginginformatics.ca.

Sonney Chan also is affiliated with the Imaging Informatics Laboratory.

Pierre Lemire is director of the medical group at Calgary Scientific Inc.

References

1. M. Hadwiger et al (2002). High-quality volume graphics on consumer PC hardware. SIGGRAPH Course Notes.

2. T. Taerum et al (2006). Real-time super resolution contextual close-up of clinical volumetric data. Proc. of the Eighth Eurographics/IEEE-VGTC Symposium on Visualization (EUROVIS), May 2006.