Eye Tracking Helps Improve Accuracy in Radiology

Dr. Elizabeth Krupinski, University of Arizona, and Josh Borah, Applied Science Laboratories

Improving the accuracy of radiology diagnostic reading is critical for a number of reasons. Greater accuracy means that a higher percentage of malignancies will be detected — and detected earlier — increasing the chances for a complete cure. It also means fewer false-positives.

Reading x-rays accurately is an acquired skill that can be enhanced through an examination of the way that a radiologist reads an image. One way to do this is by using eye-tracking technology, which reveals areas or objects at which the viewer looks and how long the viewer’s gaze dwells on them.

It can be surmised, for example, that the longer the viewer fixates on an object, the more difficult it is to discern or interpret its characteristics.

Complicating matters is the change in viewing technology. Increasingly, both monochrome and color monitors are replacing x-ray films, for savings of cost and time. However, in some cases, the use of monitors can make diagnosis more difficult, even though the radiologist has the advantage of viewing the image in real time. Eye tracking can be used to evaluate the user-friendliness of certain monitor types; it can improve speed and accuracy in x-ray diagnostic reading, and ultimately save lives through early detection.

Eye-tracking technology

Eye tracking can be accomplished by a variety of methods. For example, when researchers want to measure scan pattern (the sequence of targets fixated) in a relatively unobtrusive fashion, they usually use a video technique called pupil-to-corneal reflection.

Eye tracking provides the coordinates of the locations where the radiologist’s gaze dwells, shown here as circles. When used as an evaluative tool in a study involving radiology residents, the technique helped improve overall diagnostic performance by 16 percent.

For this technique, the eye is illuminated with a near-infrared light source and imaged with a solid-state video sensor. The image is processed to identify the pupil and the reflection (mirror image) of the light source on the front surface of the cornea. The relative position of the pupil and corneal reflection can be used to compute the orientation of the eyeball with respect to the eye camera and light source. This defines the line of gaze. The point of gaze can then be computed by calculating where the line of gaze intersects surfaces, such as a monitor displaying x-ray film.

Video eye trackers can be either remote or head-mounted. Remote systems have an optics package (eye camera and light source) that is typically placed on a table and aimed toward a seated subject. This is unobtrusive, but if the subject looks too far away from the eye-tracker optics, a measurement cannot be made. Data is also lost if subjects move to a position out of the eye tracker’s range.

Head-mounted eye trackers do not have these problems, but the headgear is more obtrusive. The advent of smaller and smaller solid-state video cameras has allowed head-mounted systems to become progressively less obtrusive.

Video-based eye-tracking systems, such as the head-mounted one shown here, are used to record and analyze the eye movements of radiologists while they diagnose images. Doing so helps improve diagnostic speed and accuracy and can help evaluate monitors that display digital images. Courtesy of Applied Science Laboratories.

Both types of systems have been used to track, record and analyze radiologists’ eye movements, providing information that can improve image reading.

Radiologists “read” medical images for anatomic and pathologic content and search for abnormalities. There is a common assumption that they are better visual analyzers than most of their medical colleagues. However, studies indicate that radiologists do not possess superior visual skills compared with laypeople. Their expertise is more likely a combination of specific visual and cognitive skills derived from medical training and experience in detecting and determining the diagnostic importance of radiographic findings.^1,2

In medical x-ray imaging, tumors and fractures make up a significant portion of the types of abnormalities to be detected and recognized. Visually scanning an x-ray image and deciding whether an abnormality is present is difficult, and it is estimated that there is about a 30 percent miss rate. Certain types of abnormalities, such as fractures in bone images, are more likely to be overlooked than others. Eye-position analysis has been a useful tool in helping to reveal how visual search is performed in radiology, isolating the perceptual and cognitive causes of error, and designing a perceptual feedback system to enhance the recognition of overlooked abnormalities.

One major assumption behind the use of eye-position recording is that the amount of time the eyes focus on an object reflects information processing, object encoding and recognition. This is assumed because radiologists often fixate and refixate on the true target, dwelling on it for prolonged periods, often without recognizing that they have discovered the object of search. Monitoring the eye position of the radiologist provides the X- and Y-coordinates of the dwelling location, which can be superimposed on the image and dynamically fed back to the radiologist for re-evaluation.

In one study, 40 chest images (half with pulmonary nodules, half without) were shown to six radiology residents whose eye positions were recorded.^1,2

In two sessions, they viewed the images for 15 seconds and were asked to indicate whether any nodules were present and to show the location. In one session, they then received perceptual feedback based on where they had looked during the initial 15 seconds. Any location during the initial search that received total dwell time of more than 1000 ms was circled during the second view. In the control session, they took one more look at the image without feedback circles.³

When the residents received feedback, performance increased by 16 percent. They found more true nodules without increasing false-positives. In the control session, detection performance did not change between the two examinations of the image.

What attracts attention?

Studies have been conducted to determine whether certain physical features of pulmonary nodules attract visual attention and contribute to increased recognition and detection.⁴ A series of posteroanterior chest images with solitary pulmonary nodules were searched by six radiologists as their eye positions were recorded. The signal-to-noise ratio, size, conspicuity, location and calcification status were measured for each nodule. Dwell parameters were correlated with nodule features and related to detection rates. Only nodule size and conspicuity influenced total dwell time, with larger, more conspicuous nodules receiving less visual attention than the smaller, less conspicuous ones.

All nodule features examined influenced overall detection performance even though most did not influence visual search and attention. It was concluded that individual nodule features do not attract attention as measured by “first hit” fixation data (i.e., the time it takes for the axis of gaze to first land on the target/lesion of interest after the image is displayed), but certain features do tend to hold attention once the nodule has been fixated. The combination of all features affects whether it is detected.

Knowing which features attract the visual attention of radiologists may help researchers develop computer-based image analysis methods for identifying lesions, and develop image processing tools that may enhance the visibility of those features of interest.

Signal-to-noise ratio

When a lesion stands out from the background tissue, detection is relatively easy, but when the targets don’t stand out well, the search becomes prolonged and detection rates decrease. Also, the more heterogeneous the background and the more difficult it is to group heterogeneous background distractors (and separate them from the potential targets), the more difficult the visual search and detection becomes.

In a chest image, a pulmonary nodule does not stand out well against the complex anatomical background of the image, which is quite heterogeneous. Thus, it is likely that although certain features, such as conspicuity and size, attract attention and dominate search, other features — such as calcification status, location and signal-to-noise ratio — must be processed and integrated to more completely discriminate a nodule from the background. When features such as calcification are lacking or the signal-to-noise ratio is weak, the predominant features may not be sufficient to separate the nodule from the background.

Being aware of this may help researchers develop computer-aided image analysis tools and image-processing tools to improve the detectability of subtle lesions.

When a radiologist examines an x-ray image, he or she must make a decision regarding observed anomalies. A study was conducted to determine whether defective pattern recognition (fixating on the lesions but failing to recognize them and moving on quickly to another area) or defective decision making (fixating on a lesion for an extended period of time but actively deciding it is not a lesion) is more to blame for “satisfaction of search” errors in chest radiography.

Fifty-eight chest radiographs — half of which demonstrated diverse, native abnormalities — were read by 20 observers as their eye positions were recorded.⁵ The radiographs were read twice, once with and once without the addition of a simulated pulmonary nodule. Observers provided a verbal account of their focus of attention, indicating suspicious features and regions. They also provided a separate account of the abnormalities they would include in a radiologic report.

In the end, more residual satisfaction-of-search errors were caused by defective pattern recognition than by faulty decision making. In other words, the lesions were missed because the observers spent more time fixating on the distractors (satisfying their search) and not enough time fixating on the targets of interest.

Analyzing eye-scanning patterns

Can the principles of search, detection and decision making described for pulmonary nodule detection be applied to lesion detection in mammographic images? An experiment was conducted wherein the eye positions of six radiologists (three staff mammographers and three radiology residents) were recorded as they searched mammograms for masses and microcalcifications.⁶

The overall detection performance of the experienced and inexperienced readers was evaluated by calculating the true- and false-positive fractions. The true-positive fraction was higher for the more experienced readers than for the less experienced readers (0.89 vs. 0.72), and the false-positive fraction was lower for the more experienced readers than for those less experienced (0.13 vs. 0.53). In general, the less experienced readers spent significantly more time scanning the mammography cases than did the experienced ones.

>By improving our understanding of what makes an experienced radiologist a better and more efficient searcher and detector, we may be able to develop better teaching methods. It might even be possible to determine and teach optimal search strategies. Currently, residents learn mainly by observing the overt behavior of attending radiologists.

X-ray film vs. monitors

In the transition from the use of x-ray film to that of real-time monitors, some problems have been noted. As mentioned earlier, signal-to-noise ratio is an important factor in the ability of the reader to quickly, accurately and decisively determine the nature of the object examined.

How have eye-tracking studies indicated that the use of monitors has introduced difficulties? In one study, 27 nonconsecutive bone trauma computed radiographs were collected from the routine emergency practice.⁷ The eye positions of three bone radiologists and three orthopedic surgeons were recorded as they searched film images on a view box and digital images at a workstation.

During the experiment, radiological images displayed on a monitor were viewed differently by readers than were images displayed on view boxes. It seems that readers initially scanned the distal image to obtain a global impression of image quality (three fixations on average) and then used the image-processing functions of the workstation to adjust the image to their liking. Approximately 20 percent of all scanning time was spent on image-processing functions.

Overall viewing time was longer for images displayed on a monitor. Time to first fixate a lesion and true-negative dwell times also were significantly longer. Absolute numbers of clusters and dwell times were greater for diagnostic image areas on the monitor than on the film. Twenty percent of the clusters for images viewed on the monitor were on the image-processing menu. It was concluded that the amount and type of information that is processed during search is different when images are viewed on a monitor rather than on film.

Longer dwell times do, however, indicate that more perceptual and cognitive types of processing are occurring and that more attention is being given to these areas. This finding suggests that more time is needed in general to detect, extract and identify relevant features when images are being viewed on a monitor. The longer time needed could be due to the lower resolution and lower brightness of the monitor compared with film.

Studies indicate that care should be taken in deciding which type of monitor to use for primary diagnostic interpretation of radiographic images in the digital environment.⁸ The images that will be viewed on a given monitor and the nature of the diagnostic task should also be considered. Observer detection performance and visual search behavior are significantly better (more efficient) with monochrome than with color monitors.

High luminance

Additionally, display luminance affects visual search performance with both film and monitor displays without affecting detection performance significantly. Higher-luminance displays yield more efficient search performance. The true-negative dwell times and number of clusters suggest that lower-luminance levels prolong the search and recognition of normal, lesion-free areas compared with lesion-containing areas.⁹

Analysis of radiological reading through the use of eye-tracking technology helps identify potential problems in radiological accuracy. Problems arising from certain types of images, viewing patterns and changes in technology can be identified and corrected. The use of this technology has shown how skills may be improved to increase speed and accuracy of radiological reading, where shortcomings in technology may be addressed, and how time and energy can be saved through reducing the number of false-positives in diagnosis.

As eye trackers become increasingly automatic and require progressively less effort and experience on the part of the equipment operator, it may become practical to move eye tracking out of the research environment, and to use it as part of the training process for radiologists and other professionals who need to perform comparable image search and recognition tasks.

Meet the authors

Elizabeth Krupinski is research associate professor at the University of Arizona in Tucson; e-mail: krupinski@radiology.arizona.edu.

Josh Borah is vice president of engineering at Applied Science Laboratories in Bedford, Mass.; e-mail: josh@a-s-l.com.

References

1. E.A. Krupinski et al (April 1993). A perceptually based method for enhancing pulmonary nodule recognition. INVEST RADIOL, pp. 289-294.

2. E.A. Krupinski et al (May 1993). Perceptual enhancement of tumor targets in chest X-ray images. PERCEPT PSYCHOPHYS, pp. 519-526.

3. H.L. Kundel et al (August 1990). Computer-displayed eye position as a visual aid to pulmonary nodule interpretation. INVEST RADIOL, pp. 890-896.

4. E.A. Krupinski et al (August 2003). Searching for nodules: what features attract attention and influence detection? ACAD RADIOL, pp. 861-868.

5. K.S. Berbaum et al (December 2000). Role of faulty decision making in the satisfaction of search effect in chest radiography. ACAD RADIOL, pp. 1098-1106.

6. E.A. Krupinski (February 1996). Visual scanning patterns of radiologists searching mammograms. ACAD RADIOL, pp. 137-144.

7. E.A. Krupinski and P.J. Lund (March 1997). Differences in time to interpretation for evaluation of bone radiographs with monitor and film viewing. ACAD RADIOL, pp. 177-182.

8. E.A. Krupinski and H. Roehrig (June 2002). Pulmonary nodule detection and visual search: P45 and P104 monochrome versus color monitor displays. ACAD RADIOL, pp. 638-645.

9. E.A. Krupinski et al (July 1999). Influence of film and monitor display luminance on observer performance and visual search. ACAD RADIOL, pp. 411-418.