Eye-tracking studies have contributed to the development of software that can aid reading of radiology images.
Dr. Elizabeth Krupinski, University of Arizona, and Josh Borah, Applied Science Laboratories
Improving the accuracy of radiology diagnostic
reading is critical for a number of reasons. Greater accuracy means that a higher
percentage of malignancies will be detected — and detected earlier —
increasing the chances for a complete cure. It also means fewer false-positives.
Reading x-rays accurately is an acquired skill
that can be enhanced through an examination of the way that a radiologist reads
an image. One way to do this is by using eye-tracking technology, which reveals
areas or objects at which the viewer looks and how long the viewer’s gaze
dwells on them.
It can be surmised, for example, that
the longer the viewer fixates on an object, the more difficult it is to discern
or interpret its characteristics.
Complicating matters is the change
in viewing technology. Increasingly, both monochrome and color monitors are replacing
x-ray films, for savings of cost and time. However, in some cases, the use of monitors
can make diagnosis more difficult, even though the radiologist has the advantage
of viewing the image in real time. Eye tracking can be used to evaluate the user-friendliness
of certain monitor types; it can improve speed and accuracy in x-ray diagnostic
reading, and ultimately save lives through early detection.
Eye-tracking technology
Eye tracking can be accomplished by a variety
of methods. For example, when researchers want to measure scan pattern (the sequence
of targets fixated) in a relatively unobtrusive fashion, they usually use a video
technique called pupil-to-corneal reflection.
Eye tracking provides
the coordinates of the locations where the radiologist’s gaze dwells, shown
here as circles. When used as an evaluative tool in a study involving radiology
residents, the technique helped improve overall diagnostic performance by 16 percent.
For this technique, the eye is illuminated
with a near-infrared light source and imaged with a solid-state video sensor. The
image is processed to identify the pupil and the reflection (mirror image) of the
light source on the front surface of the cornea. The relative position of the pupil
and corneal reflection can be used to compute the orientation of the eyeball with
respect to the eye camera and light source. This defines the line of gaze. The point
of gaze can then be computed by calculating where the line of gaze intersects surfaces,
such as a monitor displaying x-ray film.
Video eye trackers can be either remote
or head-mounted. Remote systems have an optics package (eye camera and light source)
that is typically placed on a table and aimed toward a seated subject. This is unobtrusive,
but if the subject looks too far away from the eye-tracker optics, a measurement
cannot be made. Data is also lost if subjects move to a position out of the eye
tracker’s range.
Head-mounted eye trackers do not have these problems, but
the headgear is more obtrusive. The advent of smaller and smaller solid-state video
cameras has allowed head-mounted systems to become progressively less obtrusive.
Video-based eye-tracking systems, such as the head-mounted one
shown here, are used to record and analyze the eye movements of radiologists while
they diagnose images. Doing so helps improve diagnostic speed and accuracy and can
help evaluate monitors that display digital images. Courtesy of Applied Science
Laboratories.
Both types of systems have been used
to track, record and analyze radiologists’ eye movements, providing information
that can improve image reading.
Radiologists “read” medical
images for anatomic and pathologic content and search for abnormalities. There is
a common assumption that they are better visual analyzers than most of their medical
colleagues. However, studies indicate that radiologists do not possess superior
visual skills compared with laypeople. Their expertise is more likely a combination
of specific visual and cognitive skills derived from medical training and experience
in detecting and determining the diagnostic importance of radiographic findings.1,2
In medical x-ray imaging, tumors and
fractures make up a significant portion of the types of abnormalities to be detected
and recognized. Visually scanning an x-ray image and deciding whether an abnormality
is present is difficult, and it is estimated that there is about a 30 percent miss
rate. Certain types of abnormalities, such as fractures in bone images, are more
likely to be overlooked than others. Eye-position analysis has been a useful tool
in helping to reveal how visual search is performed in radiology, isolating
the perceptual and cognitive causes of error, and designing a perceptual feedback
system to enhance the recognition of overlooked abnormalities.
One major assumption behind the use
of eye-position recording is that the amount of time the eyes focus on an object
reflects information processing, object encoding and recognition. This is assumed
because radiologists often fixate and refixate on the true target, dwelling on it
for prolonged periods, often without recognizing that they have discovered the object
of search. Monitoring the eye position of the radiologist provides the X- and Y-coordinates
of the dwelling location, which can be superimposed on the image and dynamically
fed back to the radiologist for re-evaluation.
In one study, 40 chest images (half
with pulmonary nodules, half without) were shown to six radiology residents whose
eye positions were recorded.1,2
In two sessions, they viewed the images
for 15 seconds and were asked to indicate whether any nodules were present and to
show the location. In one session, they then received perceptual feedback based
on where they had looked during the initial 15 seconds. Any location during the
initial search that received total dwell time of more than 1000 ms was circled during
the second view. In the control session, they took one more look at the image without
feedback circles.3
When the residents received feedback,
performance increased by 16 percent. They found more true nodules without increasing
false-positives. In the control session, detection performance did not change between
the two examinations of the image.
What attracts attention?
Studies have been conducted to determine whether
certain physical features of pulmonary nodules attract visual attention and contribute
to increased recognition and detection.4 A series of posteroanterior chest images
with solitary pulmonary nodules were searched by six radiologists as their eye positions
were recorded. The signal-to-noise ratio, size, conspicuity, location and calcification
status were measured for each nodule. Dwell parameters were correlated with nodule
features and related to detection rates. Only nodule size and conspicuity influenced
total dwell time, with larger, more conspicuous nodules receiving less visual attention
than the smaller, less conspicuous ones.
All nodule features examined influenced
overall detection performance even though most did not influence visual search and
attention. It was concluded that individual nodule features do not attract attention
as measured by “first hit” fixation data (i.e., the time it takes for
the axis of gaze to first land on the target/lesion of interest after the image
is displayed), but certain features do tend to hold attention once the nodule has
been fixated. The combination of all features affects whether it is detected.
Knowing which features attract the
visual attention of radiologists may help researchers develop computer-based image
analysis methods for identifying lesions, and develop image processing tools that
may enhance the visibility of those features of interest.
Signal-to-noise ratio
When a lesion stands out from the background tissue,
detection is relatively easy, but when the targets don’t stand out well, the
search becomes prolonged and detection rates decrease. Also, the more heterogeneous
the background and the more difficult it is to group heterogeneous background distractors
(and separate them from the potential targets), the more difficult the visual search
and detection becomes.
In a chest image, a pulmonary nodule
does not stand out well against the complex anatomical background of the image,
which is quite heterogeneous. Thus, it is likely that although certain features,
such as conspicuity and size, attract attention and dominate search, other features
— such as calcification status, location and signal-to-noise ratio —
must be processed and integrated to more completely discriminate a nodule from the
background. When features such as calcification are lacking or the signal-to-noise
ratio is weak, the predominant features may not be sufficient to separate the
nodule from the background.
Being aware of this may help researchers
develop computer-aided image analysis tools and image-processing tools to improve
the detectability of subtle lesions.
When a radiologist examines an x-ray
image, he or she must make a decision regarding observed anomalies. A study was
conducted to determine whether defective pattern recognition (fixating on the lesions
but failing to recognize them and moving on quickly to another area) or defective
decision making (fixating on a lesion for an extended period of time but actively
deciding it is not a lesion) is more to blame for “satisfaction of search”
errors in chest radiography.
Fifty-eight chest radiographs —
half of which demonstrated diverse, native abnormalities — were read by 20
observers as their eye positions were recorded.5 The radiographs were read twice,
once with and once without the addition of a simulated pulmonary nodule. Observers
provided a verbal account of their focus of attention, indicating suspicious features
and regions. They also provided a separate account of the abnormalities they would
include in a radiologic report.
In the end, more residual satisfaction-of-search
errors were caused by defective pattern recognition than by faulty decision making.
In other words, the lesions were missed because the observers spent more time fixating
on the distractors (satisfying their search) and not enough time fixating on the
targets of interest.
Analyzing eye-scanning patterns
Can the principles of search, detection and decision
making described for pulmonary nodule detection be applied to lesion detection in
mammographic images? An experiment was conducted wherein the eye positions of six
radiologists (three staff mammographers and three radiology residents) were recorded
as they searched mammograms for masses and microcalcifications.6
The overall detection performance of
the experienced and inexperienced readers was evaluated by calculating the true-
and false-positive fractions. The true-positive fraction was higher for the
more experienced readers than for the less experienced readers (0.89 vs. 0.72),
and the false-positive fraction was lower for the more experienced readers than
for those less experienced (0.13 vs. 0.53). In general, the less experienced readers
spent significantly more time scanning the mammography cases than did the experienced
ones.
>By improving our understanding of what
makes an experienced radiologist a better and more efficient searcher and detector,
we may be able to develop better teaching methods. It might even be possible to
determine and teach optimal search strategies. Currently, residents learn mainly
by observing the overt behavior of attending radiologists.
X-ray film vs. monitors
In the transition from the use of x-ray film to
that of real-time monitors, some problems have been noted. As mentioned earlier,
signal-to-noise ratio is an important factor in the ability of the reader to quickly,
accurately and decisively determine the nature of the object examined.
How have eye-tracking studies indicated
that the use of monitors has introduced difficulties? In one study, 27 nonconsecutive
bone trauma computed radiographs were collected from the routine emergency practice.7
The eye positions of three bone radiologists and three orthopedic surgeons were
recorded as they searched film images on a view box and digital images at a workstation.
During the experiment, radiological
images displayed on a monitor were viewed differently by readers than were images
displayed on view boxes. It seems that readers initially scanned the distal image
to obtain a global impression of image quality (three fixations on average) and
then used the image-processing functions of the workstation to adjust the image
to their liking. Approximately 20 percent of all scanning time was spent on image-processing
functions.
Overall viewing time was longer for
images displayed on a monitor. Time to first fixate a lesion and true-negative dwell
times also were significantly longer. Absolute numbers of clusters and dwell times
were greater for diagnostic image areas on the monitor than on the film. Twenty
percent of the clusters for images viewed on the monitor were on the image-processing
menu. It was concluded that the amount and type of information that is processed
during search is different when images are viewed on a monitor rather than on film.
Longer dwell times do, however, indicate
that more perceptual and cognitive types of processing are occurring and that more
attention is being given to these areas. This finding suggests that more time is
needed in general to detect, extract and identify relevant features when images
are being viewed on a monitor. The longer time needed could be due to the lower
resolution and lower brightness of the monitor compared with film.
Studies indicate that care should be
taken in deciding which type of monitor to use for primary diagnostic interpretation
of radiographic images in the digital environment.8 The images that will be viewed
on a given monitor and the nature of the diagnostic task should also be considered.
Observer detection performance and visual search behavior are significantly better
(more efficient) with monochrome than with color monitors.
High luminance
Additionally, display luminance affects
visual search performance with both film and monitor displays without affecting
detection performance significantly. Higher-luminance displays yield more efficient
search performance. The true-negative dwell times and number of clusters suggest
that lower-luminance levels prolong the search and recognition of normal, lesion-free
areas compared with lesion-containing areas.9
Analysis of radiological reading through
the use of eye-tracking technology helps identify potential problems in radiological
accuracy. Problems arising from certain types of images, viewing patterns and changes
in technology can be identified and corrected. The use of this technology has shown
how skills may be improved to increase speed and accuracy of radiological reading,
where shortcomings in technology may be addressed, and how time and energy can be
saved through reducing the number of false-positives in diagnosis.
As eye trackers become increasingly
automatic and require progressively less effort and experience on the part of the
equipment operator, it may become practical to move eye tracking out of the research
environment, and to use it as part of the training process for radiologists and
other professionals who need to perform comparable image search and recognition
tasks.
Meet the authors
Elizabeth Krupinski is research associate professor
at the University of Arizona in Tucson; e-mail: [email protected].
Josh Borah is vice president of engineering
at Applied Science Laboratories in Bedford, Mass.; e-mail: [email protected].
References
1. E.A. Krupinski et al (April 1993). A perceptually
based method for enhancing pulmonary nodule recognition. INVEST RADIOL, pp.
289-294.
2. E.A. Krupinski et al (May 1993).
Perceptual enhancement of tumor targets in chest X-ray images. PERCEPT PSYCHOPHYS,
pp. 519-526.
3. H.L. Kundel et al (August 1990).
Computer-displayed eye position as a visual aid to pulmonary nodule interpretation.
INVEST RADIOL, pp. 890-896.
4. E.A. Krupinski et al (August 2003).
Searching for nodules: what features attract attention and influence detection?
ACAD RADIOL, pp. 861-868.
5. K.S. Berbaum et al (December 2000).
Role of faulty decision making in the satisfaction of search effect in chest radiography.
ACAD RADIOL, pp. 1098-1106.
6. E.A. Krupinski (February 1996).
Visual scanning patterns of radiologists searching mammograms. ACAD RADIOL,
pp. 137-144.
7. E.A. Krupinski and P.J. Lund (March
1997). Differences in time to interpretation for evaluation of bone radiographs
with monitor and film viewing. ACAD RADIOL, pp. 177-182.
8. E.A. Krupinski and H. Roehrig (June
2002). Pulmonary nodule detection and visual search: P45 and P104 monochrome versus
color monitor displays. ACAD RADIOL, pp. 638-645.
9. E.A. Krupinski et al (July 1999).
Influence of film and monitor display luminance on observer performance and visual
search. ACAD RADIOL, pp. 411-418.