Investigating the link between radiologists' gaze, diagnostic decision, and image content

Georgia Tourassi, Sophie Voisin, Vincent Paquit, Elizabeth A Krupinski

Research output: Contribution to journalArticle

33 Citations (Scopus)

Abstract

Objective: To investigate machine learning for linking image content, human perception, cognition, and error in the diagnostic interpretation of mammograms. Methods: Gaze data and diagnostic decisions were collected from three breast imaging radiologists and three radiology residents who reviewed 20 screening mammograms while wearing a head-mounted eyetracker. Image analysis was performed in mammographic regions that attracted radiologists' attention and in all abnormal regions. Machine learning algorithms were investigated to develop predictive models that link: (i) image content with gaze, (ii) image content and gaze with cognition, and (iii) image content, gaze, and cognition with diagnostic error. Both group-based and individualized models were explored. Results: By pooling the data from all readers, machine learning produced highly accurate predictive models linking image content, gaze, and cognition. Potential linking of those with diagnostic error was also supported to some extent. Merging readers' gaze metrics and cognitive opinions with computer-extracted image features identified 59% of the readers' diagnostic errors while confirming 97.3% of their correct diagnoses. The readers' individual perceptual and cognitive behaviors could be adequately predicted by modeling the behavior of others. However, personalized tuning was in many cases beneficial for capturing more accurately individual behavior. Conclusions: There is clearly an interaction between radiologists' gaze, diagnostic decision, and image content which can be modeled with machine learning algorithms.

Original languageEnglish (US)
Pages (from-to)1067-1075
Number of pages9
JournalJournal of the American Medical Informatics Association
Volume20
Issue number6
DOIs
StatePublished - 2013

Fingerprint

Diagnostic Errors
Cognition
Radiology
Meta-Analysis
Breast
Head
Radiologists
Machine Learning

ASJC Scopus subject areas

  • Health Informatics

Cite this

Investigating the link between radiologists' gaze, diagnostic decision, and image content. / Tourassi, Georgia; Voisin, Sophie; Paquit, Vincent; Krupinski, Elizabeth A.

In: Journal of the American Medical Informatics Association, Vol. 20, No. 6, 2013, p. 1067-1075.

Research output: Contribution to journalArticle

@article{0855900120ba41d88d7bdc6e4f2c781c,
title = "Investigating the link between radiologists' gaze, diagnostic decision, and image content",
abstract = "Objective: To investigate machine learning for linking image content, human perception, cognition, and error in the diagnostic interpretation of mammograms. Methods: Gaze data and diagnostic decisions were collected from three breast imaging radiologists and three radiology residents who reviewed 20 screening mammograms while wearing a head-mounted eyetracker. Image analysis was performed in mammographic regions that attracted radiologists' attention and in all abnormal regions. Machine learning algorithms were investigated to develop predictive models that link: (i) image content with gaze, (ii) image content and gaze with cognition, and (iii) image content, gaze, and cognition with diagnostic error. Both group-based and individualized models were explored. Results: By pooling the data from all readers, machine learning produced highly accurate predictive models linking image content, gaze, and cognition. Potential linking of those with diagnostic error was also supported to some extent. Merging readers' gaze metrics and cognitive opinions with computer-extracted image features identified 59{\%} of the readers' diagnostic errors while confirming 97.3{\%} of their correct diagnoses. The readers' individual perceptual and cognitive behaviors could be adequately predicted by modeling the behavior of others. However, personalized tuning was in many cases beneficial for capturing more accurately individual behavior. Conclusions: There is clearly an interaction between radiologists' gaze, diagnostic decision, and image content which can be modeled with machine learning algorithms.",
author = "Georgia Tourassi and Sophie Voisin and Vincent Paquit and Krupinski, {Elizabeth A}",
year = "2013",
doi = "10.1136/amiajnl-2012-001503",
language = "English (US)",
volume = "20",
pages = "1067--1075",
journal = "Journal of the American Medical Informatics Association : JAMIA",
issn = "1067-5027",
publisher = "Oxford University Press",
number = "6",

}

TY - JOUR

T1 - Investigating the link between radiologists' gaze, diagnostic decision, and image content

AU - Tourassi, Georgia

AU - Voisin, Sophie

AU - Paquit, Vincent

AU - Krupinski, Elizabeth A

PY - 2013

Y1 - 2013

N2 - Objective: To investigate machine learning for linking image content, human perception, cognition, and error in the diagnostic interpretation of mammograms. Methods: Gaze data and diagnostic decisions were collected from three breast imaging radiologists and three radiology residents who reviewed 20 screening mammograms while wearing a head-mounted eyetracker. Image analysis was performed in mammographic regions that attracted radiologists' attention and in all abnormal regions. Machine learning algorithms were investigated to develop predictive models that link: (i) image content with gaze, (ii) image content and gaze with cognition, and (iii) image content, gaze, and cognition with diagnostic error. Both group-based and individualized models were explored. Results: By pooling the data from all readers, machine learning produced highly accurate predictive models linking image content, gaze, and cognition. Potential linking of those with diagnostic error was also supported to some extent. Merging readers' gaze metrics and cognitive opinions with computer-extracted image features identified 59% of the readers' diagnostic errors while confirming 97.3% of their correct diagnoses. The readers' individual perceptual and cognitive behaviors could be adequately predicted by modeling the behavior of others. However, personalized tuning was in many cases beneficial for capturing more accurately individual behavior. Conclusions: There is clearly an interaction between radiologists' gaze, diagnostic decision, and image content which can be modeled with machine learning algorithms.

AB - Objective: To investigate machine learning for linking image content, human perception, cognition, and error in the diagnostic interpretation of mammograms. Methods: Gaze data and diagnostic decisions were collected from three breast imaging radiologists and three radiology residents who reviewed 20 screening mammograms while wearing a head-mounted eyetracker. Image analysis was performed in mammographic regions that attracted radiologists' attention and in all abnormal regions. Machine learning algorithms were investigated to develop predictive models that link: (i) image content with gaze, (ii) image content and gaze with cognition, and (iii) image content, gaze, and cognition with diagnostic error. Both group-based and individualized models were explored. Results: By pooling the data from all readers, machine learning produced highly accurate predictive models linking image content, gaze, and cognition. Potential linking of those with diagnostic error was also supported to some extent. Merging readers' gaze metrics and cognitive opinions with computer-extracted image features identified 59% of the readers' diagnostic errors while confirming 97.3% of their correct diagnoses. The readers' individual perceptual and cognitive behaviors could be adequately predicted by modeling the behavior of others. However, personalized tuning was in many cases beneficial for capturing more accurately individual behavior. Conclusions: There is clearly an interaction between radiologists' gaze, diagnostic decision, and image content which can be modeled with machine learning algorithms.

UR - http://www.scopus.com/inward/record.url?scp=84886297620&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84886297620&partnerID=8YFLogxK

U2 - 10.1136/amiajnl-2012-001503

DO - 10.1136/amiajnl-2012-001503

M3 - Article

VL - 20

SP - 1067

EP - 1075

JO - Journal of the American Medical Informatics Association : JAMIA

JF - Journal of the American Medical Informatics Association : JAMIA

SN - 1067-5027

IS - 6

ER -