Detection of breast cancer with mammography: Effect of an artificial intelligence support system

Alejandro Rodríguez-Ruiz, Elizabeth A Krupinski, Jan Jurre Mordang, Kathy Schilling, Sylvia H. Heywang-Köbrunner, Ioannis Sechopoulos, Ritse M. Mann

Research output: Contribution to journalArticle

18 Citations (Scopus)

Abstract

Purpose: To compare breast cancer detection performance of radiologists reading mammographic examinations unaided versus supported by an artificial intelligence (AI) system. Materials and Methods: An enriched retrospective, fully crossed, multireader, multicase, HIPAA-compliant study was performed. Screening digital mammographic examinations from 240 women (median age, 62 years; range, 39–89 years) performed between 2013 and 2017 were included. The 240 examinations (100 showing cancers, 40 leading to false-positive recalls, 100 normal) were interpreted by 14 Mammography Quality Standards Act–qualified radiologists, once with and once without AI support. The readers provided a Breast Imaging Reporting and Data System score and probability of malignancy. AI support provided radiologists with interactive decision support (clicking on a breast region yields a local cancer likelihood score), traditional lesion markers for computer-detected abnormalities, and an examination-based cancer likelihood score. The area under the receiver operating characteristic curve (AUC), specificity and sensitivity, and reading time were compared between conditions by using mixed-models analysis dof variance and generalized linear models for multiple repeated measurements. Results: On average, the AUC was higher with AI support than with unaided reading (0.89 vs 0.87, respectively; P = .002). Sensitivity increased with AI support (86% [86 of 100] vs 83% [83 of 100]; P = .046), whereas specificity trended toward improvement (79% [111 of 140]) vs 77% [108 of 140]; P = .06). Reading time per case was similar (unaided, 146 seconds; supported by AI, 149 seconds; P = .15). The AUC with the AI system alone was similar to the average AUC of the radiologists (0.89 vs 0.87). Conclusion: Radiologists improved their cancer detection at mammography when using an artificial intelligence system for support, without requiring additional reading time.

Original languageEnglish (US)
Pages (from-to)1-10
Number of pages10
JournalRadiology
Volume290
Issue number3
DOIs
StatePublished - Mar 1 2019
Externally publishedYes

Fingerprint

Artificial Intelligence
Mammography
Breast Neoplasms
Reading
Area Under Curve
Neoplasms
Breast
Health Insurance Portability and Accountability Act
Information Systems
ROC Curve
Linear Models
Analysis of Variance
Radiologists
Sensitivity and Specificity

ASJC Scopus subject areas

  • Radiology Nuclear Medicine and imaging

Cite this

Rodríguez-Ruiz, A., Krupinski, E. A., Mordang, J. J., Schilling, K., Heywang-Köbrunner, S. H., Sechopoulos, I., & Mann, R. M. (2019). Detection of breast cancer with mammography: Effect of an artificial intelligence support system. Radiology, 290(3), 1-10. https://doi.org/10.1148/radiol.2018181371

Detection of breast cancer with mammography : Effect of an artificial intelligence support system. / Rodríguez-Ruiz, Alejandro; Krupinski, Elizabeth A; Mordang, Jan Jurre; Schilling, Kathy; Heywang-Köbrunner, Sylvia H.; Sechopoulos, Ioannis; Mann, Ritse M.

In: Radiology, Vol. 290, No. 3, 01.03.2019, p. 1-10.

Research output: Contribution to journalArticle

Rodríguez-Ruiz, A, Krupinski, EA, Mordang, JJ, Schilling, K, Heywang-Köbrunner, SH, Sechopoulos, I & Mann, RM 2019, 'Detection of breast cancer with mammography: Effect of an artificial intelligence support system', Radiology, vol. 290, no. 3, pp. 1-10. https://doi.org/10.1148/radiol.2018181371
Rodríguez-Ruiz A, Krupinski EA, Mordang JJ, Schilling K, Heywang-Köbrunner SH, Sechopoulos I et al. Detection of breast cancer with mammography: Effect of an artificial intelligence support system. Radiology. 2019 Mar 1;290(3):1-10. https://doi.org/10.1148/radiol.2018181371
Rodríguez-Ruiz, Alejandro ; Krupinski, Elizabeth A ; Mordang, Jan Jurre ; Schilling, Kathy ; Heywang-Köbrunner, Sylvia H. ; Sechopoulos, Ioannis ; Mann, Ritse M. / Detection of breast cancer with mammography : Effect of an artificial intelligence support system. In: Radiology. 2019 ; Vol. 290, No. 3. pp. 1-10.
@article{8fee5f7cff1f444bb804ba7424076c13,
title = "Detection of breast cancer with mammography: Effect of an artificial intelligence support system",
abstract = "Purpose: To compare breast cancer detection performance of radiologists reading mammographic examinations unaided versus supported by an artificial intelligence (AI) system. Materials and Methods: An enriched retrospective, fully crossed, multireader, multicase, HIPAA-compliant study was performed. Screening digital mammographic examinations from 240 women (median age, 62 years; range, 39–89 years) performed between 2013 and 2017 were included. The 240 examinations (100 showing cancers, 40 leading to false-positive recalls, 100 normal) were interpreted by 14 Mammography Quality Standards Act–qualified radiologists, once with and once without AI support. The readers provided a Breast Imaging Reporting and Data System score and probability of malignancy. AI support provided radiologists with interactive decision support (clicking on a breast region yields a local cancer likelihood score), traditional lesion markers for computer-detected abnormalities, and an examination-based cancer likelihood score. The area under the receiver operating characteristic curve (AUC), specificity and sensitivity, and reading time were compared between conditions by using mixed-models analysis dof variance and generalized linear models for multiple repeated measurements. Results: On average, the AUC was higher with AI support than with unaided reading (0.89 vs 0.87, respectively; P = .002). Sensitivity increased with AI support (86{\%} [86 of 100] vs 83{\%} [83 of 100]; P = .046), whereas specificity trended toward improvement (79{\%} [111 of 140]) vs 77{\%} [108 of 140]; P = .06). Reading time per case was similar (unaided, 146 seconds; supported by AI, 149 seconds; P = .15). The AUC with the AI system alone was similar to the average AUC of the radiologists (0.89 vs 0.87). Conclusion: Radiologists improved their cancer detection at mammography when using an artificial intelligence system for support, without requiring additional reading time.",
author = "Alejandro Rodr{\'i}guez-Ruiz and Krupinski, {Elizabeth A} and Mordang, {Jan Jurre} and Kathy Schilling and Heywang-K{\"o}brunner, {Sylvia H.} and Ioannis Sechopoulos and Mann, {Ritse M.}",
year = "2019",
month = "3",
day = "1",
doi = "10.1148/radiol.2018181371",
language = "English (US)",
volume = "290",
pages = "1--10",
journal = "Radiology",
issn = "0033-8419",
publisher = "Radiological Society of North America Inc.",
number = "3",

}

TY - JOUR

T1 - Detection of breast cancer with mammography

T2 - Effect of an artificial intelligence support system

AU - Rodríguez-Ruiz, Alejandro

AU - Krupinski, Elizabeth A

AU - Mordang, Jan Jurre

AU - Schilling, Kathy

AU - Heywang-Köbrunner, Sylvia H.

AU - Sechopoulos, Ioannis

AU - Mann, Ritse M.

PY - 2019/3/1

Y1 - 2019/3/1

N2 - Purpose: To compare breast cancer detection performance of radiologists reading mammographic examinations unaided versus supported by an artificial intelligence (AI) system. Materials and Methods: An enriched retrospective, fully crossed, multireader, multicase, HIPAA-compliant study was performed. Screening digital mammographic examinations from 240 women (median age, 62 years; range, 39–89 years) performed between 2013 and 2017 were included. The 240 examinations (100 showing cancers, 40 leading to false-positive recalls, 100 normal) were interpreted by 14 Mammography Quality Standards Act–qualified radiologists, once with and once without AI support. The readers provided a Breast Imaging Reporting and Data System score and probability of malignancy. AI support provided radiologists with interactive decision support (clicking on a breast region yields a local cancer likelihood score), traditional lesion markers for computer-detected abnormalities, and an examination-based cancer likelihood score. The area under the receiver operating characteristic curve (AUC), specificity and sensitivity, and reading time were compared between conditions by using mixed-models analysis dof variance and generalized linear models for multiple repeated measurements. Results: On average, the AUC was higher with AI support than with unaided reading (0.89 vs 0.87, respectively; P = .002). Sensitivity increased with AI support (86% [86 of 100] vs 83% [83 of 100]; P = .046), whereas specificity trended toward improvement (79% [111 of 140]) vs 77% [108 of 140]; P = .06). Reading time per case was similar (unaided, 146 seconds; supported by AI, 149 seconds; P = .15). The AUC with the AI system alone was similar to the average AUC of the radiologists (0.89 vs 0.87). Conclusion: Radiologists improved their cancer detection at mammography when using an artificial intelligence system for support, without requiring additional reading time.

AB - Purpose: To compare breast cancer detection performance of radiologists reading mammographic examinations unaided versus supported by an artificial intelligence (AI) system. Materials and Methods: An enriched retrospective, fully crossed, multireader, multicase, HIPAA-compliant study was performed. Screening digital mammographic examinations from 240 women (median age, 62 years; range, 39–89 years) performed between 2013 and 2017 were included. The 240 examinations (100 showing cancers, 40 leading to false-positive recalls, 100 normal) were interpreted by 14 Mammography Quality Standards Act–qualified radiologists, once with and once without AI support. The readers provided a Breast Imaging Reporting and Data System score and probability of malignancy. AI support provided radiologists with interactive decision support (clicking on a breast region yields a local cancer likelihood score), traditional lesion markers for computer-detected abnormalities, and an examination-based cancer likelihood score. The area under the receiver operating characteristic curve (AUC), specificity and sensitivity, and reading time were compared between conditions by using mixed-models analysis dof variance and generalized linear models for multiple repeated measurements. Results: On average, the AUC was higher with AI support than with unaided reading (0.89 vs 0.87, respectively; P = .002). Sensitivity increased with AI support (86% [86 of 100] vs 83% [83 of 100]; P = .046), whereas specificity trended toward improvement (79% [111 of 140]) vs 77% [108 of 140]; P = .06). Reading time per case was similar (unaided, 146 seconds; supported by AI, 149 seconds; P = .15). The AUC with the AI system alone was similar to the average AUC of the radiologists (0.89 vs 0.87). Conclusion: Radiologists improved their cancer detection at mammography when using an artificial intelligence system for support, without requiring additional reading time.

UR - http://www.scopus.com/inward/record.url?scp=85060384196&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85060384196&partnerID=8YFLogxK

U2 - 10.1148/radiol.2018181371

DO - 10.1148/radiol.2018181371

M3 - Article

C2 - 30457482

AN - SCOPUS:85060384196

VL - 290

SP - 1

EP - 10

JO - Radiology

JF - Radiology

SN - 0033-8419

IS - 3

ER -