The variation of radiologists' performance over the course of a reading session

Markus C. Elze, Sian Taylor-Phillips, Claudia Mello-Thoms, Elizabeth A Krupinski, Alastair G. Gale, Aileen Clarke

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Citations (Scopus)

Abstract

The radiologist's task of reviewing many cases successively is highly repetitive and requires a high level of concentration. Fatigue effects have, for example, been shown in studies comparing performance at different times of day. However, little is known about changes in performance during an individual reading session. During a session reading an enriched case set, performance may be affected by both fatigue (i.e. decreasing performance) and training (i.e. increasing performance) effects. In this paper, we reanalyze 3 datasets from 4 studies for changes in radiologist performance during a reading session. Studies feature 8-20 radiologists reading and assessing 27-60 cases in single, uninterrupted sessions. As the studies were not designed for this analysis, study setups range from bone fractures to mammograms and randomization varies between studies. Thus, they are analyzed separately using mixed-effects models. There is some indication that, as time goes on, specificity increases (shown with p<0.05 for 2 out of 3 datasets, no significant difference for the other) while sensitivity may also increase (p<0.05 for 1 out of 3 datasets). The difficulty of 'normal' (healthy / non-malignant) and 'abnormal' (unhealthy / malignant) cases differs (p<0.05 for 3 out of 3 datasets) and the reader's experience may also be relevant (p<0.05 for 1 out of 3 datasets). These results suggest that careful planning of breaks and session length may help optimize reader performance. Note that the overall results are still inconclusive and a targeted study to investigate fatigue and training effects within a reading session is recommended.

Original languageEnglish (US)
Title of host publicationProceedings of SPIE - The International Society for Optical Engineering
Volume8673
DOIs
StatePublished - 2013
EventSPIE Medical Imaging Symposium 2013: Image Perception, Observer Performance, and Technology Assessment - Lake Buena Vista, FL, United States
Duration: Feb 10 2013Feb 11 2013

Other

OtherSPIE Medical Imaging Symposium 2013: Image Perception, Observer Performance, and Technology Assessment
CountryUnited States
CityLake Buena Vista, FL
Period2/10/132/11/13

Fingerprint

Fatigue of materials
Fatigue
readers
Bone
education
Planning
Mixed Effects Model
Mammogram
reviewing
Randomisation
bones
Specificity
planning
indication
Optimise
Vary
sensitivity
Range of data
Training

Keywords

  • Fatigue
  • Observer performance evaluation
  • Radiologist
  • Training

ASJC Scopus subject areas

  • Applied Mathematics
  • Computer Science Applications
  • Electrical and Electronic Engineering
  • Electronic, Optical and Magnetic Materials
  • Condensed Matter Physics

Cite this

Elze, M. C., Taylor-Phillips, S., Mello-Thoms, C., Krupinski, E. A., Gale, A. G., & Clarke, A. (2013). The variation of radiologists' performance over the course of a reading session. In Proceedings of SPIE - The International Society for Optical Engineering (Vol. 8673). [867310] https://doi.org/10.1117/12.2007872

The variation of radiologists' performance over the course of a reading session. / Elze, Markus C.; Taylor-Phillips, Sian; Mello-Thoms, Claudia; Krupinski, Elizabeth A; Gale, Alastair G.; Clarke, Aileen.

Proceedings of SPIE - The International Society for Optical Engineering. Vol. 8673 2013. 867310.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Elze, MC, Taylor-Phillips, S, Mello-Thoms, C, Krupinski, EA, Gale, AG & Clarke, A 2013, The variation of radiologists' performance over the course of a reading session. in Proceedings of SPIE - The International Society for Optical Engineering. vol. 8673, 867310, SPIE Medical Imaging Symposium 2013: Image Perception, Observer Performance, and Technology Assessment, Lake Buena Vista, FL, United States, 2/10/13. https://doi.org/10.1117/12.2007872
Elze MC, Taylor-Phillips S, Mello-Thoms C, Krupinski EA, Gale AG, Clarke A. The variation of radiologists' performance over the course of a reading session. In Proceedings of SPIE - The International Society for Optical Engineering. Vol. 8673. 2013. 867310 https://doi.org/10.1117/12.2007872
Elze, Markus C. ; Taylor-Phillips, Sian ; Mello-Thoms, Claudia ; Krupinski, Elizabeth A ; Gale, Alastair G. ; Clarke, Aileen. / The variation of radiologists' performance over the course of a reading session. Proceedings of SPIE - The International Society for Optical Engineering. Vol. 8673 2013.
@inproceedings{8dfe71f65112431cbd787d79fb71c1d6,
title = "The variation of radiologists' performance over the course of a reading session",
abstract = "The radiologist's task of reviewing many cases successively is highly repetitive and requires a high level of concentration. Fatigue effects have, for example, been shown in studies comparing performance at different times of day. However, little is known about changes in performance during an individual reading session. During a session reading an enriched case set, performance may be affected by both fatigue (i.e. decreasing performance) and training (i.e. increasing performance) effects. In this paper, we reanalyze 3 datasets from 4 studies for changes in radiologist performance during a reading session. Studies feature 8-20 radiologists reading and assessing 27-60 cases in single, uninterrupted sessions. As the studies were not designed for this analysis, study setups range from bone fractures to mammograms and randomization varies between studies. Thus, they are analyzed separately using mixed-effects models. There is some indication that, as time goes on, specificity increases (shown with p<0.05 for 2 out of 3 datasets, no significant difference for the other) while sensitivity may also increase (p<0.05 for 1 out of 3 datasets). The difficulty of 'normal' (healthy / non-malignant) and 'abnormal' (unhealthy / malignant) cases differs (p<0.05 for 3 out of 3 datasets) and the reader's experience may also be relevant (p<0.05 for 1 out of 3 datasets). These results suggest that careful planning of breaks and session length may help optimize reader performance. Note that the overall results are still inconclusive and a targeted study to investigate fatigue and training effects within a reading session is recommended.",
keywords = "Fatigue, Observer performance evaluation, Radiologist, Training",
author = "Elze, {Markus C.} and Sian Taylor-Phillips and Claudia Mello-Thoms and Krupinski, {Elizabeth A} and Gale, {Alastair G.} and Aileen Clarke",
year = "2013",
doi = "10.1117/12.2007872",
language = "English (US)",
isbn = "9780819494474",
volume = "8673",
booktitle = "Proceedings of SPIE - The International Society for Optical Engineering",

}

TY - GEN

T1 - The variation of radiologists' performance over the course of a reading session

AU - Elze, Markus C.

AU - Taylor-Phillips, Sian

AU - Mello-Thoms, Claudia

AU - Krupinski, Elizabeth A

AU - Gale, Alastair G.

AU - Clarke, Aileen

PY - 2013

Y1 - 2013

N2 - The radiologist's task of reviewing many cases successively is highly repetitive and requires a high level of concentration. Fatigue effects have, for example, been shown in studies comparing performance at different times of day. However, little is known about changes in performance during an individual reading session. During a session reading an enriched case set, performance may be affected by both fatigue (i.e. decreasing performance) and training (i.e. increasing performance) effects. In this paper, we reanalyze 3 datasets from 4 studies for changes in radiologist performance during a reading session. Studies feature 8-20 radiologists reading and assessing 27-60 cases in single, uninterrupted sessions. As the studies were not designed for this analysis, study setups range from bone fractures to mammograms and randomization varies between studies. Thus, they are analyzed separately using mixed-effects models. There is some indication that, as time goes on, specificity increases (shown with p<0.05 for 2 out of 3 datasets, no significant difference for the other) while sensitivity may also increase (p<0.05 for 1 out of 3 datasets). The difficulty of 'normal' (healthy / non-malignant) and 'abnormal' (unhealthy / malignant) cases differs (p<0.05 for 3 out of 3 datasets) and the reader's experience may also be relevant (p<0.05 for 1 out of 3 datasets). These results suggest that careful planning of breaks and session length may help optimize reader performance. Note that the overall results are still inconclusive and a targeted study to investigate fatigue and training effects within a reading session is recommended.

AB - The radiologist's task of reviewing many cases successively is highly repetitive and requires a high level of concentration. Fatigue effects have, for example, been shown in studies comparing performance at different times of day. However, little is known about changes in performance during an individual reading session. During a session reading an enriched case set, performance may be affected by both fatigue (i.e. decreasing performance) and training (i.e. increasing performance) effects. In this paper, we reanalyze 3 datasets from 4 studies for changes in radiologist performance during a reading session. Studies feature 8-20 radiologists reading and assessing 27-60 cases in single, uninterrupted sessions. As the studies were not designed for this analysis, study setups range from bone fractures to mammograms and randomization varies between studies. Thus, they are analyzed separately using mixed-effects models. There is some indication that, as time goes on, specificity increases (shown with p<0.05 for 2 out of 3 datasets, no significant difference for the other) while sensitivity may also increase (p<0.05 for 1 out of 3 datasets). The difficulty of 'normal' (healthy / non-malignant) and 'abnormal' (unhealthy / malignant) cases differs (p<0.05 for 3 out of 3 datasets) and the reader's experience may also be relevant (p<0.05 for 1 out of 3 datasets). These results suggest that careful planning of breaks and session length may help optimize reader performance. Note that the overall results are still inconclusive and a targeted study to investigate fatigue and training effects within a reading session is recommended.

KW - Fatigue

KW - Observer performance evaluation

KW - Radiologist

KW - Training

UR - http://www.scopus.com/inward/record.url?scp=84878776647&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84878776647&partnerID=8YFLogxK

U2 - 10.1117/12.2007872

DO - 10.1117/12.2007872

M3 - Conference contribution

AN - SCOPUS:84878776647

SN - 9780819494474

VL - 8673

BT - Proceedings of SPIE - The International Society for Optical Engineering

ER -