Can a Machine Learn from Radiologists' Visual Search Behaviour and Their Interpretation of Mammograms-a Deep-Learning Study.

Visual search behaviour and the interpretation of mammograms have been studied for errors in breast cancer detection. We aim to ascertain whether machine-learning models can learn about radiologists' attentional level and the interpretation of mammograms. We seek to determine whether these models are practical and feasible for use in training and teaching programmes. Eight radiologists of varying experience levels in reading mammograms reviewed 120 two-view digital mammography cases (59 cancers). Their search behaviour and decisions were captured using a head-mounted eye-tracking device and software allowing them to record their decisions. This information from radiologists was used to build an ensembled machine-learning model using top-down hierarchical deep convolution neural network. Separately, a model to determine type of missed cancer (search, perception or decision-making) was also built. Analysis and comparison of variants of these models using different convolution networks with and without transfer learning were also performed. Our ensembled deep-learning network architecture can be trained to learn about radiologists' attentional level and decisions. High accuracy (95%, p value ≅ 0 [better than dumb/random model]) and high agreement between true and predicted values (kappa = 0.83) in such modelling can be achieved. Transfer learning techniques improve by < 10% with the performance of this model. We also show that spatial convolution neural networks are insufficient in determining the type of missed cancers. Ensembled hierarchical deep convolution machine-learning models are plausible in modelling radiologists' attentional level and their interpretation of mammograms. However, deep convolution networks fail to characterise the type of false-negative decisions.

Modeling visual search behavior of breast radiologists using a deep convolution neural network.

Visual search, the process of detecting and identifying objects using eye movements (saccades) and foveal vision, has been studied for identification of root causes of errors in the interpretation of mammograms. The aim of this study is to model visual search behavior of radiologists and their interpretation of mammograms using deep machine learning approaches. Our model is based on a deep convolutional neural network, a biologically inspired multilayer perceptron that simulates the visual cortex and is reinforced with transfer learning techniques. Eye-tracking data were obtained from eight radiologists (of varying experience levels in reading mammograms) reviewing 120 two-view digital mammography cases (59 cancers), and it has been used to train the model, which was pretrained with the ImageNet dataset for transfer learning. Areas of the mammogram that received direct (foveally fixated), indirect (peripherally fixated), or no (never fixated) visual attention were extracted from radiologists' visual search maps (obtained by a head mounted eye-tracking device). These areas along with the radiologists' assessment (including confidence in the assessment) of the presence of suspected malignancy were used to model: (1) radiologists' decision, (2) radiologists' confidence in such decisions, and (3) the attentional level (i.e., foveal, peripheral, or none) in an area of the mammogram. Our results indicate high accuracy and low misclassification in modeling such behaviors.

The role of digital breast tomosynthesis in the breast assessment clinic: a review.

Mammography has long been considered as the primary technique in breast cancer detection and assessment. Despite low specificity, mammography has been preferred over other contemporary techniques such as magnetic resonance imaging (MRI), computed tomography (CT) and ultrasonography (US) due to superior sensitivity and significant health economic benefits. The development of a new technique, a limited angle cone beam pseudo-three-dimensional tomosynthesis, digital breast tomosynthesis (DBT), has gained momentum. Several preliminary studies and ongoing trials are showing evidence of the benefits of DBT in improving lesion visibility, accuracy of cancer detection and observer performance. This raises the possibility of adoption of DBT in the breast cancer assessment clinic, wherein confirming or dismissing the presence of malignancy (at the potential site identified during screening) is of utmost importance. Identification of suspected malignancy in terms of lesion characteristics and location is also essential in assessment. In this literature review, we evaluate the role of DBT for use in breast cancer assessment and its future in biopsy.

Fixated and Not Fixated Regions of Mammograms: A Higher-Order Statistical Analysis of Visual Search Behavior.

RATIONALE AND OBJECTIVES: Visual search is an inhomogeneous yet efficient sampling process accomplished by the saccades and the central (foveal) vision. Areas that attract the central vision have been studied for errors in interpretation of medical imaging. In this study, we extend existing visual search studies to understand what characterizes areas that receive direct visual attention and elicit a mark by the radiologist (True and False Positive decisions) from those that elicit a mark but were captured by the peripheral vision. We also investigate if there are any differences between these areas and those that are never fixated by radiologists. MATERIALS AND METHODS: Eight radiologists participated in this fully crossed multi-reader multi-case visual search study of digital mammography (DM) involving 120 two-view cases (59 cancers). From these DM images, 3 types of areas, namely Fixated Clusters (FC), Marked Peripherally Fixated Clusters (MPFC) and Never Fixated Clusters (NFC), were extracted and analysed using statistical information theory (in the form of third and fourth-order cumulants and polyspectrum [specifically bispectrum and trispectrum]) in addition to traditional second-order statistics (in the form of power spectrum) and other nonspectral features to characterize these types of areas. RESULTS: Our results suggest that energy profiles of FC, MPFC, and NFC areas are distinct. We found evidence that energy profiles and dwell time of these areas influence radiologists' decisions (and confidence in such decisions). We also noted that foveated areas are selected on the basis of being most informative. CONCLUSION: We show that properties of these areas influence radiologists' decisions and their confidence in the decisions made.