Deep learning-based algorithm improved radiologists' performance in bone metastases detection on CT.

OBJECTIVES: To develop and evaluate a deep learning-based algorithm (DLA) for automatic detection of bone metastases on CT. METHODS: This retrospective study included CT scans acquired at a single institution between 2009 and 2019. Positive scans with bone metastases and negative scans without bone metastasis were collected to train the DLA. Another 50 positive and 50 negative scans were collected separately from the training dataset and were divided into validation and test datasets at a 2:3 ratio. The clinical efficacy of the DLA was evaluated in an observer study with board-certified radiologists. Jackknife alternative free-response receiver operating characteristic analysis was used to evaluate observer performance. RESULTS: A total of 269 positive scans including 1375 bone metastases and 463 negative scans were collected for the training dataset. The number of lesions identified in the validation and test datasets was 49 and 75, respectively. The DLA achieved a sensitivity of 89.8% (44 of 49) with 0.775 false positives per case for the validation dataset and 82.7% (62 of 75) with 0.617 false positives per case for the test dataset. With the DLA, the overall performance of nine radiologists with reference to the weighted alternative free-response receiver operating characteristic figure of merit improved from 0.746 to 0.899 (p < .001). Furthermore, the mean interpretation time per case decreased from 168 to 85 s (p = .004). CONCLUSION: With the aid of the algorithm, the overall performance of radiologists in bone metastases detection improved, and the interpretation time decreased at the same time. KEY POINTS: • A deep learning-based algorithm for automatic detection of bone metastases on CT was developed. • In the observer study, overall performance of radiologists in bone metastases detection improved significantly with the aid of the algorithm. • Radiologists' interpretation time decreased at the same time.

Temporal subtraction CT with nonrigid image registration improves detection of bone metastases by radiologists: results of a large-scale observer study.

To determine whether temporal subtraction (TS) CT obtained with non-rigid image registration improves detection of various bone metastases during serial clinical follow-up examinations by numerous radiologists. Six board-certified radiologists retrospectively scrutinized CT images for patients with history of malignancy sequentially. These radiologists selected 50 positive and 50 negative subjects with and without bone metastases, respectively. Furthermore, for each subject, they selected a pair of previous and current CT images satisfying predefined criteria by consensus. Previous images were non-rigidly transformed to match current images and subtracted from current images to automatically generate TS images. Subsequently, 18 radiologists independently interpreted the 100 CT image pairs to identify bone metastases, both without and with TS images, with each interpretation separated from the other by an interval of at least 30 days. Jackknife free-response receiver operating characteristics (JAFROC) analysis was conducted to assess observer performance. Compared with interpretation without TS images, interpretation with TS images was associated with a significantly higher mean figure of merit (0.710 vs. 0.658; JAFROC analysis, P = 0.0027). Mean sensitivity at lesion-based was significantly higher for interpretation with TS compared with that without TS (46.1% vs. 33.9%; P = 0.003). Mean false positive count per subject was also significantly higher for interpretation with TS than for that without TS (0.28 vs. 0.15; P < 0.001). At the subject-based, mean sensitivity was significantly higher for interpretation with TS images than that without TS images (73.2% vs. 65.4%; P = 0.003). There was no significant difference in mean specificity (0.93 vs. 0.95; P = 0.083). TS significantly improved overall performance in the detection of various bone metastases.

Adaptive Voxel Matching for Temporal CT Subtraction.

Temporal subtraction (TS) technique calculates a subtraction image between a pair of registered images acquired from the same patient at different times. Previous studies have shown that TS is effective for visualizing pathological changes over time; therefore, TS should be a useful tool for radiologists. However, artifacts caused by partial volume effects degrade the quality of thick-slice subtraction images, even with accurate image registration. Here, we propose a subtraction method for reducing artifacts in thick-slice images and discuss its implementation in high-speed processing. The proposed method is based on voxel matching, which reduces artifacts by considering gaps in discretized positions of two images in subtraction calculations. There are two different features between the proposed method and conventional voxel matching: (1) the size of a searching region to reduce artifacts is determined based on discretized position gaps between images and (2) the searching region is set on both images for symmetrical subtraction. The proposed method is implemented by adopting an accelerated subtraction calculation method that exploit the nature of liner interpolation for calculating the signal value at a point among discretized positions. We quantitatively evaluated the proposed method using synthetic data and qualitatively using clinical data interpreted by radiologists. The evaluation showed that the proposed method was superior to conventional methods. Moreover, the processing speed using the proposed method was almost unchanged from that of the conventional methods. The results indicate that the proposed method can improve the quality of subtraction images acquired from thick-slice images.

Temporal subtraction of computed tomography images improves detectability of bone metastases by radiology residents.

OBJECTIVE: Temporal subtraction of CT (TS) images improves detection of newly developed bone metastases (BM). We sought to determine whether TS improves detection of BM by radiology residents as well. METHODS: We performed an observer study using a previously reported dataset, consisting of 60 oncology patients, each with previous and current CT images. TS images were calculated using in-house software. Four residents independently interpreted twice the 60 sets of CT images, without and with TS. They identified BM by marking suspicious lesions likely to be BM. Lesion-based sensitivity and number of false positives per patient were calculated. Figure-of-merit (FOM) was calculated. Detectability of BM, with and without TS, was compared between radiology residents and board-certified radiologists, as published previously. RESULTS: FOM of residents significantly improved by implementing TS (p value < 0.0001). Lesion-based sensitivity, false positives per patients, and FOM were 40.8%, 0.121, and 0.657, respectively, without TS, and 58.1%, 0.0958, and 0.796, respectively, with TS. These findings were comparable with the previously published values for board-certified radiologists without TS (58.0%, 0.19, and 0.758, respectively). CONCLUSION: The detectability of BM by residents improved markedly by implementing TS and reached that of board-certified radiologists without TS. KEY POINTS: • Detectability of bone metastases on CT by residents improved significantly when using temporal subtraction of CT (TS). • Detections by residents with TS and board-certified radiologists without TS were comparable. • TS is useful for residents as it is for board-certified radiologists.

CT temporal subtraction improves early detection of bone metastases compared to SPECT.

OBJECTIVES: To compare observer performance of detecting bone metastases between bone scintigraphy, including planar scan and single-photon emission computed tomography, and computed tomography (CT) temporal subtraction (TS). METHODS: Data on 60 patients with cancer who had undergone CT (previous and current) and bone scintigraphy were collected. Previous CT images were registered to the current ones by large deformation diffeomorphic metric mapping; the registered previous images were subtracted from the current ones to produce TS. Definitive diagnosis of bone metastases was determined by consensus between two radiologists. Twelve readers independently interpreted the following pairs of examinations: NM-pair, previous and current CTs and bone scintigraphy, and TS-pair, previous and current CTs and TS. The readers assigned likelihood levels to suspected bone metastases for diagnosis. Sensitivity, number of false positives per patient (FPP), and reading time for each pair of examinations were analysed for evaluating observer performance by performing the Wilcoxon signed-rank test. Figure-of-merit (FOM) was calculated using jackknife alternative free-response receiver operating characteristic analysis. RESULTS: The sensitivity of TS was significantly higher than that of bone scintigraphy (54.3% vs. 41.3%, p = 0.006). FPP with TS was significantly higher than that with bone scintigraphy (0.189 vs. 0.0722, p = 0.003). FOM of TS tended to be better than that of bone scintigraphy (0.742 vs. 0.691, p = 0.070). CONCLUSION: Sensitivity of TS in detecting bone metastasis was significantly higher than that of bone scintigraphy, but still limited to 54%. TS might be superior to bone scintigraphy for early detection of bone metastasis. KEY POINTS: • Computed tomography temporal subtraction was helpful in early detection of bone metastases. • Sensitivity for bone metastasis was higher for computed tomography temporal subtraction than for bone scintigraphy. • Figure-of-merit of computed tomography temporal subtraction was better than that of bone scintigraphy.

Detection of suspected brain infarctions on CT can be significantly improved with temporal subtraction images.

OBJECTIVE: To assess whether temporal subtraction (TS) images of brain CT improve the detection of suspected brain infarctions. METHODS: Study protocols were approved by our institutional review board, and informed consent was waived because of the retrospective nature of this study. Forty-two sets of brain CT images of 41 patients, each consisting of a pair of brain CT images scanned at two time points (previous and current) between January 2011 and November 2016, were collected for an observer performance study. The 42 sets consisted of 23 cases with a total of 77 newly developed brain infarcts or hyperdense artery signs confirmed by two radiologists who referred to additional clinical information and 19 negative control cases. To create TS images, the previous images were registered to the current images by partly using a non-rigid registration algorithm and then subtracted. Fourteen radiologists independently interpreted the images to identify the lesions with and without TS images with an interval of over 4 weeks. A figure of merit (FOM) was calculated along with the jackknife alternative free-response receiver-operating characteristic analysis. Sensitivity, number of false positives per case (FPC) and reading time were analyzed by the Wilcoxon signed-rank test. RESULTS: The mean FOM increased from 0.528 to 0.737 with TS images (p < 0.0001). The mean sensitivity and FPC improved from 26.5% and 0.243 to 56.0% and 0.153 (p < 0.0001 and p = 0.239), respectively. The mean reading time was 173 s without TS and 170 s with TS (p = 0.925). CONCLUSION: The detectability of suspected brain infarctions was significantly improved with TS CT images. KEY POINTS: • Although it is established that MRI is superior to CT in the detection of strokes, the first choice of modality for suspected stroke patients is often CT. • An observer performance study with 14 radiologists was performed to evaluate whether temporal subtraction images derived from a non-rigid transformation algorithm can significantly improve the detectability of newly developed brain infarcts on CT. • Temporal subtraction images were shown to significantly improve the detectability of newly developed brain infarcts on CT.

Automatic inference model construction for computer-aided diagnosis of lung nodule: Explanation adequacy, inference accuracy, and experts' knowledge.

We aimed to describe the development of an inference model for computer-aided diagnosis of lung nodules that could provide valid reasoning for any inferences, thereby improving the interpretability and performance of the system. An automatic construction method was used that considered explanation adequacy and inference accuracy. In addition, we evaluated the usefulness of prior experts' (radiologists') knowledge while constructing the models. In total, 179 patients with lung nodules were included and divided into 79 and 100 cases for training and test data, respectively. F-measure and accuracy were used to assess explanation adequacy and inference accuracy, respectively. For F-measure, reasons were defined as proper subsets of Evidence that had a strong influence on the inference result. The inference models were automatically constructed using the Bayesian network and Markov chain Monte Carlo methods, selecting only those models that met the predefined criteria. During model constructions, we examined the effect of including radiologist's knowledge in the initial Bayesian network models. Performance of the best models in terms of F-measure, accuracy, and evaluation metric were as follows: 0.411, 72.0%, and 0.566, respectively, with prior knowledge, and 0.274, 65.0%, and 0.462, respectively, without prior knowledge. The best models with prior knowledge were then subjectively and independently evaluated by two radiologists using a 5-point scale, with 5, 3, and 1 representing beneficial, appropriate, and detrimental, respectively. The average scores by the two radiologists were 3.97 and 3.76 for the test data, indicating that the proposed computer-aided diagnosis system was acceptable to them. In conclusion, the proposed method incorporating radiologists' knowledge could help in eliminating radiologists' distrust of computer-aided diagnosis and improving its performance.

Temporal Subtraction of Serial CT Images with Large Deformation Diffeomorphic Metric Mapping in the Identification of Bone Metastases.

Purpose To determine the improvement of radiologist efficiency and performance in the detection of bone metastases at serial follow-up computed tomography (CT) by using a temporal subtraction (TS) technique based on an advanced nonrigid image registration algorithm. Materials and Methods This retrospective study was approved by the institutional review board, and informed consent was waived. CT image pairs (previous and current scans of the torso) in 60 patients with cancer (primary lesion location: prostate, n = 14; breast, n = 16; lung, n = 20; liver, n = 10) were included. These consisted of 30 positive cases with a total of 65 bone metastases depicted only on current images and confirmed by two radiologists who had access to additional imaging examinations and clinical courses and 30 matched negative control cases (no bone metastases). Previous CT images were semiautomatically registered to current CT images by the algorithm, and TS images were created. Seven radiologists independently interpreted CT image pairs to identify newly developed bone metastases without and with TS images with an interval of at least 30 days. Jackknife free-response receiver operating characteristics (JAFROC) analysis was conducted to assess observer performance. Reading time was recorded, and usefulness was evaluated with subjective scores of 1-5, with 5 being extremely useful and 1 being useless. Significance of these values was tested with the Wilcoxon signed-rank test. Results The subtraction images depicted various types of bone metastases (osteolytic, n = 28; osteoblastic, n = 26; mixed osteolytic and blastic, n = 11) as temporal changes. The average reading time was significantly reduced (384.3 vs 286.8 seconds; Wilcoxon signed rank test, P = .028). The average figure-of-merit value increased from 0.758 to 0.835; however, this difference was not significant (JAFROC analysis, P = .092). The subjective usefulness survey response showed a median score of 5 for use of the technique (range, 3-5). Conclusion TS images obtained from serial CT scans using nonrigid registration successfully depicted newly developed bone metastases and showed promise for their efficient detection. © RSNA, 2017 Online supplemental material is available for this article.

Automated prediction of emphysema visual score using homology-based quantification of low-attenuation lung region.

OBJECTIVE: The purpose of this study was to investigate the relationship between visual score of emphysema and homology-based emphysema quantification (HEQ) and evaluate whether visual score was accurately predicted by machine learning and HEQ. MATERIALS AND METHODS: A total of 115 anonymized computed tomography images from 39 patients were obtained from a public database. Emphysema quantification of these images was performed by measuring the percentage of low-attenuation lung area (LAA%). The following values related to HEQ were obtained: nb0 and nb1. LAA% and HEQ were calculated at various threshold levels ranging from -1000 HU to -700 HU. Spearman's correlation coefficients between emphysema quantification and visual score were calculated at the various threshold levels. Visual score was predicted by machine learning and emphysema quantification (LAA% or HEQ). Random Forest was used as a machine learning algorithm, and accuracy of prediction was evaluated by leave-one-patient-out cross validation. The difference in the accuracy was assessed using McNemar's test. RESULTS: The correlation coefficients between emphysema quantification and visual score were as follows: LAA% (-950 HU), 0.567; LAA% (-910 HU), 0.654; LAA% (-875 HU), 0.704; nb0 (-950 HU), 0.552; nb0 (-910 HU), 0.629; nb0 (-875 HU), 0.473; nb1 (-950 HU), 0.149; nb1 (-910 HU), 0.519; and nb1 (-875 HU), 0.716. The accuracy of prediction was as follows: LAA%, 55.7% and HEQ, 66.1%. The difference in accuracy was statistically significant (p = 0.0290). CONCLUSION: LAA% and HEQ at -875 HU showed a stronger correlation with visual score than those at -910 or -950 HU. HEQ was more useful than LAA% for predicting visual score.

A study of computer-aided diagnosis for pulmonary nodule: comparison between classification accuracies using calculated image features and imaging findings annotated by radiologists.

PURPOSE: In our previous study, we developed a computer-aided diagnosis (CADx) system using imaging findings annotated by radiologists. The system, however, requires radiologists to input many imaging findings. In order to reduce such an interaction of radiologists, we further developed a CADx system using derived imaging findings based on calculated image features, in which the system only requires few user operations. The purpose of this study is to check whether calculated image features (CFT) or derived imaging findings (DFD) can represent information in imaging findings annotated by radiologists (AFD). METHODS: We calculate 2282 image features and derive 39 imaging findings by using information on a nodule position and its type (solid or ground-glass). These image features are categorized into shape features, texture features and imaging findings-specific features. Each imaging finding is derived based on each corresponding classifier using random forest. To check whether CFT or DFD can represent information in AFD, under an assumption that the accuracies of classifiers are the same if information included in input is the same, we constructed classifiers by using various types of information (CTT, DFD and AFD) and compared accuracies on an inferred diagnosis of a nodule. We employ SVM with RBF kernel as classifier to infer a diagnosis name. RESULTS: Accuracies of classifiers using DFD, CFT, AFD and CFT [Formula: see text] AFD were 0.613, 0.577, 0.773 and 0.790, respectively. Concordance rates between DFD and AFD of shape findings, texture findings and surrounding findings were 0.644, 0.871 and 0.768, respectively. CONCLUSIONS: The results suggest that CFT and AFD are similar information and CFT represent only a portion of AFD. Particularly, CFT did not contain shape information in AFD. In order to decrease an interaction of radiologists, a development of a method which overcomes these problems is necessary.

[Investigation of practical method for quality assurance and control of medical displays].

Quality assurance (QA) guidelines for medical display systems in Japan, JESRA X-0093, were published in August 2005 and have been used in many medical fields to maintain image quality on medical displays. This report offers detailed explanations of terms and testing methodologies in the guidelines, taking into account users with little knowledge of display technology. The management grade classifications, luminance meters, test patterns, and evaluation methods for executing the QA are supplementally described based on the technical background of related things. In addition, the validity of the evaluation methods and judgment criteria for uniformity and contrast response tests were examined in some experiments. The experimental results of the contrast response indicated that some cases presented inadequate display contrast even if the contrast responses were set within ± 15% of the standard acceptable range for grade 1. The luminance responses of displays used in two computed tomography systems (CTs) and one magnetic resonance imaging system (MRI) were also measured, and the results indicated that their responses with conventional gamma responses were problematic for comparing images with those of medical displays.