Monitoring Breast Cancer Response to Neoadjuvant Chemotherapy Using Probability Maps Derived from Quantitative Ultrasound Parametric Images.

OBJECTIVE: Neoadjuvant chemotherapy (NAC) is widely used in the treatment of breast cancer. However, to date, there are no fully reliable, non-invasive methods for monitoring NAC. In this article, we propose a new method for classifying NAC-responsive and unresponsive tumors using quantitative ultrasound. METHODS: The study used ultrasound data collected from breast tumors treated with NAC. The proposed method is based on the hypothesis that areas that characterize the effect of therapy particularly well can be found. For this purpose, parametric images of texture features calculated from tumor images were converted into NAC response probability maps, and areas with a probability above 0.5 were used for classification. RESULTS: The results obtained after the third cycle of NAC show that the classification of tumors using the traditional method (area under the ROC curve AUC = 0.81 - 0.88) can be significantly improved thanks to the proposed new approach (AUC = 0.84-0.94). This improvement is achieved over a wide range of cutoff values (0.2-0.7), and the probability maps obtained from different quantitative parameters correlate well. CONCLUSION: The results suggest that there are tumor areas that are particularly well suited to assessing response to NAC. SIGNIFICANCE: The proposed approach to monitoring the effects of NAC not only leads to a better classification of responses, but also may contribute to a better understanding of the microstructure of neoplastic tumors observed in an ultrasound examination.

Prediction of response to neoadjuvant chemotherapy in breast cancer with recurrent neural networks and raw ultrasound signals.

Objective. Prediction of the response to neoadjuvant chemotherapy (NAC) in breast cancer is important for patient outcomes. In this work, we propose a deep learning based approach to NAC response prediction in ultrasound (US) imaging.Approach.We develop recurrent neural networks that can process serial US imaging data to predict chemotherapy outcomes. We present models that can process either raw radio-frequency (RF) US data or regular US images. The proposed approach is evaluated based on 204 sequences of US data from 51 breast cancers. Each sequence included US data collected before the chemotherapy and after each subsequent dose, up to the 4th course. We investigate three pre-trained convolutional neural networks (CNNs) as back-bone feature extractors for the recurrent network. The CNNs were pre-trained using raw US RF data, US b-mode images and RGB images from the ImageNet dataset. The first two networks were developed using US data collected from malignant and benign breast masses.Main results. For the pre-treatment data, the better performing network, with back-bone CNN pre-trained on US images, achieved area under the receiver operating curve (AUC) of 0.81 (±0.04). Performance of the recurrent networks improved with each course of the chemotherapy. For the 4th course, the better performing model, based on the CNN pre-trained with RGB images, achieved AUC value of 0.93 (±0.03). Statistical analysis based on the DeLong test presented that there were no significant differences in AUC values between the pre-trained networks at each stage of the chemotherapy (p-values > 0.05).Significance. Our study demonstrates the feasibility of using recurrent neural networks for the NAC response prediction in breast cancer US.

Explaining a Deep Learning Based Breast Ultrasound Image Classifier with Saliency Maps.

Aim of the study: Deep neural networks have achieved good performance in breast mass classification in ultrasound imaging. However, their usage in clinical practice is still limited due to the lack of explainability of decisions conducted by the networks. In this study, to address the explainability problem, we generated saliency maps indicating ultrasound image regions important for the network's classification decisions. Material and methods: Ultrasound images were collected from 272 breast masses, including 123 malignant and 149 benign. Transfer learning was applied to develop a deep network for breast mass classification. Next, the class activation mapping technique was used to generate saliency maps for each image. Breast mass images were divided into three regions: the breast mass region, the peritumoral region surrounding the breast mass, and the region below the breast mass. The pointing game metric was used to quantitatively assess the overlap between the saliency maps and the three selected US image regions. Results: Deep learning classifier achieved the area under the receiver operating characteristic curve, accuracy, sensitivity, and specificity of 0.887, 0.835, 0.801, and 0.868, respectively. In the case of the correctly classified test US images, analysis of the saliency maps revealed that the decisions of the network could be associated with the three selected regions in 71% of cases. Conclusions: Our study is an important step toward better understanding of deep learning models developed for breast mass diagnosis. We demonstrated that the decisions made by the network can be related to the appearance of certain tissue regions in breast mass US images.

Unsupervised deep learning based approach to temperature monitoring in focused ultrasound treatment.

Temperature monitoring in ultrasound (US) imaging is important for various medical treatments, such as high-intensity focused US (HIFU) therapy or hyperthermia. In this work, we present a deep learning based approach to temperature monitoring based on radio-frequency (RF) US data. We used Siamese neural networks in an unsupervised way to spatially compare RF data collected at different time points of the heating process. The Siamese model consisted of two identical networks initially trained on a large set of simulated RF data to assess tissue backscattering properties. To illustrate our approach, we experimented with a tissue-mimicking phantom and an ex-vivo tissue sample, which were both heated with a HIFU transducer. During the experiments, we collected RF data with a regular US scanner. To determine spatiotemporal variations in temperature distribution within the samples, we extracted small 2D patches of RF data and compared them with the Siamese network. Our method achieved good performance in determining the spatiotemporal distribution of temperature during heating. Compared with the temperature monitoring based on the change in radio-frequency signal backscattered energy parameter, our method provided more smooth spatial parametric maps and did not generate ripple artifacts. The proposed approach, when fully developed, might be used for US based temperature monitoring of tissues.

Joint segmentation and classification of breast masses based on ultrasound radio-frequency data and convolutional neural networks.

In this paper, we propose a novel deep learning method for joint classification and segmentation of breast masses based on radio-frequency (RF) ultrasound (US) data. In comparison to commonly used classification and segmentation techniques, utilizing B-mode US images, we train the network with RF data (data before envelope detection and dynamic compression), which are considered to include more information on tissue's physical properties than standard B-mode US images. Our multi-task network, based on the Y-Net architecture, can effectively process large matrices of RF data by mixing 1D and 2D convolutional filters. We use data collected from 273 breast masses to compare the performance of networks trained with RF data and US images. The multi-task model developed based on the RF data achieved good classification performance, with area under the receiver operating characteristic curve (AUC) of 0.90. The network based on the US images achieved AUC of 0.87. In the case of the segmentation, we obtained mean Dice scores of 0.64 and 0.60 for the approaches utilizing US images and RF data, respectively. Moreover, the interpretability of the networks was studied using class activation mapping technique and by filter weights visualizations.

Assessment of breast cancer response to neoadjuvant chemotherapy based on ultrasound backscattering envelope statistics.

PURPOSE: Neo-adjuvant chemotherapy (NAC) is used in breast cancer before tumor surgery to reduce the size of the tumor and the risk of spreading. Monitoring the effects of NAC is important because in a number of cases the response to therapy is poor and requires a change in treatment. A new method that uses quantitative ultrasound to assess tumor response to NAC has been presented. The aim was to detect NAC unresponsive tumors at an early stage of treatment. METHODS: The method assumes that ultrasound scattering is different for responsive and nonresponsive tumors. The assessment of the NAC effects was based on the differences between the histograms of the ultrasound echo amplitude recorded from the tumor after each NAC dose and from the tissue phantom, estimated using the Kolmogorov-Smirnov statistics (KSS) and the symmetrical Kullback-Leibler divergence (KLD). After therapy, tumors were resected and histopathologically evaluated. The percentage of residual malignant cells was determined and was the basis for assessing the tumor response. The data set included ultrasound data obtained from 37 tumors. The performance of the methods was assessed by means of the area under the receiver operating characteristic curve (AUC). RESULTS: For responding tumors, a decrease in the mean KLD and KSS values was observed after subsequent doses of NAC. In nonresponding tumors, the KLD was higher and did not change in subsequent NAC courses. Classification based on the KSS or KLD parameters allowed to detect tumors not responding to NAC after the first dose of the drug, with AUC equal 0.83 ± $\pm$ 0.06 and 0.84 ± $\pm$ 0.07, respectively. After the third dose, the AUC increased to 0.90 ± $\pm$ 0.05 and 0.91 ± $\pm$ 0.04, respectively. CONCLUSIONS: The results indicate the potential usefulness of the proposed parameters in assessing the effectiveness of the NAC and early detection of nonresponding cases.

Quantitative Assessment of the Echogenicity of a Breast Tumor Predicts the Response to Neoadjuvant Chemotherapy.

The aim of the study was to improve monitoring the treatment response in breast cancer patients undergoing neoadjuvant chemotherapy (NAC). The IRB approved this prospective study. Ultrasound examinations were performed prior to treatment and 7 days after four consecutive NAC cycles. Residual malignant cell (RMC) measurement at surgery was the standard of reference. Alteration in B-mode ultrasound (tumor echogenicity and volume) and the Kullback-Leibler divergence (kld), as a quantitative measure of amplitude difference, were used. Correlations of these parameters with RMC were assessed and Receiver Operating Characteristic curve (ROC) analysis was performed. Thirty-nine patients (mean age 57 y.) with 50 tumors were included. There was a significant correlation between RMC and changes in quantitative parameters (KLD) after the second, third and fourth course of NAC, and alteration in echogenicity after the third and fourth course. Multivariate analysis of the echogenicity and KLD after the third NAC course revealed a sensitivity of 91%, specificity of 92%, PPV = 77%, NPV = 97%, accuracy = 91%, and AUC of 0.92 for non-responding tumors (RMC ≥ 70%). In conclusion, monitoring the echogenicity and KLD parameters made it possible to accurately predict the treatment response from the second course of NAC.

Aortic root movement correlation with the function of the left ventricle.

Echocardiographic assessment of systolic and diastolic function of the heart is often limited by image quality. However, the aortic root is well visualized in most patients. We hypothesize that the aortic root motion may correlate with the systolic and diastolic function of the left ventricle of the heart. Data obtained from 101 healthy volunteers (mean age 46.6 ± 12.4) was used in the study. The data contained sequences of standard two-dimensional (2D) echocardiographic B-mode (brightness mode, classical ultrasound grayscale presentation) images corresponding to single cardiac cycles. They also included sets of standard echocardiographic Doppler parameters of the left ventricular systolic and diastolic function. For each B-mode image sequence, the aortic root was tracked with use of a correlation tracking algorithm and systolic and diastolic values of traveled distances and velocities were determined. The aortic root motion parameters were correlated with the standard Doppler parameters used for the assessment of LV function. The aortic root diastolic distance (ARDD) mean value was 1.66 ± 0.26 cm and showed significant, moderate correlation (r up to 0.59, p < 0.0001) with selected left ventricular diastolic Doppler parameters. The aortic root maximal diastolic velocity (ARDV) was 10.8 ± 2.4 cm/s and also correlated (r up to 0.51, p < 0.0001) with some left ventricular diastolic Doppler parameters. The aortic root systolic distance (ARSD) was 1.63 ± 0.19 cm and showed no significant moderate correlation (all r values < 0.40). The aortic root maximal systolic velocity (ARSV) was 9.2 ± 1.6 cm/s and correlated in moderate range only with peak systolic velocity of medial mitral annulus (r = 0.44, p < 0.0001). Based on these results, we conclude, that in healthy subjects, aortic root motion parameters correlate significantly with established measurements of left ventricular function. Aortic root motion parameters can be especially useful in patients with low ultrasound image quality precluding usage of typical LV function parameters.

Early Prediction of Response to Neoadjuvant Chemotherapy in Breast Cancer Sonography Using Siamese Convolutional Neural Networks.

Early prediction of response to neoadjuvant chemotherapy (NAC) in breast cancer is crucial for guiding therapy decisions. In this work, we propose a deep learning based approach for the early NAC response prediction in ultrasound (US) imaging. We used transfer learning with deep convolutional neural networks (CNNs) to develop the response prediction models. The usefulness of two transfer learning techniques was examined. First, a CNN pre-trained on the ImageNet dataset was utilized. Second, we applied double transfer learning, the CNN pre-trained on the ImageNet dataset was additionally fine-tuned with breast mass US images to differentiate malignant and benign lesions. Two prediction tasks were investigated. First, a L1 regularized logistic regression prediction model was developed based on generic neural features extracted from US images collected before the chemotherapy (a priori prediction). Second, Siamese CNNs were used to quantify differences between US images collected before the treatment and after the first and second course of NAC. The proposed methods were evaluated using US data collected from 39 tumors. The better performing deep learning models achieved areas under the receiver operating characteristic curve of 0.797 and 0.847 in the case of the a priori prediction and the Siamese model, respectively. The proposed approach was compared with a method based on handcrafted morphological features. Our study presents the feasibility of using transfer learning with CNNs for the NAC response prediction in US imaging.

Breast-lesions characterization using Quantitative Ultrasound features of peritumoral tissue.

The presented studies evaluate for the first time the efficiency of tumour classification based on the quantitative analysis of ultrasound data originating from the tissue surrounding the tumour. 116 patients took part in the study after qualifying for biopsy due to suspicious breast changes. The RF signals collected from the tumour and tumour-surroundings were processed to determine quantitative measures consisting of Nakagami distribution shape parameter, entropy, and texture parameters. The utility of parameters for the classification of benign and malignant lesions was assessed in relation to the results of histopathology. The best multi-parametric classifier reached an AUC of 0.92 and of 0.83 for outer and intra-tumour data, respectively. A classifier composed of two types of parameters, parameters based on signals scattered in the tumour and in the surrounding tissue, allowed the classification of breast changes with sensitivity of 93%, specificity of 88%, and AUC of 0.94. Among the 4095 multi-parameter classifiers tested, only in eight cases the result of classification based on data from the surrounding tumour tissue was worse than when using tumour data. The presented results indicate the high usefulness of QUS analysis of echoes from the tissue surrounding the tumour in the classification of breast lesions.

Monitoring breast cancer response to neoadjuvant chemotherapy with ultrasound signal statistics and integrated backscatter.

BACKGROUND: Neoadjuvant chemotherapy (NAC) is used in patients with breast cancer to reduce tumor focus, metastatic risk, and patient mortality. Monitoring NAC effects is necessary to capture resistant patients and stop or change treatment. The existing methods for evaluating NAC results have some limitations. The aim of this study was to assess the tumor response at an early stage, after the first doses of the NAC, based on the variability of the backscattered ultrasound energy, and backscatter statistics. The backscatter statistics has not previously been used to monitor NAC effects. METHODS: The B-mode ultrasound images and raw radio frequency data from breast tumors were obtained using an ultrasound scanner before chemotherapy and 1 week after each NAC cycle. The study included twenty-four malignant breast cancers diagnosed in sixteen patients and qualified for neoadjuvant treatment before surgery. The shape parameter of the homodyned K distribution and integrated backscatter, along with the tumor size in the longest dimension, were determined based on ultrasound data and used as markers for NAC response. Cancer tumors were assigned to responding and non-responding groups, according to histopathological evaluation, which was a reference in assessing the utility of markers. Statistical analysis was performed to rate the ability of markers to predict the final NAC response based on data obtained after subsequent therapeutic doses. RESULTS: Statistically significant differences (p<0.05) between groups were obtained after 2, 3, 4, and 5 doses of NAC for quantitative ultrasound markers and after 5 doses for the assessment based on maximum tumor dimension. Statistical analysis showed that, after the second and third NAC courses the classification based on integrated backscatter marker was characterized by an AUC of 0.69 and 0.82, respectively. The introduction of the second quantitative marker describing the statistical properties of scattering increased the corresponding AUC values to 0.82 and 0.91. CONCLUSIONS: Quantitative ultrasound information can characterize the tumor's pathological response better and at an earlier stage of therapy than the assessment of the reduction of its dimensions. The introduction of statistical parameters of ultrasonic backscatter to monitor the effects of chemotherapy can increase the effectiveness of monitoring and contribute to a better personalization of NAC therapy.

Ultrasound echogenicity reveals the response of breast cancer to chemotherapy.

PURPOSE: To evaluate the ultrasound (US) response in patients with breast cancer (BC) during neoadjuvant chemotherapy (NAC). METHODS: Prospective US analysis was performed on 19 malignant tumors prior to NAC treatment and 7 days after each first four courses of NAC in 13 patients (median age = 57 years). Echogenicity, size, vascularity, and sonoelastography were measured and compared with posttreatment scores of residual cancers burden. RESULTS: Changes in the echogenicity of tumors after 3 courses of NAC had the most statistically strong correlation with the percentage of residual malignant cells used in histopathology to assess the response to treatment (odds ratio = 60, p < 0.05). Changes in lesion size and elasticity were also significant (p < 0.05). CONCLUSIONS: There is a statistically significant relationship between breast tumors' echogenicity in US, neoplasm size, and stiffness and the response to NAC. In particular, our results show that the change in tumor echogenicity could predict a pathological response with satisfactory accuracy and may be considered in NAC monitoring.

Tissue Attenuation Estimation by Mean Frequency Downshift and Bandwidth Limitation.

Attenuation of ultrasound in tissue can be estimated from the propagating pulse center frequency downshift. This method assumes that the envelope of the emitted pulse can be approximated by a Gaussian function and that the attenuation linearly depends on frequency. The resulting downshift of the mean frequency depends not only on attenuation but also on pulse bandwidth and propagation distance. This kind of approach is valid for narrowband pulses and shallow penetration depth. However, for short pulses and deep penetration, the frequency downshift is rather large and the received spectra are modified by the limited bandwidth of the receiving system. In this paper, the modified formula modeling the mean frequency of backscattered echoes is presented. The equation takes into account the limitation of the bandwidth due to bandpass filtration of the received echoes. This approach was applied to simulate the variation of the mean frequency of the pulse propagating for both weakly and strongly attenuating media and for narrowband and wideband pulses. The behavior of both the standard and modified estimates of attenuation has been validated using RF data from a tissue-mimicking phantom. The ultrasound attenuation of the phantom, determined with a corrected equation, was close to its true value, while the result obtained using the original formula was lower by as much as 50% at a depth of 8 cm.

Sound fields for coded excitations in water and tissue: experimental approach.

Coded ultrasonography is intensively studied in many laboratories due to its remarkable properties, particularly increased penetration depth and signal-to-noise ratio (SNR). However, no data on the spatial behavior of the pressure field generated by coded bursts transmissions in the tissue were yet reported. This paper reports the results of investigations of the field structure in water, in degassed beef liver and in pork tissue using four different excitations signals, two and 16 periods sine bursts and sinusoidal sequences with phase modulation using 13-bits Barker code and 16-bits Golay complementary codes. The results of measured pressure field distributions before and after compression were compared with those recorded using short pulse excitation.