Combiner and HyperCombiner networks: Rules to combine multimodality MR images for prostate cancer localisation.

One of the distinct characteristics of radiologists reading multiparametric prostate MR scans, using reporting systems like PI-RADS v2.1, is to score individual types of MR modalities, including T2-weighted, diffusion-weighted, and dynamic contrast-enhanced, and then combine these image-modality-specific scores using standardised decision rules to predict the likelihood of clinically significant cancer. This work aims to demonstrate that it is feasible for low-dimensional parametric models to model such decision rules in the proposed Combiner networks, without compromising the accuracy of predicting radiologic labels. First, we demonstrate that either a linear mixture model or a nonlinear stacking model is sufficient to model PI-RADS decision rules for localising prostate cancer. Second, parameters of these combining models are proposed as hyperparameters, weighing independent representations of individual image modalities in the Combiner network training, as opposed to end-to-end modality ensemble. A HyperCombiner network is developed to train a single image segmentation network that can be conditioned on these hyperparameters during inference for much-improved efficiency. Experimental results based on 751 cases from 651 patients compare the proposed rule-modelling approaches with other commonly-adopted end-to-end networks, in this downstream application of automating radiologist labelling on multiparametric MR. By acquiring and interpreting the modality combining rules, specifically the linear-weights or odds ratios associated with individual image modalities, three clinical applications are quantitatively presented and contextualised in the prostate cancer segmentation application, including modality availability assessment, importance quantification and rule discovery.

Development of a Three-Dimensional Carotid Ultrasound Image Segmentation Workflow for Improved Efficiency, Reproducibility and Accuracy in Measuring Vessel Wall and Plaque Volume and Thickness.

Automated segmentation of carotid lumen-intima boundary (LIB) and media-adventitia boundary (MAB) by deep convolutional neural networks (CNN) from three-dimensional ultrasound (3DUS) images has made assessment and monitoring of carotid atherosclerosis more efficient than manual segmentation. However, training of CNN still requires manual segmentation of LIB and MAB. Therefore, there is a need to improve the efficiency of manual segmentation and develop strategies to improve segmentation accuracy by the CNN for serial monitoring of carotid atherosclerosis. One strategy to reduce segmentation time is to increase the interslice distance (ISD) between segmented axial slices of a 3DUS image while maintaining the segmentation reliability. We, for the first time, investigated the effect of ISD on the reproducibility of MAB and LIB segmentations. The intra-observer reproducibility of LIB and MAB segmentations at ISDs of 1 mm and 2 mm was not statistically significantly different, whereas the reproducibility at ISD = 3 mm was statistically lower. Therefore, we conclude that segmentation with an ISD of 2 mm provides sufficient reliability for CNN training. We further proposed training the CNN by the baseline images of the entire cohort of patients for automatic segmentation of the follow-up images acquired for the same cohort. We validated that segmentation with this time-based partitioning approach is more accurate than that produced by patient-based partitioning, especially at the carotid bifurcation. This study forms the basis for an efficient, reproducible, and accurate 3DUS workflow for serial monitoring of carotid atherosclerosis useful in risk stratification of cardiovascular events and in evaluating the efficacy of new treatments.

A Dual-Stream Centerline-Guided Network for Segmentation of the Common and Internal Carotid Arteries From 3D Ultrasound Images.

Segmentation of the carotid section encompassing the common carotid artery (CCA), the bifurcation and the internal carotid artery (ICA) from three-dimensional ultrasound (3DUS) is required to measure the vessel wall volume (VWV) and localized vessel-wall-plus-plaque thickness (VWT), shown to be sensitive to treatment effect. We proposed an approach to combine a centerline extraction network (CHG-Net) and a dual-stream centerline-guided network (DSCG-Net) to segment the lumen-intima (LIB) and media-adventitia boundaries (MAB) from 3DUS images. Correct arterial location is essential for successful segmentation of the carotid section encompassing the bifurcation. We addressed this challenge by using the arterial centerline to enhance the localization accuracy of the segmentation network. The CHG-Net was developed to generate a heatmap indicating high probability regions for the centerline location, which was then integrated with the 3DUS image by the DSCG-Net to generate the MAB and LIB. The DSCG-Net includes a scale-based and a spatial attention mechanism to fuse multi-level features extracted by the encoder, and a centerline heatmap reconstruction side-branch connected to the end of the encoder to increase the generalization ability of the network. Experiments involving 224 3DUS volumes produce a Dice similarity coefficient (DSC) of 95.8±1.9% and 92.3±5.4% for CCA MAB and LIB, respectively, and 93.2±4.4% and 89.0±10.0% for ICA MAB and LIB, respectively. Our approach outperformed four state-of-the-art 3D CNN models, even after their performances were boosted by centerline guidance. The efficiency afforded by the framework would allow it to be incorporated into the clinical workflow for improved quantification of plaque change.

Prostate cancer segmentation from MRI by a multistream fusion encoder.

BACKGROUND: Targeted prostate biopsy guided by multiparametric magnetic resonance imaging (mpMRI) detects more clinically significant lesions than conventional systemic biopsy. Lesion segmentation is required for planning MRI-targeted biopsies. The requirement for integrating image features available in T2-weighted and diffusion-weighted images poses a challenge in prostate lesion segmentation from mpMRI. PURPOSE: A flexible and efficient multistream fusion encoder is proposed in this work to facilitate the multiscale fusion of features from multiple imaging streams. A patch-based loss function is introduced to improve the accuracy in segmenting small lesions. METHODS: The proposed multistream encoder fuses features extracted in the three imaging streams at each layer of the network, thereby allowing improved feature maps to propagate downstream and benefit segmentation performance. The fusion is achieved through a spatial attention map generated by optimally weighting the contribution of the convolution outputs from each stream. This design provides flexibility for the network to highlight image modalities according to their relative influence on the segmentation performance. The encoder also performs multiscale integration by highlighting the input feature maps (low-level features) with the spatial attention maps generated from convolution outputs (high-level features). The Dice similarity coefficient (DSC), serving as a cost function, is less sensitive to incorrect segmentation for small lesions. We address this issue by introducing a patch-based loss function that provides an average of the DSCs obtained from local image patches. This local average DSC is equally sensitive to large and small lesions, as the patch-based DSCs associated with small and large lesions have equal weights in this average DSC. RESULTS: The framework was evaluated in 931 sets of images acquired in several clinical studies at two centers in Hong Kong and the United Kingdom. In particular, the training, validation, and test sets contain 615, 144, and 172 sets of images, respectively. The proposed framework outperformed single-stream networks and three recently proposed multistream networks, attaining F1 scores of 82.2 and 87.6% in the lesion and patient levels, respectively. The average inference time for an axial image was 11.8 ms. CONCLUSION: The accuracy and efficiency afforded by the proposed framework would accelerate the MRI interpretation workflow of MRI-targeted biopsy and focal therapies.

Quantification of Local Vessel Wall and Plaque Volume Change for Assessment of Effects of Therapies on Carotid Atherosclerosis Based on 3-D Ultrasound Imaging.

We developed a new method to measure the voxel-based vessel-wall-plus-plaque volume (VWV). In addition to quantifying local thickness change as in the previously introduced vessel-wall-plus-plaque thickness (VWT) metric, voxel-based VWV further considers the circumferential change associated with vascular remodeling. Three-dimensional ultrasound images were acquired at baseline and 1 y afterward. The vessel wall region was divided into small voxels with the voxel-based VWV change (ΔVVol%) computed by taking the percentage volume difference between corresponding voxels in the baseline and follow-up images. A 3-D carotid atlas was developed to allow visualization of the local thickness and circumferential change patterns in the pomegranate versus the placebo groups. A new patient-based biomarker was obtained by computing the mean ΔVVol% over the entire 3-D map for each patient (ΔVVol%¯). ΔVVol%¯ detected a significant difference between patients randomized to pomegranate juice/extract and placebo groups (p = 0.0002). The number of patients required by ΔVVol%¯ to establish statistical significance was approximately a third of that required by the local VWT biomarker. The increased sensitivity afforded by the proposed biomarker improves the cost-effectiveness of clinical studies evaluating new anti-atherosclerotic treatments.

Cascaded Triplanar Autoencoder M-Net for Fully Automatic Segmentation of Left Ventricle Myocardial Scar From Three-Dimensional Late Gadolinium-Enhanced MR Images.

While three-dimensional (3D) late gadolinium-enhanced (LGE) magnetic resonance (MR) imaging provides good conspicuity of small myocardial lesions with short acquisition time, it poses a challenge for image analysis as a large number of axial images are required to be segmented. We developed a fully automatic convolutional neural network (CNN) called cascaded triplanar autoencoder M-Net (CTAEM-Net) to segment myocardial scar from 3D LGE MRI. Two sub-networks were cascaded to segment the left ventricle (LV) myocardium and then the scar within the pre-segmented LV myocardium. Each sub-network contains three autoencoder M-Nets (AEM-Nets) segmenting the axial, sagittal and coronal slices of the 3D LGE MR image, with the final segmentation determined by voting. The AEM-Net integrates three features: (1) multi-scale inputs, (2) deep supervision and (3) multi-tasking. The multi-scale inputs allow consideration of the global and local features in segmentation. Deep supervision provides direct supervision to deeper layers and facilitates CNN convergence. Multi-task learning reduces segmentation overfitting by acquiring additional information from autoencoder reconstruction, a task closely related to segmentation. The framework provides an accuracy of 86.43% and 90.18% for LV myocardium and scar segmentation, respectively, which are the highest among existing methods to our knowledge. The time required for CTAEM-Net to segment LV myocardium and the scar was 49.72 ± 9.69s and 120.25 ± 23.18s per MR volume, respectively. The accuracy and efficiency afforded by CTAEM-Net will make possible future large population studies. The generalizability of the framework was also demonstrated by its competitive performance in two publicly available datasets of different imaging modalities.

Multimodal Image Classification by Multiview Latent Pattern Extraction, Selection, and Correlation.

The large amount of data available in the modern big data era opens new opportunities to expand our knowledge by integrating information from heterogeneous sources. Multiview learning has recently achieved tremendous success in deriving complementary information from multiple data modalities. This article proposes a framework called multiview latent space projection (MVLSP) to integrate features extracted from multiple sources in a discriminative way to facilitate binary and multiclass classifications. Our approach is associated with three innovations. First, most existing multiview learning algorithms promote pairwise consistency between two views and do not have a natural extension to applications with more than two views. MVLSP finds optimum mappings from a common latent space to match the feature space in each of the views. As the matching is performed on a view-by-view basis, the framework can be readily extended to multiview applications. Second, feature selection in the common latent space can be readily achieved by adding a class view, which matches the latent space representations of training samples with their corresponding labels. Then, high-order view correlations are extracted by considering feature-label correlations. Third, a technique is proposed to optimize the integration of different latent patterns based on their correlations. The experimental results on the prostate image dataset demonstrate the effectiveness of the proposed method.

Multilabel Classification With Group-Based Mapping: A Framework With Local Feature Selection and Local Label Correlation.

Multilabel learning, which handles instances associated with multiple labels, has attracted much attention in recent years. Many extant multilabel feature selection methods target global feature selection, which means feature selection weights for each label are shared by all instances. Also, many extant multilabel classification methods exploit global label selection, which means labels correlations are shared by all instances. In real-world objects, however, different subsets of instances may share different feature selection weights and different label correlations. In this article, we propose a novel framework with local feature selection and local label correlation, where we assume instances can be clustered into different groups, and the feature selection weights and label correlations can only be shared by instances in the same group. The proposed framework includes a group-specific feature selection process and a label-specific group selection process. The former process projects instances into different groups by extracting the instance-group correlation. The latter process selects labels for each instance based on its related groups by extracting the group-label correlation. In addition, we also exploit the intergroup correlation. These three kinds of group-based correlations are combined to perform effective multilabel classification. The experimental results on various datasets validate the effectiveness of our approach.

Segmentation of common and internal carotid arteries from 3D ultrasound images based on adaptive triple loss.

PURPOSE: Vessel wall volume (VWV) and localized vessel-wall-plus-plaque thickness (VWT) measured from three-dimensional (3D) ultrasound (US) carotid images are sensitive to anti-atherosclerotic effects of medical/dietary treatments. VWV and VWT measurements require the lumen-intima (LIB) and media-adventitia boundaries (MAB) at the common and internal carotid arteries (CCA and ICA). However, most existing segmentation techniques were capable of segmenting the CCA only. An approach capable of segmenting the MAB and LIB from the CCA and ICA was required to accelerate VWV and VWT quantification. METHODS: Segmentation for CCA and ICA was performed independently using the proposed two-channel U-Net, which was driven by a novel loss function known as the adaptive triple Dice loss (ADTL) function. The training set was augmented by interpolating manual segmentation along the longitudinal direction, thereby taking continuity of the artery into account. A test-time augmentation (TTA) approach was applied, in which segmentation was performed three times based on the input axial images and its flipped versions; the final segmentation was generated by pixel-wise majority voting. RESULTS: Experiments involving 224 3DUS volumes produce a Dice similarity coefficient (DSC) of 95.1% ± 4.1% and 91.6% ± 6.6% for the MAB and LIB, in the CCA, respectively, and 94.2% ± 3.3% and 89.0% ± 8.1% for the MAB and LIB, in the ICA, respectively. TTA and ATDL independently contributed to a statistically significant improvement to all boundaries except the LIB in ICA. CONCLUSIONS: The proposed two-channel U-Net with ADTL and TTA can segment the CCA and ICA accurately and efficiently from the 3DUS volume. Our approach has the potential to accelerate the transition of 3DUS measurements of carotid atherosclerosis to clinical research.

Three-dimensional ultrasound assessment of effects of therapies on carotid atherosclerosis using vessel wall thickness maps.

We present a new method for assessing the effects of therapies on atherosclerosis, by measuring the weighted average of carotid vessel-wall-plus-plaque thickness change (ΔVWT¯Weighted) in 120 patients randomized to pomegranate juice/extract versus placebo. Three-dimensional ultrasound images were acquired at baseline and one year after. Three-dimensional VWT maps were reconstructed and then projected onto a carotid template to obtain two-dimensional VWT maps. Anatomic correspondence on the two-dimensional VWT maps was optimized to reduce misalignment for the same subject and across subjects. A weight was computed at each point on the two-dimensional VWT map to highlight anatomic locations likely to exhibit plaque progression/regression, resulting in ΔVWT¯Weighted for each subject. The weighted average of VWT-Change measured from the two-dimensional VWT maps with correspondence alignment (ΔVWT¯Weighted,MDL) detected a significant difference between the pomegranate and placebo groups (P = 0.008). This method improves the cost-effectiveness of proof-of-concept studies involving new therapies for atherosclerosis.

Segmentation of 3D ultrasound carotid vessel wall using U-Net and segmentation average network.

Segmentation of carotid vessel wall is required in vessel wall volume (VWV) and local vessel-wall-plus-plaque thickness (VWT) quantification of the carotid artery. Manual segmentation of the vessel wall is time-consuming and prone to interobserver variability. In this paper, we proposed a convolutional neural network (CNN) to segment the common carotid artery (CCA) from 3D carotid ultrasound images. The proposed CNN involves three U-Nets that segmented the 3D ultrasound (3DUS) images in the axial, lateral and frontal orientations. The segmentation maps generated by three U-Nets were consolidated by a novel segmentation average network (SAN) we proposed in this paper. The experimental results show that the proposed CNN improved the segmentation accuracies. Compared to only using U-Net alone, the proposed CNN improved the Dice similarity coefficient (DSC) for vessel wall segmentation from 64.8% to 67.5%, the sensitivity from 63.8% to 70.5%, and the area under receiver operator characteristic curve (AUC) from 0.89 to 0.94.

Identification of Retinal Ganglion Cells from ß-III Stained Fluorescent Microscopic Images.

Optic nerve crush in mouse model is widely used for investigating the course following retinal ganglion cell (RGCs) injury. Manual cell counting from ß-III tubulin stained microscopic images has been routinely performed to monitor RGCs after an optic nerve crush injury, but is time-consuming and prone to observer variability. This paper describes an automatic technique for RGC identification. We developed and validated (i) a sensitive cell candidate segmentation scheme and (ii) a classifier that removed false positives while retaining true positives. Two major contributions were made in cell candidate segmentation. First, a homomorphic filter was designed to adjust for the inhomogeneous illumination caused by uneven penetration of ß-III tubulin antibody. Second, the optimal segmentation parameters for cell detection are highly image-specific. To address this issue, we introduced an offline-online parameter tuning approach. Offline tuning optimized model parameters based on training images and online tuning further optimized the parameters at the testing stage without needing access to the ground truth. In the cell identification stage, 31 geometric, statistical and textural features were extracted from each segmented cell candidate, which was subsequently classified as true or false positives by support vector machine. The homomorphic filter and the online parameter tuning approach together increased cell recall by 28%. The entire pipeline attained a recall, precision and coefficient of determination (r2) of 85.3%, 97.1% and 0.994. The availability of the proposed pipeline will allow efficient, accurate and reproducible RGC quantification required for assessing the death/survival of RGCs in disease models.

Longitudinal assessment of carotid plaque texture in three-dimensional ultrasound images based on semi-supervised graph-based dimensionality reduction and feature selection.

With continuous development of therapeutic options for atherosclerosis, image-based biomarkers sensitive to the effect of new interventions are required to be developed for cost-effective clinical evaluation. Although 3D ultrasound measurement of total plaque volume (TPV) showed the efficacy of high-dose statin, more sensitive biomarkers are needed to establish the efficacy of dietary supplements expected to confer a smaller beneficial effect. This study involved 171 subjects who participated in a one-year placebo-controlled trial evaluating the effect of pomegranate. A framework involving a feature selection technique known as discriminative feature selection (DFS) and a semi-supervised graph-based regression (SSGBR) technique was proposed for sensitive detection of plaque textural changes over the trial. 376 textual features of plaques were extracted from 3D ultrasound images acquired at baseline and a follow-up session. A scalar biomarker for each subject were generated by SSGBR based on prominent textural features selected by DFS. The ability of this biomarker for discriminating pomegranate from placebo subjects was quantified by the p-values obtained in Mann-Whitney U test. The discriminative power of SSGBR was compared with global and local dimensionality reduction techniques, including linear discriminant analysis (LDA), maximum margin criterion (MMC) and Laplacian Eigenmap (LE). Only SSGBR (p=4.12×10-6) and normalized LE (p=0.002) detected a difference between the two groups at the 5% significance level. As compared with ΔTPV, SSGBR reduced the sample size required to establish a significant difference by a factor of 60. The application of this framework will substantially reduce the cost incurred in clinical trials.

Area-Preserving Mapping of 3D Carotid Ultrasound Images Using Density-Equalizing Reference Map.

Atherosclerotic plaques are focal and tend to occur at arterial bends and bifurcations. To quantitatively monitor the local changes in the carotid vessel-wall-plus-plaque thickness (VWT) and compare the VWT distributions for different patients or for the same patients at different ultrasound scanning sessions, a mapping technique is required to adjust for the geometric variability of different carotid artery models. In this work, we propose a novel method called density-equalizing reference map (DERM) for mapping 3D carotid surfaces to a standardized 2D carotid template, with an emphasis on preserving the local geometry of the carotid surface by minimizing the local area distortion. The initial map was generated by a previously described arc-length scaling (ALS) mapping method, which projects a 3D carotid surface onto a 2D non-convex L-shaped domain. A smooth and area-preserving flattened map was subsequently constructed by deforming the ALS map using the proposed algorithm that combines the density-equalizing map and the reference map techniques. This combination allows, for the first time, one-to-one mapping from a 3D surface to a standardized non-convex planar domain in an area-preserving manner. Evaluations using 20 carotid surface models show that the proposed method reduced the area distortion of the flattening maps by over 80% as compared to the ALS mapping method. The proposed method is capable of improving the accuracy of area estimation for plaque regions without compromising inter-scan reproducibility.

Three-dimensional ultrasound evaluation of the effects of pomegranate therapy on carotid plaque texture using locality preserving projection.

BACKGROUND AND OBJECTIVE: Dietary supplements are expected to confer a smaller beneficial effect than medical treatments. Therefore, there is a need to develop cost-effective biomarkers that can demonstrate the efficacy of such supplements for carotid atherosclerosis. The aim of this study is to develop such a biomarker based on the changes of 376 plaque textural features measured from 3D ultrasound images. METHODS: Since the number of features (376) was greater than the number of subjects (171) in this study, principal component analysis (PCA) was applied to reduce the dimensionality of feature vectors. To generate a scalar biomarker for each subject, elements in the reduced feature vectors produced by PCA were weighted using locality preserving projections (LPP) to capture essential patterns exhibited locally in the feature space. 96 subjects treated by pomegranate juice and tablets, and 75 subjects receiving placebo-matching juice and tablets were evaluated in this study. The discriminative power of the proposed biomarker was evaluated and compared with existing biomarkers using t-tests. As the cost of a clinical trial is directly related to the number of subjects enrolled, the cost-effectiveness of the proposed biomarker was evaluated by sample size estimation. RESULTS: The proposed biomarker was more able to discriminate plaque changes exhibited by the pomegranate and placebo groups than total plaque volume (TPV) according to the result of t-tests (TPV: p=0.34, Proposed biomarker: p=1.5×10-5). The sample size required by the new biomarker to detect a significant effect was 20 times smaller than that required by TPV. CONCLUSION: With the increase in cost-effectiveness afforded by the proposed biomarker, more proof-of-principle studies for novel treatment options could be performed.

Plaque components segmentation in carotid artery on simultaneous non-contrast angiography and intraplaque hemorrhage imaging using machine learning.

PURPOSE: This study sought to determine the feasibility of using Simultaneous Non-contrast Angiography and intraPlaque Hemorrhage (SNAP) to detect the lipid-rich/necrotic core (LRNC), and develop a machine learning based algorithm to segment plaque components on SNAP images. METHODS: Sixty-eight patients (age: 58±9 years, 24 males) with carotid artery atherosclerotic plaque were imaged on a 3 T MR scanner with both traditional multi-contrast vessel wall MR sequences (TOF, T1W, and T2W) and 3D SNAP sequence. The manual segmentations of carotid plaque components including LRNC, intraplaque hemorrhage (IPH), calcification (CA) and fibrous tissue (FT) on traditional multi-contrast images were used as reference. By utilizing the intensity and morphological information from SNAP, a machine learning based two steps algorithm was developed to firstly identify LRNC (with or without IPH), CA and FT, and then segmented IPH from LRNC. Ten-fold cross-validation was used to evaluate the performance of proposed method. The overall pixel-wise accuracy, the slice-wise sensitivity & specificity & Youden's index, and the Pearson's correlation coefficient of the component area between the proposed method and the manual segmentation were reported. RESULTS: In the first step, all tested classifiers (Naive Bayes (NB), Support Vector Machine (SVM), Random Forest (RF), Gradient Boosting Decision Tree (GBDT) and Artificial Neural Network (ANN)) had overall pixel-wise accuracy higher than 0.88. For RF, GBDT and ANN classifiers, the correlation coefficients of areas were all higher than 0.82 (p < 0.001) for LRNC and 0.79 for CA (p < 0.001), and the Youden's indexes were all higher than 0.79 for LRNC and 0.76 for CA, which were better than that of NB and SVM. In the second step, the overall pixel-wise accuracy was higher than 0.78 for the five classifiers, and RF achieved the highest Youden's index (0.69) with the correlation coefficients as 0.63 (p < 0.001). CONCLUSIONS: The RF is the overall best classifier for our proposed method, and the feasibility of using SNAP to identify plaque components, including LRNC, IPH, CA, and FT has been validated. The proposed segmentation method using a single SNAP sequence might be a promising tool for atherosclerotic plaque components assessment.

Prostate lesion delineation from multiparametric magnetic resonance imaging based on locality alignment discriminant analysis.

PURPOSE: Multiparametric MRI (mpMRI) has shown promise in the detection and localization of prostate cancer foci. Although techniques have been previously introduced to delineate lesions from mpMRI, these techniques were evaluated in datasets with T2 maps available. The generation of T2 map is not included in the clinical prostate mpMRI consensus guidelines; the acquisition of which requires repeated T2-weighted (T2W) scans and would significantly lengthen the scan time currently required for the clinically recommended acquisition protocol, which includes T2W, diffusion-weighted (DW), and dynamic contrast-enhanced (DCE) imaging. The goal of this study is to develop and evaluate an algorithm that provides pixel-accurate lesion delineation from images acquired based on the clinical protocol. METHODS: Twenty-five pixel-based features were extracted from the T2-weighted (T2W), apparent diffusion coefficient (ADC), and dynamic contrast-enhanced (DCE) images. The pixel-wise classification was performed on the reduced space generated by locality alignment discriminant analysis (LADA), a version of linear discriminant analysis (LDA) localized to patches in the feature space. Postprocessing procedures, including the removal of isolated points identified and filling of holes inside detected regions, were performed to improve delineation accuracy. The segmentation result was evaluated against the lesions manually delineated by four expert observers according to the Prostate Imaging-Reporting and Data System (PI-RADS) detection guideline. RESULTS: The LADA-based classifier (60 ± 11%) achieved a higher sensitivity than the LDA-based classifier (51 ± 10%), thereby demonstrating, for the first time, that higher classification performance was attained on the reduced space generated by LADA than by LDA. Further sensitivity improvement (75 ± 14%) was obtained after postprocessing, approaching the sensitivities attained by previous mpMRI lesion delineation studies in which nonclinical T2 maps were available. CONCLUSION: The proposed algorithm delineated lesions accurately and efficiently from images acquired following the clinical protocol. The development of this framework may potentially accelerate the clinical uses of mpMRI in prostate cancer diagnosis and treatment planning.

A self-tuned graph-based framework for localization and grading prostate cancer lesions: An initial evaluation based on multiparametric magnetic resonance imaging.

Multiparametric magnetic resonance imaging (mpMRI) has been established as the state-of-the-art examination for the detection and localization of prostate cancer lesions. Prostate Imaging-Reporting and Data System (PI-RADS) has been established as a scheme to standardize the reporting of mpMRI findings. Although lesion delineation and PI-RADS ratings could be performed manually, human delineation and ratings are subjective and time-consuming. In this article, we developed and validated a self-tuned graph-based model for PI-RADS rating prediction. 34 features were obtained at the pixel level from T2-weighted (T2W), apparent diffusion coefficient (ADC) and dynamic contrast enhanced (DCE) images, from which PI-RADS scores were predicted. Two major innovations were involved in this self-tuned graph-based model. First, graph-based approaches are sensitive to the choice of the edge weight. The proposed model tuned the edge weights automatically based on the structure of the data, thereby obviating empirical edge weight selection. Second, the feature weights were tuned automatically to give heavier weights to features important for PI-RADS rating estimation. The proposed framework was evaluated for its lesion localization performance in mpMRI datasets of 12 patients. In the evaluation, the PI-RADS score distribution map generated by the algorithm and from the observers' ratings were binarized by thresholds of 3 and 4. The sensitivity, specificity and accuracy obtained in these two threshold settings ranged from 65 to 77%, 86 to 93% and 85 to 88% respectively, which are comparable to results obtained in previous studies in which non-clinical T2 maps were available. The proposed algorithm took 10s to estimate the PI-RADS score distribution in an axial image. The efficiency achievable suggests that this technique can be developed into a prostate MR analysis system suitable for clinical use after a thorough validation involving more patients.

Carotid plaque segmentation from three-dimensional ultrasound images by direct three-dimensional sparse field level-set optimization.

Total plaque volume (TPV) measured from 3D carotid ultrasound has been shown to be able to predict cardiovascular events and is sensitive in detecting treatment effects. Manual plaque segmentation was performed in previous studies to quantify TPV, but is tedious, requires long training times and is prone to observer variability. This article introduces the first 3D direct volume-based level-set algorithm to segment plaques from 3D carotid ultrasound images. The plaque surfaces were first initialized based on the lumen and outer wall boundaries generated by a previously described semi-automatic algorithm and then deformed by a direct three-dimensional sparse field level-set algorithm, which enforced the longitudinal continuity of the segmented plaque surfaces. This is a marked advantage as compared to a previously proposed 2D slice-by-slice plaque segmentation method. In plaque boundary initialization, the previous technique performed a search on lines connecting corresponding point pairs of the outer wall and lumen boundaries. A limitation of this initialization strategy was that an inaccurate initial plaque boundary would be generated if the plaque was not enclosed entirely by the wall and lumen boundaries. A mechanism is proposed to extend the search range in order to capture the entire plaque if the outer wall boundary lies on a weak edge in the 3D ultrasound image. The proposed method was compared with the previously described 2D slice-by-slice plaque segmentation method in 26 three-dimensional carotid ultrasound images containing 27 plaques with volumes ranging from 12.5 to 450.0 mm3. The manually segmented plaque boundaries serve as the surrogate gold standard. Segmentation accuracy was quantified by volume-, area- and distance-based metrics, including absolute plaque volume difference (|ΔPV|), Dice similarity coefficient (DSC), mean and maximum absolute distance (MAD and MAXD). The proposed direct 3D plaque segmentation algorithm was associated with a significantly lower |ΔPV|, MAD and MAXD, and a significantly higher DSC compared to the previously described slice-by-slice algorithm (|ΔPV|:p=0.012, DSC: p=2.1×10-4, MAD: p=1.3×10-4, MAXD: p=5.2×10-4). The proposed 3D volume-based algorithm required 72±22 s to segment a plaque, which is 40% lower than the 2D slice-by-slice algorithm (114±18 s). The proposed automatic plaque segmentation method generates accurate and reproducible boundaries efficiently and will allow for streamlining plaque quantification based on 3D ultrasound images.

Breast lesion classification based on supersonic shear-wave elastography and automated lesion segmentation from B-mode ultrasound images.

Supersonic shear-wave elastography (SWE) has emerged as a useful imaging modality for breast lesion assessment. Regions of interest (ROIs) were required to be specified for extracting features that characterize malignancy of lesions. Although analyses have been performed in small rectangular ROIs identified manually by expert observers, the results were subject to observer variability and the analysis of small ROIs would potentially miss out important features available in other parts of the lesion. Recent investigations extracted features from the entire lesion segmented by B-mode ultrasound images either manually or semi-automatically, but lesion delineation using existing techniques is time-consuming and prone to variability as intensive user interactions are required. In addition, rich diagnostic features were available along the rim surrounding the lesion. The width of the rim analyzed was subjectively and empirically determined by expert observers in previous studies after intensive visual study on the images, which is time-consuming and susceptible to observer variability. This paper describes an analysis pipeline to segment and classify lesions efficiently. The lesion boundary was first initialized and then deformed based on energy fields generated by the dyadic wavelet transform. Features of the SWE images were extracted from inside and outside of a lesion for different widths of the surrounding rim. Then, feature selection was performed followed by the Support Vector Machine (SVM) classification. This strategy obviates the empirical and time-consuming selection of the surrounding rim width before the analysis. The pipeline was evaluated on 137 lesions. Feature selection was performed 20 times using different sets of 14 lesions (7 malignant, 7 benign). Leave-one-out SVM classification was performed in each of the 20 experiments with a mean sensitivity, specificity and accuracy of 95.1%, 94.6% and 94.8% respectively. The pipeline took an average of 20 s to process a lesion. The fact that this efficient pipeline generated classification accuracy superior to that of existing algorithms suggests that improved efficiency did not compromise classification accuracy. The ability to streamline the quantitative assessment of SWE images will potentially accelerate the adoption of the combined use of ultrasound and elastography in clinical practice.