Multi-Task Learning for Motion Analysis and Segmentation in 3D Echocardiography.

Characterizing left ventricular deformation and strain using 3D+time echocardiography provides useful insights into cardiac function and can be used to detect and localize myocardial injury. To achieve this, it is imperative to obtain accurate motion estimates of the left ventricle. In many strain analysis pipelines, this step is often accompanied by a separate segmentation step; however, recent works have shown both tasks to be highly related and can be complementary when optimized jointly. In this work, we present a multi-task learning network that can simultaneously segment the left ventricle and track its motion between multiple time frames. Two task-specific networks are trained using a composite loss function. Cross-stitch units combine the activations of these networks by learning shared representations between the tasks at different levels. We also propose a novel shape-consistency unit that encourages motion propagated segmentations to match directly predicted segmentations. Using a combined synthetic and in-vivo 3D echocardiography dataset, we demonstrate that our proposed model can achieve excellent estimates of left ventricular motion displacement and myocardial segmentation. Additionally, we observe strong correlation of our image-based strain measurements with crystal-based strain measurements as well as good correspondence with SPECT perfusion mappings. Finally, we demonstrate the clinical utility of the segmentation masks in estimating ejection fraction and sphericity indices that correspond well with benchmark measurements.

Patient-Specific Heart Geometry Modeling for Solid Biomechanics Using Deep Learning.

Automated volumetric meshing of patient-specific heart geometry can help expedite various biomechanics studies, such as post-intervention stress estimation. Prior meshing techniques often neglect important modeling characteristics for successful downstream analyses, especially for thin structures like the valve leaflets. In this work, we present DeepCarve (Deep Cardiac Volumetric Mesh): a novel deformation-based deep learning method that automatically generates patient-specific volumetric meshes with high spatial accuracy and element quality. The main novelty in our method is the use of minimally sufficient surface mesh labels for precise spatial accuracy and the simultaneous optimization of isotropic and anisotropic deformation energies for volumetric mesh quality. Mesh generation takes only 0.13 seconds/scan during inference, and each mesh can be directly used for finite element analyses without any manual post-processing. Calcification meshes can also be subsequently incorporated for increased simulation accuracy. Numerous stent deployment simulations validate the viability of our approach for large-batch analyses. Our code is available at https://github.com/danpak94/Deep-Cardiac-Volumetric-Mesh.

Co-attention spatial transformer network for unsupervised motion tracking and cardiac strain analysis in 3D echocardiography.

Myocardial ischemia/infarction causes wall-motion abnormalities in the left ventricle. Therefore, reliable motion estimation and strain analysis using 3D+time echocardiography for localization and characterization of myocardial injury is valuable for early detection and targeted interventions. Previous unsupervised cardiac motion tracking methods rely on heavily-weighted regularization functions to smooth out the noisy displacement fields in echocardiography. In this work, we present a Co-Attention Spatial Transformer Network (STN) for improved motion tracking and strain analysis in 3D echocardiography. Co-Attention STN aims to extract inter-frame dependent features between frames to improve the motion tracking in otherwise noisy 3D echocardiography images. We also propose a novel temporal constraint to further regularize the motion field to produce smooth and realistic cardiac displacement paths over time without prior assumptions on cardiac motion. Our experimental results on both synthetic and in vivo 3D echocardiography datasets demonstrate that our Co-Attention STN provides superior performance compared to existing methods. Strain analysis from Co-Attention STNs also correspond well with the matched SPECT perfusion maps, demonstrating the clinical utility for using 3D echocardiography for infarct localization.

Simultaneous Segmentation and Motion Estimation of Left Ventricular Myocardium in 3D Echocardiography Using Multi-task Learning.

Motion estimation and segmentation are both critical steps in identifying and assessing myocardial dysfunction, but are traditionally treated as unique tasks and solved as separate steps. However, many motion estimation techniques rely on accurate segmentations. It has been demonstrated in the computer vision and medical image analysis literature that both these tasks may be mutually beneficial when solved simultaneously. In this work, we propose a multi-task learning network that can concurrently predict volumetric segmentations of the left ventricle and estimate motion between 3D echocardiographic image pairs. The model exploits complementary latent features between the two tasks using a shared feature encoder with task-specific decoding branches. Anatomically inspired constraints are incorporated to enforce realistic motion patterns. We evaluate our proposed model on an in vivo 3D echocardiographic canine dataset. Results suggest that coupling these two tasks in a learning framework performs favorably when compared against single task learning and other alternative methods.

Outcomes of mechanical circulatory support for acute myocardial infarction complicated by cardiogenic shock.

BACKGROUND: Mechanical circulatory support (MCS) with the Impella device (Abiomed, Danvers, MA) has been associated with higher in-hospital mortality than intra-aortic balloon pump (IABP) in the Premier Healthcare Database and National Cardiovascular Data Registry. METHODS: The objective of this retrospective cohort study was to describe trends and outcomes of Impella usage in acute myocardial infarction complicated by cardiogenic shock (AMICS) treated with MCS (Impella or IABP) using real-world observational data from the National Inpatient Sample (NIS) including hospitalizations for AMICS managed with MCS between January 2012 to December 2017. The primary outcomes included in-hospital mortality, transfusion, acute kidney injury, stroke, total costs, and length of stay. Propensity score matching was performed with hierarchical models using risk factor and Elixhauser comorbidity variables. RESULTS AND CONCLUSION: We identified 54,480 hospitalizations for AMICS managed with MCS including 5750 (10.5%) utilizing Impella. Throughout the study period, Impella usage increased yearly to 19.9% of AMICS cases in 2017. After propensity score matching, Impella was associated with higher in-hospital mortality (odds ratio [OR] 1.74, 95% confidence interval [CI] 1.41-2.13) and transfusions (OR 1.97, 95% CI 1.40-2.78) than IABP, without association with acute kidney injury or stroke. Impella use was associated with higher hospital costs (mean difference $22,416.80 [95% CI $17,029-27,804]). Impella usage for AMICS increased significantly from 2012 to 2017 and was associated with increased in-hospital mortality and costs. Randomized controlled trials are urgently needed to assess the safety and efficacy of Impella.

Multi-frame Attention Network for Left Ventricle Segmentation in 3D Echocardiography.

Echocardiography is one of the main imaging modalities used to assess the cardiovascular health of patients. Among the many analyses performed on echocardiography, segmentation of left ventricle is crucial to quantify the clinical measurements like ejection fraction. However, segmentation of left ventricle in 3D echocardiography remains a challenging and tedious task. In this paper, we propose a multi-frame attention network to improve the performance of segmentation of left ventricle in 3D echocardiography. The multi-frame attention mechanism allows highly correlated spatiotemporal features in a sequence of images that come after a target image to be used to augment the performance of segmentation. Experimental results shown on 51 in vivo porcine 3D+time echocardiography images show that utilizing correlated spatiotemporal features significantly improves the performance of left ventricle segmentation when compared to other standard deep learning-based medical image segmentation models.

SHAPE-REGULARIZED UNSUPERVISED LEFT VENTRICULAR MOTION NETWORK WITH SEGMENTATION CAPABILITY IN 3D+TIME ECHOCARDIOGRAPHY.

Accurate motion estimation and segmentation of the left ventricle from medical images are important tasks for quantitative evaluation of cardiovascular health. Echocardiography offers a cost-efficient and non-invasive modality for examining the heart, but provides additional challenges for automated analyses due to the low signal-to-noise ratio inherent in ultrasound imaging. In this work, we propose a shape regularized convolutional neural network for estimating dense displacement fields between sequential 3D B-mode echocardiography images with the capability of also predicting left ventricular segmentation masks. Manually traced segmentations are used as a guide to assist in the unsupervised estimation of displacement between a source and a target image while also serving as labels to train the network to additionally predict segmentations. To enforce realistic cardiac motion patterns, a flow incompressibility term is also incorporated to penalize divergence. Our proposed network is evaluated on an in vivo canine 3D+t B-mode echocardiographic dataset. It is shown that the shape regularizer improves the motion estimation performance of the network and our overall model performs favorably against competing methods.

Learning-Based Regularization for Cardiac Strain Analysis via Domain Adaptation.

Reliable motion estimation and strain analysis using 3D+ time echocardiography (4DE) for localization and characterization of myocardial injury is valuable for early detection and targeted interventions. However, motion estimation is difficult due to the low-SNR that stems from the inherent image properties of 4DE, and intelligent regularization is critical for producing reliable motion estimates. In this work, we incorporated the notion of domain adaptation into a supervised neural network regularization framework. We first propose a semi-supervised Multi-Layered Perceptron (MLP) network with biomechanical constraints for learning a latent representation that is shown to have more physiologically plausible displacements. We extended this framework to include a supervised loss term on synthetic data and showed the effects of biomechanical constraints on the network's ability for domain adaptation. We validated the semi-supervised regularization method on in vivo data with implanted sonomicrometers. Finally, we showed the ability of our semi-supervised learning regularization approach to identify infarct regions using estimated regional strain maps with good agreement to manually traced infarct regions from postmortem excised hearts.

A SEMI-SUPERVISED JOINT LEARNING APPROACH TO LEFT VENTRICULAR SEGMENTATION AND MOTION TRACKING IN ECHOCARDIOGRAPHY.

Accurate interpretation and analysis of echocardiography is important in assessing cardiovascular health. However, motion tracking often relies on accurate segmentation of the myocardium, which can be difficult to obtain due to inherent ultrasound properties. In order to address this limitation, we propose a semi-supervised joint learning network that exploits overlapping features in motion tracking and segmentation. The network simultaneously trains two branches: one for motion tracking and one for segmentation. Each branch learns to extract features relevant to their respective tasks and shares them with the other. Learned motion estimations propagate a manually segmented mask through time, which is used to guide future segmentation predictions. Physiological constraints are introduced to enforce realistic cardiac behavior. Experimental results on synthetic and in vivo canine 2D+t echocardiographic sequences outperform some competing methods in both tasks.

A Semi-supervised Joint Network for Simultaneous Left Ventricular Motion Tracking and Segmentation in 4D Echocardiography.

This work presents a novel deep learning method to combine segmentation and motion tracking in 4D echocardiography. The network iteratively trains a motion branch and a segmentation branch. The motion branch is initially trained entirely unsupervised and learns to roughly map the displacements between a source and a target frame. The estimated displacement maps are then used to generate pseudo-ground truth labels to train the segmentation branch. The labels predicted by the trained segmentation branch are fed back into the motion branch and act as landmarks to help retrain the branch to produce smoother displacement estimations. These smoothed out displacements are then used to obtain smoother pseudo-labels to retrain the segmentation branch. Additionally, a biomechanically-inspired incompressibility constraint is implemented in order to encourage more realistic cardiac motion. The proposed method is evaluated against other approaches using synthetic and in-vivo canine studies. Both the segmentation and motion tracking results of our model perform favorably against competing methods.

Unsupervised Motion Tracking of Left Ventricle in Echocardiography.

Accurate motion tracking of the left ventricle is critical in detecting wall motion abnormalities in the heart after an injury such as a myocardial infarction. We propose an unsupervised motion tracking framework with physiological constraints to learn dense displacement fields between sequential pairs of 2-D B-mode echocardiography images. Current deep-learning motion-tracking algorithms require large amounts of data to provide ground-truth, which is difficult to obtain for in vivo datasets (such as patient data and animal studies), or are unsuccessful in tracking motion between echocardiographic images due to inherent ultrasound properties (such as low signal-to-noise ratio and various image artifacts). We design a U-Net inspired convolutional neural network that uses manually traced segmentations as a guide to learn displacement estimations between a source and target image without ground-truth displacement fields by minimizing the difference between a transformed source frame and the original target frame. We then penalize divergence in the displacement field in order to enforce incompressibility within the left ventricle. We demonstrate the performance of our model on synthetic and in vivo canine 2-D echocardiography datasets by comparing it against a non-rigid registration algorithm and a shape-tracking algorithm. Our results show favorable performance of our model against both methods.

Three-dimensional motion correction using speckle and phase for in vivo computed optical interferometric tomography.

Over the years, many computed optical interferometric techniques have been developed to perform high-resolution volumetric tomography. By utilizing the phase and amplitude information provided with interferometric detection, post-acquisition corrections for defocus and optical aberrations can be performed. The introduction of the phase, though, can dramatically increase the sensitivity to motion (most prominently along the optical axis). In this paper, we present two algorithms which, together, can correct for motion in all three dimensions with enough accuracy for defocus and aberration correction in computed optical interferometric tomography. The first algorithm utilizes phase differences within the acquired data to correct for motion along the optical axis. The second algorithm utilizes the addition of a speckle tracking system using temporally- and spatially-coherent illumination to measure motion orthogonal to the optical axis. The use of coherent illumination allows for high-contrast speckle patterns even when imaging apparently uniform samples or when highly aberrated beams cannot be avoided.