Medical Transformer: Universal Encoder for 3-D Brain MRI Analysis.

Transfer learning has attracted considerable attention in medical image analysis because of the limited number of annotated 3-D medical datasets available for training data-driven deep learning models in the real world. We propose Medical Transformer, a novel transfer learning framework that effectively models 3-D volumetric images as a sequence of 2-D image slices. To improve the high-level representation in 3-D-form empowering spatial relations, we use a multiview approach that leverages information from three planes of the 3-D volume, while providing parameter-efficient training. For building a source model generally applicable to various tasks, we pretrain the model using self-supervised learning (SSL) for masked encoding vector prediction as a proxy task, using a large-scale normal, healthy brain magnetic resonance imaging (MRI) dataset. Our pretrained model is evaluated on three downstream tasks: 1) brain disease diagnosis; 2) brain age prediction; and 3) brain tumor segmentation, which are widely studied in brain MRI research. Experimental results demonstrate that our Medical Transformer outperforms the state-of-the-art (SOTA) transfer learning methods, efficiently reducing the number of parameters by up to approximately 92% for classification and regression tasks and 97% for segmentation task, and it also achieves good performance in scenarios where only partial training samples are used.

Multi-View Integrative Attention-Based Deep Representation Learning for Irregular Clinical Time-Series Data.

Electronic health record (EHR) data are sparse and irregular as they are recorded at irregular time intervals, and different clinical variables are measured at each observation point. In this work, to handle irregular multivariate time-series data, we consider the human knowledge of the aspects to be measured and time to measure them in different situations, known as multi-view features, which are indirectly represented in the data. We propose a scheme to realize multi-view features integration learning via a self-attention mechanism. Specifically, we devise a novel multi-integration attention module (MIAM) to extract complex information that is inherent in irregular time-series data. We explicitly learn the relationships among the observed values, missing indicators, and time interval between the consecutive observations in a simultaneous manner. In addition, we build an attention-based decoder as a missing value imputer that helps empower the representation learning of the interrelations among multi-view observations for the prediction task this decoder operates only in the training phase so that the final model is implemented in an imputation-free manner. We validated the effectiveness of our method over the public MIMIC-III and PhysioNet challenge 2012 datasets by comparing with and outperforming the state-of-the-art methods in three downstream tasks i.e., prediction of the in-hospital mortality, prediction of the length of stay, and phenotyping. Moreover, we conduct a layer-wise relevance propagation (LRP) analysis based on case studies to highlight the explainability of the trained model.

Uncertainty-Aware Variational-Recurrent Imputation Network for Clinical Time Series.

Electronic health records (EHR) consist of longitudinal clinical observations portrayed with sparsity, irregularity, and high dimensionality, which become major obstacles in drawing reliable downstream clinical outcomes. Although there exist great numbers of imputation methods to tackle these issues, most of them ignore correlated features, temporal dynamics, and entirely set aside the uncertainty. Since the missing value estimates involve the risk of being inaccurate, it is appropriate for the method to handle the less certain information differently than the reliable data. In that regard, we can use the uncertainties in estimating the missing values as the fidelity score to be further utilized to alleviate the risk of biased missing value estimates. In this work, we propose a novel variational-recurrent imputation network, which unifies an imputation and a prediction network by taking into account the correlated features, temporal dynamics, as well as uncertainty. Specifically, we leverage the deep generative model in the imputation, which is based on the distribution among variables, and a recurrent imputation network to exploit the temporal relations, in conjunction with utilization of the uncertainty. We validated the effectiveness of our proposed model on two publicly available real-world EHR datasets: 1) PhysioNet Challenge 2012 and 2) MIMIC-III, and compared the results with other competing state-of-the-art methods in the literature.

Deep recurrent model for individualized prediction of Alzheimer's disease progression.

Alzheimer's disease (AD) is known as one of the major causes of dementia and is characterized by slow progression over several years, with no treatments or available medicines. In this regard, there have been efforts to identify the risk of developing AD in its earliest time. While many of the previous works considered cross-sectional analysis, more recent studies have focused on the diagnosis and prognosis of AD with longitudinal or time series data in a way of disease progression modeling. Under the same problem settings, in this work, we propose a novel computational framework that can predict the phenotypic measurements of MRI biomarkers and trajectories of clinical status along with cognitive scores at multiple future time points. However, in handling time series data, it generally faces many unexpected missing observations. In regard to such an unfavorable situation, we define a secondary problem of estimating those missing values and tackle it in a systematic way by taking account of temporal and multivariate relations inherent in time series data. Concretely, we propose a deep recurrent network that jointly tackles the four problems of (i) missing value imputation, (ii) phenotypic measurements forecasting, (iii) trajectory estimation of a cognitive score, and (iv) clinical status prediction of a subject based on his/her longitudinal imaging biomarkers. Notably, the learnable parameters of all the modules in our predictive models are trained in an end-to-end manner by taking the morphological features and cognitive scores as input, with our circumspectly defined loss function. In our experiments over The Alzheimers Disease Prediction Of Longitudinal Evolution (TADPOLE) challenge cohort, we measured performance for various metrics and compared our method to competing methods in the literature. Exhaustive analyses and ablation studies were also conducted to better confirm the effectiveness of our method.

Uncertainty-Gated Stochastic Sequential Model for EHR Mortality Prediction.

Electronic health records (EHRs) are characterized as nonstationary, heterogeneous, noisy, and sparse data; therefore, it is challenging to learn the regularities or patterns inherent within them. In particular, sparseness caused mostly by many missing values has attracted the attention of researchers who have attempted to find a better use of all available samples for determining the solution of a primary target task through defining a secondary imputation problem. Methodologically, existing methods, either deterministic or stochastic, have applied different assumptions to impute missing values. However, once the missing values are imputed, most existing methods do not consider the fidelity or confidence of the imputed values in the modeling of downstream tasks. Undoubtedly, an erroneous or improper imputation of missing variables can cause difficulties in the modeling as well as a degraded performance. In this study, we present a novel variational recurrent network that: 1) estimates the distribution of missing variables (e.g., the mean and variance) allowing to represent uncertainty in the imputed values; 2) updates hidden states by explicitly applying fidelity based on a variance of the imputed values during a recurrence (i.e., uncertainty propagation over time); and 3) predicts the possibility of in-hospital mortality. It is noteworthy that our model can conduct these procedures in a single stream and learn all network parameters jointly in an end-to-end manner. We validated the effectiveness of our method using the public data sets of MIMIC-III and PhysioNet challenge 2012 by comparing with and outperforming other state-of-the-art methods for mortality prediction considered in our experiments. In addition, we identified the behavior of the model that well represented the uncertainties for the imputed estimates, which showed a high correlation between the uncertainties and mean absolute error (MAE) scores for imputation.

Identifying resting-state effective connectivity abnormalities in drug-naïve major depressive disorder diagnosis via graph convolutional networks.

Major depressive disorder (MDD) is a leading cause of disability; its symptoms interfere with social, occupational, interpersonal, and academic functioning. However, the diagnosis of MDD is still made by phenomenological approach. The advent of neuroimaging techniques allowed numerous studies to use resting-state functional magnetic resonance imaging (rs-fMRI) and estimate functional connectivity for brain-disease identification. Recently, attempts have been made to investigate effective connectivity (EC) that represents causal relations among regions of interest. In the meantime, to identify meaningful phenotypes for clinical diagnosis, graph-based approaches such as graph convolutional networks (GCNs) have been leveraged recently to explore complex pairwise similarities in imaging/nonimaging features among subjects. In this study, we validate the use of EC for MDD identification by estimating its measures via a group sparse representation along with a structured equation modeling approach in a whole-brain data-driven manner from rs-fMRI. To distinguish drug-naïve MDD patients from healthy controls, we utilize spectral GCNs based on a population graph to successfully integrate EC and nonimaging phenotypic information. Furthermore, we devise a novel sensitivity analysis method to investigate the discriminant connections for MDD identification in our trained GCNs. Our experimental results validated the effectiveness of our method in various scenarios, and we identified altered connectivities associated with the diagnosis of MDD.

Modeling regional dynamics in low-frequency fluctuation and its application to Autism spectrum disorder diagnosis.

With the advent of neuroimaging techniques, many studies in the literature have validated the use of resting-state fMRI (rs-fMRI) for understanding functional mechanisms of the brain, as well as for identifying brain disorders or diseases. One of the main streams in recent studies of modeling and analyzing rs-fMRI data is to account for the dynamic characteristics of a brain. In this study, we propose a novel method that directly models the regional temporal BOLD fluctuations in a stochastic manner and estimates the dynamic characteristics in the form of likelihoods. Specifically, we modeled temporal BOLD fluctuation of individual Regions Of Interest (ROIs) by means of Hidden Markov Models (HMMs), and then estimated the 'goodness-of-fit' of each ROI's BOLD signals to the corresponding trained HMM in terms of a likelihood. Using estimated likelihoods of the ROIs over the whole brain as features, we built a classifier that can discriminate subjects with Autism Spectrum Disorder (ASD) from Typically Developing (TD) controls at an individual level. In order to interpret the trained HMMs and a classifier from a neuroscience perspective, we also conducted model analysis. First, we investigated the learned weight coefficients of a classifier by transforming them into activation patterns, from which we could identify the ROIs that are highly associated with ASD and TD groups. Second, we explored the characteristics of temporal BOLD signals in terms of functional networks by clustering them based on sequences of the hidden states decoded with the trained HMMs. We validated the effectiveness of the proposed method by achieving the state-of-the-art performance on the ABIDE dataset and observed insightful patterns related to ASD.