Latent Transformer Models for out-of-distribution detection.

Any clinically-deployed image-processing pipeline must be robust to the full range of inputs it may be presented with. One popular approach to this challenge is to develop predictive models that can provide a measure of their uncertainty. Another approach is to use generative modelling to quantify the likelihood of inputs. Inputs with a low enough likelihood are deemed to be out-of-distribution and are not presented to the downstream predictive model. In this work, we evaluate several approaches to segmentation with uncertainty for the task of segmenting bleeds in 3D CT of the head. We show that these models can fail catastrophically when operating in the far out-of-distribution domain, often providing predictions that are both highly confident and wrong. We propose to instead perform out-of-distribution detection using the Latent Transformer Model: a VQ-GAN is used to provide a highly compressed latent representation of the input volume, and a transformer is then used to estimate the likelihood of this compressed representation of the input. We demonstrate this approach can identify images that are both far- and near- out-of-distribution, as well as provide spatial maps that highlight the regions considered to be out-of-distribution. Furthermore, we find a strong relationship between an image's likelihood and the quality of a model's segmentation on it, demonstrating that this approach is viable for filtering out unsuitable images.

Predicting mortality in acutely hospitalised older patients: the impact of model dimensionality.

BACKGROUND: The prediction of long-term mortality following acute illness can be unreliable for older patients, inhibiting the delivery of targeted clinical interventions. The difficulty plausibly arises from the complex, multifactorial nature of the underlying biology in this population, which flexible, multimodal models based on machine learning may overcome. Here, we test this hypothesis by quantifying the comparative predictive fidelity of such models in a large consecutive sample of older patients acutely admitted to hospital and characterise their biological support. METHODS: A set of 804 admission episodes involving 616 unique patients with a mean age of 84.5 years consecutively admitted to the Acute Geriatric service at University College Hospital were identified, in whom clinical diagnoses, blood tests, cognitive status, computed tomography of the head, and mortality within 600 days after admission were available. We trained and evaluated out-of-sample an array of extreme gradient boosted trees-based predictive models of incrementally greater numbers of investigational modalities and modelled features. Both linear and non-linear associations with investigational features were quantified. RESULTS: Predictive models of mortality showed progressively increasing fidelity with greater numbers of modelled modalities and dimensions. The area under the receiver operating characteristic curve rose from 0.67 (sd = 0.078) for age and sex to 0.874 (sd = 0.046) for the most comprehensive model. Extracranial bone and soft tissue features contributed more than intracranial features towards long-term mortality prediction. The anterior cingulate and angular gyri, and serum albumin, were the greatest intracranial and biochemical model contributors respectively. CONCLUSIONS: High-dimensional, multimodal predictive models of mortality based on routine clinical data offer higher predictive fidelity than simpler models, facilitating individual level prognostication and interventional targeting. The joint contributions of both extracranial and intracranial features highlight the potential importance of optimising somatic as well as neural functions in healthy ageing. Our findings suggest a promising path towards a high-fidelity, multimodal index of frailty.

ICAM-Reg: Interpretable Classification and Regression With Feature Attribution for Mapping Neurological Phenotypes in Individual Scans.

An important goal of medical imaging is to be able to precisely detect patterns of disease specific to individual scans; however, this is challenged in brain imaging by the degree of heterogeneity of shape and appearance. Traditional methods, based on image registration, historically fail to detect variable features of disease, as they utilise population-based analyses, suited primarily to studying group-average effects. In this paper we therefore take advantage of recent developments in generative deep learning to develop a method for simultaneous classification, or regression, and feature attribution (FA). Specifically, we explore the use of a VAE-GAN (variational autoencoder - general adversarial network) for translation called ICAM, to explicitly disentangle class relevant features, from background confounds, for improved interpretability and regression of neurological phenotypes. We validate our method on the tasks of Mini-Mental State Examination (MMSE) cognitive test score prediction for the Alzheimer's Disease Neuroimaging Initiative (ADNI) cohort, as well as brain age prediction, for both neurodevelopment and neurodegeneration, using the developing Human Connectome Project (dHCP) and UK Biobank datasets. We show that the generated FA maps can be used to explain outlier predictions and demonstrate that the inclusion of a regression module improves the disentanglement of the latent space. Our code is freely available on GitHub https://github.com/CherBass/ICAM.

Unsupervised brain imaging 3D anomaly detection and segmentation with transformers.

Pathological brain appearances may be so heterogeneous as to be intelligible only as anomalies, defined by their deviation from normality rather than any specific set of pathological features. Amongst the hardest tasks in medical imaging, detecting such anomalies requires models of the normal brain that combine compactness with the expressivity of the complex, long-range interactions that characterise its structural organisation. These are requirements transformers have arguably greater potential to satisfy than other current candidate architectures, but their application has been inhibited by their demands on data and computational resources. Here we combine the latent representation of vector quantised variational autoencoders with an ensemble of autoregressive transformers to enable unsupervised anomaly detection and segmentation defined by deviation from healthy brain imaging data, achievable at low computational cost, within relative modest data regimes. We compare our method to current state-of-the-art approaches across a series of experiments with 2D and 3D data involving synthetic and real pathological lesions. On real lesions, we train our models on 15,000 radiologically normal participants from UK Biobank and evaluate performance on four different brain MR datasets with small vessel disease, demyelinating lesions, and tumours. We demonstrate superior anomaly detection performance both image-wise and pixel/voxel-wise, achievable without post-processing. These results draw attention to the potential of transformers in this most challenging of imaging tasks.