The future of digital health with federated learning.

Data-driven machine learning (ML) has emerged as a promising approach for building accurate and robust statistical models from medical data, which is collected in huge volumes by modern healthcare systems. Existing medical data is not fully exploited by ML primarily because it sits in data silos and privacy concerns restrict access to this data. However, without access to sufficient data, ML will be prevented from reaching its full potential and, ultimately, from making the transition from research to clinical practice. This paper considers key factors contributing to this issue, explores how federated learning (FL) may provide a solution for the future of digital health and highlights the challenges and considerations that need to be addressed.

Automatic classification of dopamine transporter SPECT: deep convolutional neural networks can be trained to be robust with respect to variable image characteristics.

PURPOSE: This study investigated the potential of deep convolutional neural networks (CNN) for automatic classification of FP-CIT SPECT in multi-site or multi-camera settings with variable image characteristics. METHODS: The study included FP-CIT SPECT of 645 subjects from the Parkinson's Progression Marker Initiative (PPMI), 207 healthy controls, and 438 Parkinson's disease patients. SPECT images were smoothed with an isotropic 18-mm Gaussian kernel resulting in 3 different PPMI settings: (i) original (unsmoothed), (ii) smoothed, and (iii) mixed setting comprising all original and all smoothed images. A deep CNN with 2,872,642 parameters was trained, validated, and tested separately for each setting using 10 random splits with 60/20/20% allocation to training/validation/test sample. The putaminal specific binding ratio (SBR) was computed using a standard anatomical ROI predefined in MNI space (AAL atlas) or using the hottest voxels (HV) analysis. Both SBR measures were trained (ROC analysis, Youden criterion) using the same random splits as for the CNN. CNN and SBR trained in the mixed PPMI setting were also tested in an independent sample from clinical routine patient care (149 with non-neurodegenerative and 149 with neurodegenerative parkinsonian syndrome). RESULTS: Both SBR measures performed worse in the mixed PPMI setting compared to the pure PPMI settings (e.g., AAL-SBR accuracy = 0.900 ± 0.029 in the mixed setting versus 0.957 ± 0.017 and 0.952 ± 0.015 in original and smoothed setting, both p < 0.01). In contrast, the CNN showed similar accuracy in all PPMI settings (0.967 ± 0.018, 0.972 ± 0.014, and 0.955 ± 0.009 in mixed, original, and smoothed setting). Similar results were obtained in the clinical sample. After training in the mixed PPMI setting, only the CNN provided acceptable performance in the clinical sample. CONCLUSIONS: These findings provide proof of concept that a deep CNN can be trained to be robust with respect to variable site-, camera-, or scan-specific image characteristics without a large loss of diagnostic accuracy compared with mono-site/mono-camera settings. We hypothesize that a single CNN can be used to support the interpretation of FP-CIT SPECT at many different sites using different acquisition hardware and/or reconstruction software with only minor harmonization of acquisition and reconstruction protocols.

Cloud Deployment of High-Resolution Medical Image Analysis With TOMAAT.

BACKGROUND: Deep learning has been recently applied to a multitude of computer vision and medical image analysis problems. Although recent research efforts have improved the state of the art, most of the methods cannot be easily accessed, compared or used by other researchers or clinicians. Even if developers publish their code and pre-trained models on the internet, integration in stand-alone applications and existing workflows is often not straightforward, especially for clinical research partners. In this paper, we propose an open-source framework to provide AI-enabled medical image analysis through the network. METHODS: TOMAAT provides a cloud environment for general medical image analysis, composed of three basic components: (i) an announcement service, maintaining a public registry of (ii) multiple distributed server nodes offering various medical image analysis solutions, and (iii) client software offering simple interfaces for users. Deployment is realized through HTTP-based communication, along with an API and wrappers for common image manipulations during pre- and post-processing. RESULTS: We demonstrate the utility and versatility of TOMAAT on several hallmark medical image analysis tasks: segmentation, diffeomorphic deformable atlas registration, landmark localization, and workflow integration. Through TOMAAT, the high hardware demands, setup and model complexity of demonstrated approaches are transparent to users, who are provided with simple client interfaces. We present example clients in three-dimensional Slicer, in the web browser, on iOS devices and in a commercially available, certified medical image analysis suite. CONCLUSION: TOMAAT enables deployment of state-of-the-art image segmentation in the cloud, fostering interaction among deep learning researchers and medical collaborators in the clinic. Currently, a public announcement service is hosted by the authors, and several ready-to-use services are registered and enlisted at http://tomaat.cloud.

Robust navigation support in lowest dose image setting.

PURPOSE: Clinical cardiac electrophysiology (EP) is concerned with diagnosis and treatment of cardiac arrhythmia describing abnormality or perturbation in the normal activation sequence of the myocardium. With the recent introduction of lowest dose X-ray imaging protocol for EP procedures, interventional image enhancement has gained crucial importance for the well-being of patients as well as medical staff. METHODS: In this paper, we introduce a novel method to detect and track different EP catheter electrodes in lowest dose fluoroscopic sequences based on [Formula: see text]-sparse coding and online robust PCA (ORPCA). Besides being able to work on real lowest dose sequences, the underlying methodology achieves simultaneous detection and tracking of three main EP catheters used during ablation procedures. RESULTS: We have validated our algorithm on 16 lowest dose fluoroscopic sequences acquired during real cardiac ablation procedures. In addition to expert labels for 2 sequences, we have employed a crowdsourcing strategy to obtain ground truth labels for the remaining 14 sequences. In order to validate the effect of different training data, we have employed a leave-one-out cross-validation scheme yielding an average detection rate of [Formula: see text]. CONCLUSION: Besides these promising quantitative results, our medical partners also expressed their high satisfaction. Being based on [Formula: see text]-sparse coding and online robust PCA (ORPCA), our method advances previous approaches by being able to detect and track electrodes attached to multiple different catheters.

3D intra-operative ultrasound and MR image guidance: pursuing an ultrasound-based management of brainshift to enhance neuronavigation.

BACKGROUND: Brainshift is still a major issue in neuronavigation. Incorporating intra-operative ultrasound (iUS) with advanced registration algorithms within the surgical workflow is regarded as a promising approach for a better understanding and management of brainshift. This work is intended to (1) provide three-dimensional (3D) ultrasound reconstructions specifically for brain imaging in order to detect brainshift observed intra-operatively, (2) evaluate a novel iterative intra-operative ultrasound-based deformation correction framework, and (3) validate the performance of the proposed image-registration-based deformation estimation in a clinical environment. METHODS: Eight patients with brain tumors undergoing surgical resection are enrolled in this study. For each patient, a 3D freehand iUS system is employed in combination with an intra-operative navigation (iNav) system, and intra-operative ultrasound data are acquired at three timepoints during surgery. On this foundation, we present a novel resolution-preserving 3D ultrasound reconstruction, as well as a framework to detect brainshift through iterative registration of iUS images. To validate the system, the target registration error (TRE) is evaluated for each patient, and both rigid and elastic registration algorithms are analyzed. RESULTS: The mean TRE based on 3D-iUS improves significantly using the proposed brainshift compensation compared to neuronavigation (iNav) before (2.7 vs. 5.9 mm; [Formula: see text]) and after dural opening (4.2 vs. 6.2 mm, [Formula: see text]), but not after resection (6.7 vs. 7.5 mm; [Formula: see text]). iUS depicts a significant ([Formula: see text]) dynamic spatial brainshift throughout the three timepoints. Accuracy of registration can be improved through rigid and elastic registrations by 29.2 and 33.3%, respectively, after dural opening, and by 5.2 and 0.4%, after resection. CONCLUSION: 3D-iUS systems can improve the detection of brainshift and significantly increase the accuracy of the navigation in a real scenario. 3D-iUS can thus be regarded as a robust, reliable, and feasible technology to enhance neuronavigation.

Standardized Evaluation System for Left Ventricular Segmentation Algorithms in 3D Echocardiography.

Real-time 3D Echocardiography (RT3DE) has been proven to be an accurate tool for left ventricular (LV) volume assessment. However, identification of the LV endocardium remains a challenging task, mainly because of the low tissue/blood contrast of the images combined with typical artifacts. Several semi and fully automatic algorithms have been proposed for segmenting the endocardium in RT3DE data in order to extract relevant clinical indices, but a systematic and fair comparison between such methods has so far been impossible due to the lack of a publicly available common database. Here, we introduce a standardized evaluation framework to reliably evaluate and compare the performance of the algorithms developed to segment the LV border in RT3DE. A database consisting of 45 multivendor cardiac ultrasound recordings acquired at different centers with corresponding reference measurements from three experts are made available. The algorithms from nine research groups were quantitatively evaluated and compared using the proposed online platform. The results showed that the best methods produce promising results with respect to the experts' measurements for the extraction of clinical indices, and that they offer good segmentation precision in terms of mean distance error in the context of the experts' variability range. The platform remains open for new submissions.

Automatic 3D reconstruction of electrophysiology catheters from two-view monoplane C-arm image sequences.

PURPOSE: Catheter guidance is a vital task for the success of electrophysiology interventions. It is usually provided through fluoroscopic images that are taken intra-operatively. The cardiologists, who are typically equipped with C-arm systems, scan the patient from multiple views rotating the fluoroscope around one of its axes. The resulting sequences allow the cardiologists to build a mental model of the 3D position of the catheters and interest points from the multiple views. METHOD: We describe and compare different 3D catheter reconstruction strategies and ultimately propose a novel and robust method for the automatic reconstruction of 3D catheters in non-synchronized fluoroscopic sequences. This approach does not purely rely on triangulation but incorporates prior knowledge about the catheters. In conjunction with an automatic detection method, we demonstrate the performance of our method compared to ground truth annotations. RESULTS: In our experiments that include 20 biplane datasets, we achieve an average reprojection error of 0.43 mm and an average reconstruction error of 0.67 mm compared to gold standard annotation. CONCLUSIONS: In clinical practice, catheters suffer from complex motion due to the combined effect of heartbeat and respiratory motion. As a result, any 3D reconstruction algorithm via triangulation is imprecise. We have proposed a new method that is fully automatic and highly accurate to reconstruct catheters in three dimensions.

Multimodal image-guided prostate fusion biopsy based on automatic deformable registration.

PURPOSE: Transrectal ultrasound (TRUS)-guided random prostate biopsy is, in spite of its low sensitivity, the gold standard for the diagnosis of prostate cancer. The recent advent of PET imaging using a novel dedicated radiotracer, [Formula: see text]-labeled prostate-specific membrane antigen (PSMA), combined with MRI provides improved pre-interventional identification of suspicious areas. This work proposes a multimodal fusion image-guided biopsy framework that combines PET-MRI images with TRUS, using automatic segmentation and registration, and offering real-time guidance. METHODS: The prostate TRUS images are automatically segmented with a Hough transform-based random forest approach. The registration is based on the Coherent Point Drift algorithm to align surfaces elastically and to propagate the deformation field calculated from thin-plate splines to the whole gland. RESULTS: The method, which has minimal requirements and temporal overhead in the existing clinical workflow, is evaluated in terms of surface distance and landmark registration error with respect to the clinical ground truth. Evaluations on agar-gelatin phantoms and clinical data of 13 patients confirm the validity of this approach. CONCLUSION: The system is able to successfully map suspicious regions from PET/MRI to the interventional TRUS image.

3D transcranial ultrasound as a novel intra-operative imaging technique for DBS surgery: a feasibility study.

PURPOSE: Intra-operative image guidance during deep brain stimulation (DBS) surgery is usually avoided due to cost and overhead of intra-operative MRI and CT acquisitions. Recently, there has been interest in the community towards the usage of non-invasive transcranial ultrasound (TCUS) through the preauricular bone window. In this work, we investigate, for the first time, the feasibility of using 3D-TCUS for imaging of already implanted DBS electrodes. As a first step towards this goal, we report imaging methods and electrode localisation errors outside of the operating room on eight previously operated DBS patients. METHODS: We evaluate the feasibility of using 3D-TCUS by registering volumes to pre-operative T1-MRI. US-MRI registration is achieved through a two-step point-based approach. First, a rough surface scan of the subjects' skin surface in 3D-TCUS space is registered to a segmented skin-surface point cloud from MRI. Next, we perform a refinement using rigid registration of multiple pairs of manually marked anatomical landmarks. We validate against post-operative CT scans which are also registered to pre-operative MRI. RESULTS: Qualitative results are given in form of 3D reconstruction examples at 2.5 and 3.5 MHz TCUS image frequency, overlaid on pre-operative T1-MRI and post-operative CT. Quantitative evaluation is performed by reporting the accuracy of electrode tip localisation at 2.5 and 3.5 MHz after our US-MRI approach. As a baseline, we also report RMSE errors for pairs of anatomical landmarks in pre-operative MRI and 3D-TCUS. CONCLUSION: Multiple image examples show the appearance and quality of 3D-TCUS scans, depending on the bone window. Overall accuracy of anatomic point pairs lies on the order of 3.2 mm, using our registration approach. Compared to this baseline, electrode tip localisation in 3D-TCUS has a mean accuracy on the order of 4.8 mm and a precision on the order of 2.3 mm. While insufficient at first glance, we argue why these results are promising nonetheless. Our work motivates further future work in improved TCUS scanning, advanced TCUS-MRI registration and computer-aided electrode detection in 3D-TCUS.

Fully automatic catheter localization in C-arm images using l1-sparse coding.

We propose a method to perform automatic detection and tracking of electrophysiology (EP) catheters in C-arm fluoroscopy sequences. Our approach does not require any initialization, is completely automatic, and can concurrently track an arbitrary number of overlapping catheters. After a pre-processing step, we employ sparse coding to first detect candidate catheter tips, and subsequently detect and track the catheters. The proposed technique is validated on 2835 C-arm images, which include 39,690 manually selected ground-truth catheter electrodes. Results demonstrated sub-millimeter detection accuracy and real-time tracking performances.

Automatic detection of multiple and overlapping EP catheters in fluoroscopic sequences.

We propose a method to perform automatic detection of electrophysiology (EP) catheters in fluoroscopic sequences. Our approach does not need any initialization, is completely automatic, and can detect an arbitrary number of catheters at the same time. The method is based on the usage of blob detectors and clustering in order to detect all catheter electrodes, overlapping or not, within the X-ray images. The proposed technique is validated on 1422 fluoroscopic images yielding a tip detection rate of 99.3% and mean distance of 0.5mm from manually labeled ground truth centroids for all electrodes.