Erratum: On the role of artificial intelligence in medical imaging of COVID-19.

[This corrects the article DOI: 10.1016/j.patter.2021.100269.].

On the role of artificial intelligence in medical imaging of COVID-19.

Although a plethora of research articles on AI methods on COVID-19 medical imaging are published, their clinical value remains unclear. We conducted the largest systematic review of the literature addressing the utility of AI in imaging for COVID-19 patient care. By keyword searches on PubMed and preprint servers throughout 2020, we identified 463 manuscripts and performed a systematic meta-analysis to assess their technical merit and clinical relevance. Our analysis evidences a significant disparity between clinical and AI communities, in the focus on both imaging modalities (AI experts neglected CT and ultrasound, favoring X-ray) and performed tasks (71.9% of AI papers centered on diagnosis). The vast majority of manuscripts were found to be deficient regarding potential use in clinical practice, but 2.7% (n = 12) publications were assigned a high maturity level and are summarized in greater detail. We provide an itemized discussion of the challenges in developing clinically relevant AI solutions with recommendations and remedies.

Creation and validation of a chest X-ray dataset with eye-tracking and report dictation for AI development.

We developed a rich dataset of Chest X-Ray (CXR) images to assist investigators in artificial intelligence. The data were collected using an eye-tracking system while a radiologist reviewed and reported on 1,083 CXR images. The dataset contains the following aligned data: CXR image, transcribed radiology report text, radiologist's dictation audio and eye gaze coordinates data. We hope this dataset can contribute to various areas of research particularly towards explainable and multimodal deep learning/machine learning methods. Furthermore, investigators in disease classification and localization, automated radiology report generation, and human-machine interaction can benefit from these data. We report deep learning experiments that utilize the attention maps produced by the eye gaze dataset to show the potential utility of this dataset.

An End-to-End Deep Learning Pipeline for Emphysema Quantification Using Multi-label Learning.

We propose and validate an end-to-end deep learning pipeline employing multi-label learning as a tool for creating differential diagnoses of lung pathology as well as quantifying the extent and distribution of emphysema in chest CT images. The proposed pipeline first employs deep learning based volumetric lung segmentation using a 3D CNN to extract the entire lung out of CT images. Then, a multi-label learning model is exploited for the classification creation differential diagnoses for emphysema and then used to correlate with the emphysema diagnosed by radiologists. The five lung tissue patterns which are involved in most lung disease differential diagnoses were classified as: ground glass, fibrosis, micronodules (random, perilymphatic and centrilobular lung nodules), normal appearing lung, and emphysematous lung tissue. To the best of our knowledge, this is the first end-to-end deep learning pipeline for the creation of differential diagnoses for lung disease and the quantification of emphysema. A comparative analysis shows the performance of the proposed pipeline on two publicly available datasets.

Improving the Path from Diagnoses to Documentation: A Cognitive Review Tool for Clinical Notes and Administrative Records.

EMR systems are intended to improve patient-centered care management and hospital administrative processing. However, the information stored in EMRs can be disorganized, incomplete, or inconsistent, creating problems at the patient and system level. We present a technology that reconciles inconsistencies between clinical diagnoses and administrative records by analyzing free-text notes, problem lists and recorded diagnoses in real time. A fully integrated pipeline has been developed for efficient, knowledge-driven extraction, normalization, and matching of disease terms among structured and unstructured data, with modular precision of 94-98% on over 1000 patients. This cognitive data review tool improves the path from diagnosis to documentation, facilitating accurate and timely clinical and administrative decision-making.

Metric hashing forests.

In this paper, we propose metric Hashing Forests (mHF) which is a supervised variant of random forests tailored for the task of nearest neighbor retrieval through hashing. This is achieved by training independent hashing trees that parse and encode the feature space such that local class neighborhoods are preserved and encoded with similar compact binary codes. At the level of each internal node, locality preserving projections are employed to project data to a latent subspace, where separability between dissimilar points is enhanced. Following which, we define an oblique split that maximally preserves this separability and facilitates defining local neighborhoods of similar points. By incorporating the inverse-lookup search scheme within the mHF, we can then effectively mitigate pairwise neuron similarity comparisons, which allows for scalability to massive databases with little additional time overhead. Exhaustive experimental validations on 22,265 neurons curated from over 120 different archives demonstrate the superior efficacy of mHF in terms of its retrieval performance and precision of classification in contrast to state-of-the-art hashing and metric learning based methods. We conclude that the proposed method can be utilized effectively for similarity-preserving retrieval and categorization in large neuron databases.

Mining echocardiography workflows for disease discriminative patterns.

OBJECTIVE: To provide quick diagnostic insights to medical practitioners into echocardiograms by only analyzing the echocardiogram workflows (defined as the sequence of modalities examined). METHODS: We define a dictionary of workflows, called subflows, which are commonly encountered in echocardiography workflows but are mutually exclusive. We represent each workflow as a mixture of dictionary subflows and learn discriminative models for various cardiac diseases using Support Vector Machines. Using these discriminative models, we can predict occurrences of diseases for any, yet unseen, echocardiogram workflow. RESULTS: Working with a corpus of 2300 echocardiograms workflows, we build a dictionary of 172 subflows. Using the associated reports (expert created) we identify the ground-truth diagnoses. We then build discriminative models for 7 different cardiac diseases. Using just the workflow as input, these models can predict diseases on average with over 75% accuracy. CONCLUSIONS: Mining collection of echocardiography workflows, for the first time, we are able to predict diseases without even looking at the image contents.

AALIM: a cardiac clinical decision support system powered by advanced multi-modal analytics.

Modern Electronic Medical Record (EMR) systems often integrate large amounts of data from multiple disparate sources. To do so, EMR systems must align the data to create consistency between these sources. The data should also be presented in a manner that allows a clinician to quickly understand the complete condition and history of a patient's health. We develop the AALIM system to address these issues using advanced multimodal analytics. First, it extracts and computes multiple features and cues from the patient records and medical tests. This additional metadata facilitates more accurate alignment of the various modalities, enables consistency check and empowers a clear, concise presentation of the patient's complete health information. The system further provides a multimodal search for similar cases within the EMR system, and derives related conditions and drugs information from them. We applied our approach to cardiac data from a major medical care organization and found that it produced results with sufficient quality to assist the clinician making appropriate clinical decisions.

Closed-form Jensen-Renyi divergence for mixture of Gaussians and applications to group-wise shape registration.

In this paper, we propose a generalized group-wise non-rigid registration strategy for multiple unlabeled point-sets of unequal cardinality, with no bias toward any of the given point-sets. To quantify the divergence between the probability distributions--specifically Mixture of Gaussians--estimated from the given point sets, we use a recently developed information-theoretic measure called Jensen-Renyi (JR) divergence. We evaluate a closed-form JR divergence between multiple probabilistic representations for the general case where the mixture models differ in variance and the number of components. We derive the analytic gradient of the divergence measure with respect to the non-rigid registration parameters, and apply it to numerical optimization of the group-wise registration, leading to a computationally efficient and accurate algorithm. We validate our approach on synthetic data, and evaluate it on 3D cardiac shapes.

Cardiac disease recognition in echocardiograms using spatio-temporal statistical models.

In this paper we present a method of automatic disease recognition by using statistical spatio-temporal disease models in cardiac echo videos. Starting from echo videos of known viewpoints as training data, we form a statistical model of shape and motion information within a cardiac cycle for each disease. Specifically, an active shape model (ASM) is used to model shape and texture information in an echo frame. The motion information derived by tracking ASMs through a heart cycle is then represented compactly using eigen-motion features to constitute a joint spatio-temporal statistical model per disease class and observation viewpoint. Each of these models is then fit to a new cardiac echo video of an unknown disease, and the best fitting model is used to label the disease class. Results are presented that show the method can discriminate patients with hypokinesia from normal patients.

Shape-based matching of ECG recordings.

An electrocardiogram (ECG) is an important and commonly used diagnostic aid in cardiovascular disease diagnosis. Physicians routinely perform diagnosis by a simple visual examination of ECG waveform shapes. In this paper, we address the problem of shape-based retrieval of ECG recordings, both digital and scanned from paper, to infer similarity in diagnosed diseases. Specifically, we use the knowledge of ECG recording structure to segment and extract curves representing various recording channels from ECG images. We then present a method of capturing the perceptual shape similarity of ECG waveforms by combining shape matching with dynamic time warping. The shape similarity of each recording channel is combined to develop an overall shape similarity measure between ECG recordings. Results are presented that demonstrate the method on shape-based matching of various cardiovascular diseases.