COVID-19 and Pneumonia detection and web deployment from CT scan and X-ray images using deep learning.

During the COVID-19 pandemic, pneumonia was the leading cause of respiratory failure and death. In addition to SARS-COV-2, it can be caused by several other bacterial and viral agents. Even today, variants of SARS-COV-2 are endemic and COVID-19 cases are common in many places. The symptoms of COVID-19 are highly diverse and robust, ranging from invisible to severe respiratory failure. Current detection methods for the disease are time-consuming and expensive with low accuracy and precision. To address such situations, we have designed a framework for COVID-19 and Pneumonia detection using multiple deep learning algorithms further accompanied by a deployment scheme. In this study, we have utilized four prominent deep learning models, which are VGG-19, ResNet-50, Inception V3 and Xception, on two separate datasets of CT scan and X-ray images (COVID/Non-COVID) to identify the best models for the detection of COVID-19. We achieved accuracies ranging from 86% to 99% depending on the model and dataset. To further validate our findings, we have applied the four distinct models on two more supplementary datasets of X-ray images of bacterial pneumonia and viral pneumonia. Additionally, we have implemented a flask app to visualize the outcome of our framework to show the identified COVID and Non-COVID images. The findings of this study will be helpful to develop an AI-driven automated tool for the cost effective and faster detection and better management of COVID-19 patients.

Stepwise incremental pretraining for integrating discriminative, restorative, and adversarial learning.

We have developed a United framework that integrates three self-supervised learning (SSL) ingredients (discriminative, restorative, and adversarial learning), enabling collaborative learning among the three learning ingredients and yielding three transferable components: a discriminative encoder, a restorative decoder, and an adversary encoder. To leverage this collaboration, we redesigned nine prominent self-supervised methods, including Rotation, Jigsaw, Rubik's Cube, Deep Clustering, TransVW, MoCo, BYOL, PCRL, and Swin UNETR, and augmented each with its missing components in a United framework for 3D medical imaging. However, such a United framework increases model complexity, making 3D pretraining difficult. To overcome this difficulty, we propose stepwise incremental pretraining, a strategy that unifies the pretraining, in which a discriminative encoder is first trained via discriminative learning, the pretrained discriminative encoder is then attached to a restorative decoder, forming a skip-connected encoder-decoder, for further joint discriminative and restorative learning. Last, the pretrained encoder-decoder is associated with an adversarial encoder for final full discriminative, restorative, and adversarial learning. Our extensive experiments demonstrate that the stepwise incremental pretraining stabilizes United models pretraining, resulting in significant performance gains and annotation cost reduction via transfer learning in six target tasks, ranging from classification to segmentation, across diseases, organs, datasets, and modalities. This performance improvement is attributed to the synergy of the three SSL ingredients in our United framework unleashed through stepwise incremental pretraining. Our codes and pretrained models are available at GitHub.com/JLiangLab/StepwisePretraining.

Seeking an optimal approach for Computer-aided Diagnosis of Pulmonary Embolism.

Pulmonary Embolism (PE) represents a thrombus ("blood clot"), usually originating from a lower extremity vein, that travels to the blood vessels in the lung, causing vascular obstruction and in some patients death. This disorder is commonly diagnosed using Computed Tomography Pulmonary Angiography (CTPA). Deep learning holds great promise for the Computer-aided Diagnosis (CAD) of PE. However, numerous deep learning methods, such as Convolutional Neural Networks (CNN) and Transformer-based models, exist for a given task, causing great confusion regarding the development of CAD systems for PE. To address this confusion, we present a comprehensive analysis of competing deep learning methods applicable to PE diagnosis based on four datasets. First, we use the RSNA PE dataset, which includes (weak) slice-level and exam-level labels, for PE classification and diagnosis, respectively. At the slice level, we compare CNNs with the Vision Transformer (ViT) and the Swin Transformer. We also investigate the impact of self-supervised versus (fully) supervised ImageNet pre-training, and transfer learning over training models from scratch. Additionally, at the exam level, we compare sequence model learning with our proposed transformer-based architecture, Embedding-based ViT (E-ViT). For the second and third datasets, we utilize the CAD-PE Challenge Dataset and Ferdowsi University of Mashad's PE Dataset, where we convert (strong) clot-level masks into slice-level annotations to evaluate the optimal CNN model for slice-level PE classification. Finally, we use our in-house PE-CAD dataset, which contains (strong) clot-level masks. Here, we investigate the impact of our vessel-oriented image representations and self-supervised pre-training on PE false positive reduction at the clot level across image dimensions (2D, 2.5D, and 3D). Our experiments show that (1) transfer learning boosts performance despite differences between photographic images and CTPA scans; (2) self-supervised pre-training can surpass (fully) supervised pre-training; (3) transformer-based models demonstrate comparable performance but slower convergence compared with CNNs for slice-level PE classification; (4) model trained on the RSNA PE dataset demonstrates promising performance when tested on unseen datasets for slice-level PE classification; (5) our E-ViT framework excels in handling variable numbers of slices and outperforms sequence model learning for exam-level diagnosis; and (6) vessel-oriented image representation and self-supervised pre-training both enhance performance for PE false positive reduction across image dimensions. Our optimal approach surpasses state-of-the-art results on the RSNA PE dataset, enhancing AUC by 0.62% (slice-level) and 2.22% (exam-level). On our in-house PE-CAD dataset, 3D vessel-oriented images improve performance from 80.07% to 91.35%, a remarkable 11% gain. Codes are available at GitHub.com/JLiangLab/CAD_PE.

POPAR: Patch Order Prediction and Appearance Recovery for Self-supervised Medical Image Analysis.

Vision transformer-based self-supervised learning (SSL) approaches have recently shown substantial success in learning visual representations from unannotated photographic images. However, their acceptance in medical imaging is still lukewarm, due to the significant discrepancy between medical and photographic images. Consequently, we propose POPAR (patch order prediction and appearance recovery), a novel vision transformer-based self-supervised learning framework for chest X-ray images. POPAR leverages the benefits of vision transformers and unique properties of medical imaging, aiming to simultaneously learn patch-wise high-level contextual features by correcting shuffled patch orders and fine-grained features by recovering patch appearance. We transfer POPAR pretrained models to diverse downstream tasks. The experiment results suggest that (1) POPAR outperforms state-of-the-art (SoTA) self-supervised models with vision transformer backbone; (2) POPAR achieves significantly better performance over all three SoTA contrastive learning methods; and (3) POPAR also outperforms fully-supervised pretrained models across architectures. In addition, our ablation study suggests that to achieve better performance on medical imaging tasks, both fine-grained and global contextual features are preferred. All code and models are available at GitHub.com/JLiangLab/POPAR.

Discriminative, Restorative, and Adversarial Learning: Stepwise Incremental Pretraining.

Uniting three self-supervised learning (SSL) ingredients (discriminative, restorative, and adversarial learning) enables collaborative representation learning and yields three transferable components: a discriminative encoder, a restorative decoder, and an adversary encoder. To leverage this advantage, we have redesigned five prominent SSL methods, including Rotation, Jigsaw, Rubik's Cube, Deep Clustering, and TransVW, and formulated each in a United framework for 3D medical imaging. However, such a United framework increases model complexity and pretraining difficulty. To overcome this difficulty, we develop a stepwise incremental pretraining strategy, in which a discriminative encoder is first trained via discriminative learning, the pretrained discriminative encoder is then attached to a restorative decoder, forming a skip-connected encoder-decoder, for further joint discriminative and restorative learning, and finally, the pretrained encoder-decoder is associated with an adversarial encoder for final full discriminative, restorative, and adversarial learning. Our extensive experiments demonstrate that the stepwise incremental pretraining stabilizes United models training, resulting in significant performance gains and annotation cost reduction via transfer learning for five target tasks, encompassing both classification and segmentation, across diseases, organs, datasets, and modalities. This performance is attributed to the synergy of the three SSL ingredients in our United framework unleashed via stepwise incremental pretraining. All codes and pretrained models are available at GitHub.com/JLiangLab/StepwisePretraining.

Benchmarking and Boosting Transformers for Medical Image Classification.

Visual transformers have recently gained popularity in the computer vision community as they began to outrank convolutional neural networks (CNNs) in one representative visual benchmark after another. However, the competition between visual transformers and CNNs in medical imaging is rarely studied, leaving many important questions unanswered. As the first step, we benchmark how well existing transformer variants that use various (supervised and self-supervised) pre-training methods perform against CNNs on a variety of medical classification tasks. Furthermore, given the data-hungry nature of transformers and the annotation-deficiency challenge of medical imaging, we present a practical approach for bridging the domain gap between photographic and medical images by utilizing unlabeled large-scale in-domain data. Our extensive empirical evaluations reveal the following insights in medical imaging: (1) good initialization is more crucial for transformer-based models than for CNNs, (2) self-supervised learning based on masked image modeling captures more generalizable representations than supervised models, and (3) assembling a larger-scale domain-specific dataset can better bridge the domain gap between photographic and medical images via self-supervised continuous pre-training. We hope this benchmark study can direct future research on applying transformers to medical imaging analysis. All codes and pre-trained models are available on our GitHub page https://github.com/JLiangLab/BenchmarkTransformers.

Seeking an Optimal Approach for Computer-Aided Pulmonary Embolism Detection.

Pulmonary embolism (PE) represents a thrombus ("blood clot"), usually originating from a lower extremity vein, that travels to the blood vessels in the lung, causing vascular obstruction and in some patients, death. This disorder is commonly diagnosed using CT pulmonary angiography (CTPA). Deep learning holds great promise for the computer-aided CTPA diagnosis (CAD) of PE. However, numerous competing methods for a given task in the deep learning literature exist, causing great confusion regarding the development of a CAD PE system. To address this confusion, we present a comprehensive analysis of competing deep learning methods applicable to PE diagnosis using CTPA at the both image and exam levels. At the image level, we compare convolutional neural networks (CNNs) with vision transformers, and contrast self-supervised learning (SSL) with supervised learning, followed by an evaluation of transfer learning compared with training from scratch. At the exam level, we focus on comparing conventional classification (CC) with multiple instance learning (MIL). Our extensive experiments consistently show: (1) transfer learning consistently boosts performance despite differences between natural images and CT scans, (2) transfer learning with SSL surpasses its supervised counterparts; (3) CNNs outperform vision transformers, which otherwise show satisfactory performance; and (4) CC is, surprisingly, superior to MIL. Compared with the state of the art, our optimal approach provides an AUC gain of 0.2% and 1.05% for image-level and exam-level, respectively.

Ambulatory blood pressure monitoring and peri-hemodialysis blood pressures in a southeast US hemodialysis unit.

AIMS: End-stage renal disease (ESRD) patients on dialysis are perceived to have difficult-to-control blood pressure (BP) and commonly treated with complex antihypertensive regimens. Our hypothesis was that peri-dialysis BP will overestimate the true burden of hypertension in these patients. SUBJECTS AND METHODS: We performed 44-h ambulatory blood pressure monitoring (ABPM) in 43 patients recruited from the University of Mississippi outpatient dialysis unit. Data collected included routine peri-dialysis blood systolic blood pressure (SBP), diastolic blood pressure (DBP), weight gain, and demographic information. We investigated whether the pre-dialysis or post-dialysis blood pressure would better correspond to the ABPM results. RESULTS: The mean age of participants was 50.5 ± 12.05 years, 95% African-American, and 44% diabetic with an average dialysis vintage of 31.1 ± 30 months. The mean SBP and DBP were 164.6/87.9 mmHg ± 22.3/15 before dialysis, 151.5/81.3 mmHg ± 24.1/13 after dialysis and 136/80.6 mmHg ± 23.5/14.7 during ABPM. There were wide limits of agreements between peri-dialysis BP and ABPM, the largest with pre-dialysis SBP (28.5 ± 16.6 mmHg) and the least with post-dialysis DBP (0.7 ± 10 mmHg). With both peri-dialysis BP measurements as explanatory variables in a linear regression model, only the post-dialysis SBP (ß 0.716; p < 0.001) but not pre-dialysis SBP (ß 0.157; p = 0.276) had a significant independent association with ABPM systolic BP. For DBP, both pre-dialysis (ß 0.543; p = 0.001) and post-dialysis (ß 0.317; p = 0.037) values retained correlation with DBP on ABPM. CONCLUSION: Peri-dialysis measurements overestimated true BP burden in this Southeastern U.S. cohort of ESRD patients. When BP readings from outside the dialysis unit are notavailable, assessment of BP control should focus pre-dialysis on DBP and post-dialysison both SBP and DBP.

Physical activity assessment: validation of a clinical assessment tool.

BACKGROUND: Physicians report that they fail to counsel patients about physical activity due to a lack of practical tools, time, reimbursement, knowledge, and confidence. This paper reports concurrent and criterion validation of the Physical Activity Assessment Tool (PAAT), designed to rapidly assess patient physical activity in clinical settings and reduce time for assessment, and thus to facilitate counseling. METHODS: Adult volunteers (n=68) completed the PAAT and International Physical Activity Questionnaire-Long Form (IPAQ-Long) twice and wore a Manufacturing Technology, Inc. (MTI) accelerometer for 14 days in 2003. Continuous and categorical measures of physical activity by PAAT were compared to MTI accelerometer and IPAQ-Long in analyses conducted in 2003 to 2006. Consistent with national recommendations, participants were classified as active if they accumulated more than 150 minutes per week of moderate to vigorous physical activity (MVPA) or more than 60 minutes per week of vigorous physical activity. RESULTS: The PAAT was significantly correlated with the IPAQ (r=0.562, p<0.001) and MTI (r=0.392, p=0.015) for MVPA. Seven-day test-retest reliability was comparable for PAAT (r=0.618, p<0.001) and MTI (r=0.527, p<0.001). PAAT classified participants as "active" or "under-active" concordantly with MTI for 69.8% of participants and with IPAQ for 66.7%; strength of agreement was fair (kappa=0.338 and 0.212, respectively). The PAAT classified fewer participants as active than either the MTI (p=0.169) or IPAQ (p<0.001), and measured physical activity more like the direct objective measure (MTI) than did IPAQ. CONCLUSIONS: The concurrent and criterion validity of the PAAT are comparable to self-report instruments used in epidemiologic research.