A multimodal integration pipeline for accurate diagnosis, pathogen identification, and prognosis prediction of pulmonary infections.

Pulmonary infections pose formidable challenges in clinical settings with high mortality rates across all age groups worldwide. Accurate diagnosis and early intervention are crucial to improve patient outcomes. Artificial intelligence (AI) has the capability to mine imaging features specific to different pathogens and fuse multimodal features to reach a synergistic diagnosis, enabling more precise investigation and individualized clinical management. In this study, we successfully developed a multimodal integration (MMI) pipeline to differentiate among bacterial, fungal, and viral pneumonia and pulmonary tuberculosis based on a real-world dataset of 24,107 patients. The area under the curve (AUC) of the MMI system comprising clinical text and computed tomography (CT) image scans yielded 0.910 (95% confidence interval [CI]: 0.904-0.916) and 0.887 (95% CI: 0.867-0.909) in the internal and external testing datasets respectively, which were comparable to those of experienced physicians. Furthermore, the MMI system was utilized to rapidly differentiate between viral subtypes with a mean AUC of 0.822 (95% CI: 0.805-0.837) and bacterial subtypes with a mean AUC of 0.803 (95% CI: 0.775-0.830). Here, the MMI system harbors the potential to guide tailored medication recommendations, thus mitigating the risk of antibiotic misuse. Additionally, the integration of multimodal factors in the AI-driven system also provided an evident advantage in predicting risks of developing critical illness, contributing to more informed clinical decision-making. To revolutionize medical care, embracing multimodal AI tools in pulmonary infections will pave the way to further facilitate early intervention and precise management in the foreseeable future.

Automated segmentation for early detection of uveal melanoma.

OBJECTIVE: Uveal melanoma is the most common intraocular malignancy in adults. Current screening and triaging methods for melanocytic choroidal tumours face inherent limitations, particularly in regions with limited access to specialized ocular oncologists. This study explores the potential of machine learning to automate tumour segmentation. We develop and evaluate a machine-learning model for lesion segmentation using ultra-wide-field fundus photography. METHOD: A retrospective chart review was conducted of patients diagnosed with uveal melanoma, choroidal nevi, or congenital hypertrophy of the retinal pigmented epithelium at a tertiary academic medical centre. Included patients had a single ultra-wide-field fundus photograph (Optos PLC, Dunfermline, Fife, Scotland) of adequate quality to visualize the lesion of interest, as confirmed by a single ocular oncologist. These images were used to develop and test a machine-learning algorithm for lesion segmentation. RESULTS: A total of 396 images were used to develop a machine-learning algorithm for lesion segmentation. Ninety additional images were used in the testing data set along with images of 30 healthy control individuals. Of the images with successfully detected lesions, the machine-learning segmentation yielded Dice coefficients of 0.86, 0.81, and 0.85 for uveal melanoma, choroidal nevi, and congenital hypertrophy of the retinal pigmented epithelium, respectively. Sensitivities for any lesion detection per image were 1.00, 0.90, and 0.87, respectively. For images without lesions, specificity was 0.93. CONCLUSION: Our study demonstrates a novel machine-learning algorithm's performance, suggesting its potential clinical utility as a widely accessible method of screening choroidal tumours. Additional evaluation methods are necessary to further enhance the model's lesion classification and diagnostic accuracy.

The long-term health outcomes, pathophysiological mechanisms and multidisciplinary management of long COVID.

There have been hundreds of millions of cases of coronavirus disease 2019 (COVID-19), which is caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). With the growing population of recovered patients, it is crucial to understand the long-term consequences of the disease and management strategies. Although COVID-19 was initially considered an acute respiratory illness, recent evidence suggests that manifestations including but not limited to those of the cardiovascular, respiratory, neuropsychiatric, gastrointestinal, reproductive, and musculoskeletal systems may persist long after the acute phase. These persistent manifestations, also referred to as long COVID, could impact all patients with COVID-19 across the full spectrum of illness severity. Herein, we comprehensively review the current literature on long COVID, highlighting its epidemiological understanding, the impact of vaccinations, organ-specific sequelae, pathophysiological mechanisms, and multidisciplinary management strategies. In addition, the impact of psychological and psychosomatic factors is also underscored. Despite these crucial findings on long COVID, the current diagnostic and therapeutic strategies based on previous experience and pilot studies remain inadequate, and well-designed clinical trials should be prioritized to validate existing hypotheses. Thus, we propose the primary challenges concerning biological knowledge gaps and efficient remedies as well as discuss the corresponding recommendations.

Predicting gene mutation status via artificial intelligence technologies based on multimodal integration (MMI) to advance precision oncology.

Personalized treatment strategies for cancer frequently rely on the detection of genetic alterations which are determined by molecular biology assays. Historically, these processes typically required single-gene sequencing, next-generation sequencing, or visual inspection of histopathology slides by experienced pathologists in a clinical context. In the past decade, advances in artificial intelligence (AI) technologies have demonstrated remarkable potential in assisting physicians with accurate diagnosis of oncology image-recognition tasks. Meanwhile, AI techniques make it possible to integrate multimodal data such as radiology, histology, and genomics, providing critical guidance for the stratification of patients in the context of precision therapy. Given that the mutation detection is unaffordable and time-consuming for a considerable number of patients, predicting gene mutations based on routine clinical radiological scans or whole-slide images of tissue with AI-based methods has become a hot issue in actual clinical practice. In this review, we synthesized the general framework of multimodal integration (MMI) for molecular intelligent diagnostics beyond standard techniques. Then we summarized the emerging applications of AI in the prediction of mutational and molecular profiles of common cancers (lung, brain, breast, and other tumor types) pertaining to radiology and histology imaging. Furthermore, we concluded that there truly exist multiple challenges of AI techniques in the way of its real-world application in the medical field, including data curation, feature fusion, model interpretability, and practice regulations. Despite these challenges, we still prospect the clinical implementation of AI as a highly potential decision-support tool to aid oncologists in future cancer treatment management.

Predicting Malignant Transformation of Choroidal Nevi Using Machine Learning.

Objective: This study aims to assess a machine learning (ML) algorithm using multimodal imaging to accurately identify risk factors for uveal melanoma (UM) and aid in the diagnosis of melanocytic choroidal tumors. Subjects and Methods: This study included 223 eyes from 221 patients with melanocytic choroidal lesions seen at the eye clinic of the University of Illinois at Chicago between 01/2010 and 07/2022. An ML algorithm was developed and trained on ultra-widefield fundus imaging and B-scan ultrasonography to detect risk factors of malignant transformation of choroidal lesions into UM. The risk factors were verified using all multimodal imaging available from the time of diagnosis. We also explore classification of lesions into UM and choroidal nevi using the ML algorithm. Results: The ML algorithm assessed features of ultra-widefield fundus imaging and B-scan ultrasonography to determine the presence of the following risk factors for malignant transformation: lesion thickness, subretinal fluid, orange pigment, proximity to optic nerve, ultrasound hollowness, and drusen. The algorithm also provided classification of lesions into UM and choroidal nevi. A total of 115 patients with choroidal nevi and 108 patients with UM were included. The mean lesion thickness for choroidal nevi was 1.6 mm and for UM was 5.9 mm. Eleven ML models were implemented and achieved high accuracy, with an area under the curve of 0.982 for thickness prediction and 0.964 for subretinal fluid prediction. Sensitivity/specificity values ranged from 0.900/0.818 to 1.000/0.727 for different features. The ML algorithm demonstrated high accuracy in identifying risk factors and differentiating lesions based on the analyzed imaging data. Conclusions: This study provides proof of concept that ML can accurately identify risk factors for malignant transformation in melanocytic choroidal tumors based on a single ultra-widefield fundus image or B-scan ultrasound at the time of initial presentation. By leveraging the efficiency and availability of ML, this study has the potential to provide a non-invasive tool that helps to prevent unnecessary treatment, improve our ability to predict malignant transformation, reduce the risk of metastasis, and potentially save patient lives.

Radiogenomic System for Non-Invasive Identification of Multiple Actionable Mutations and PD-L1 Expression in Non-Small Cell Lung Cancer Based on CT Images.

PURPOSE: Personalized treatments such as targeted therapy and immunotherapy have revolutionized the predominantly therapeutic paradigm for non-small cell lung cancer (NSCLC). However, these treatment decisions require the determination of targetable genomic and molecular alterations through invasive genetic or immunohistochemistry (IHC) tests. Numerous previous studies have demonstrated that artificial intelligence can accurately predict the single-gene status of tumors based on radiologic imaging, but few studies have achieved the simultaneous evaluation of multiple genes to reflect more realistic clinical scenarios. METHODS: We proposed a multi-label multi-task deep learning (MMDL) system for non-invasively predicting actionable NSCLC mutations and PD-L1 expression utilizing routinely acquired computed tomography (CT) images. This radiogenomic system integrated transformer-based deep learning features and radiomic features of CT volumes from 1096 NSCLC patients based on next-generation sequencing (NGS) and IHC tests. RESULTS: For each task cohort, we randomly split the corresponding dataset into training (80%), validation (10%), and testing (10%) subsets. The area under the receiver operating characteristic curves (AUCs) of the MMDL system achieved 0.862 (95% confidence interval (CI), 0.758-0.969) for discrimination of a panel of 8 mutated genes, including EGFR, ALK, ERBB2, BRAF, MET, ROS1, RET and KRAS, 0.856 (95% CI, 0.663-0.948) for identification of a 10-molecular status panel (previous 8 genes plus TP53 and PD-L1); and 0.868 (95% CI, 0.641-0.972) for classifying EGFR / PD-L1 subtype, respectively. CONCLUSIONS: To the best of our knowledge, this study is the first deep learning system to simultaneously analyze 10 molecular expressions, which might be utilized as an assistive tool in conjunction with or in lieu of ancillary testing to support precision treatment options.

Immune regulation and emerging roles of noncoding RNAs in Mycobacterium tuberculosis infection.

Tuberculosis, caused by Mycobacterium tuberculosis, engenders an onerous burden on public hygiene. Congenital and adaptive immunity in the human body act as robust defenses against the pathogens. However, in coevolution with humans, this microbe has gained multiple lines of mechanisms to circumvent the immune response to sustain its intracellular persistence and long-term survival inside a host. Moreover, emerging evidence has revealed that this stealthy bacterium can alter the expression of demic noncoding RNAs (ncRNAs), leading to dysregulated biological processes subsequently, which may be the rationale behind the pathogenesis of tuberculosis. Meanwhile, the differential accumulation in clinical samples endows them with the capacity to be indicators in the time of tuberculosis suffering. In this article, we reviewed the nearest insights into the impact of ncRNAs during Mycobacterium tuberculosis infection as realized via immune response modulation and their potential as biomarkers for the diagnosis, drug resistance identification, treatment evaluation, and adverse drug reaction prediction of tuberculosis, aiming to inspire novel and precise therapy development to combat this pathogen in the future.

The Application of Artificial Intelligence in the Diagnosis and Drug Resistance Prediction of Pulmonary Tuberculosis.

With the increasing incidence and mortality of pulmonary tuberculosis, in addition to tough and controversial disease management, time-wasting and resource-limited conventional approaches to the diagnosis and differential diagnosis of tuberculosis are still awkward issues, especially in countries with high tuberculosis burden and backwardness. In the meantime, the climbing proportion of drug-resistant tuberculosis poses a significant hazard to public health. Thus, auxiliary diagnostic tools with higher efficiency and accuracy are urgently required. Artificial intelligence (AI), which is not new but has recently grown in popularity, provides researchers with opportunities and technical underpinnings to develop novel, precise, rapid, and automated implements for pulmonary tuberculosis care, including but not limited to tuberculosis detection. In this review, we aimed to introduce representative AI methods, focusing on deep learning and radiomics, followed by definite descriptions of the state-of-the-art AI models developed using medical images and genetic data to detect pulmonary tuberculosis, distinguish the infection from other pulmonary diseases, and identify drug resistance of tuberculosis, with the purpose of assisting physicians in deciding the appropriate therapeutic schedule in the early stage of the disease. We also enumerated the challenges in maximizing the impact of AI in this field such as generalization and clinical utility of the deep learning models.

Development and validation of an abnormality-derived deep-learning diagnostic system for major respiratory diseases.

Respiratory diseases impose a tremendous global health burden on large patient populations. In this study, we aimed to develop DeepMRDTR, a deep learning-based medical image interpretation system for the diagnosis of major respiratory diseases based on the automated identification of a wide range of radiological abnormalities through computed tomography (CT) and chest X-ray (CXR) from real-world, large-scale datasets. DeepMRDTR comprises four networks (two CT-Nets and two CXR-Nets) that exploit contrastive learning to generate pre-training parameters that are fine-tuned on the retrospective dataset collected from a single institution. The performance of DeepMRDTR was evaluated for abnormality identification and disease diagnosis on data from two different institutions: one was an internal testing dataset from the same institution as the training data and the second was collected from an external institution to evaluate the model generalizability and robustness to an unrelated population dataset. In such a difficult multi-class diagnosis task, our system achieved the average area under the receiver operating characteristic curve (AUC) of 0.856 (95% confidence interval (CI):0.843-0.868) and 0.841 (95%CI:0.832-0.887) for abnormality identification, and 0.900 (95%CI:0.872-0.958) and 0.866 (95%CI:0.832-0.887) for major respiratory diseases' diagnosis on CT and CXR datasets, respectively. Furthermore, to achieve a clinically actionable diagnosis, we deployed a preliminary version of DeepMRDTR into the clinical workflow, which was performed on par with senior experts in disease diagnosis, with an AUC of 0.890 and a Cohen's k of 0.746-0.877 at a reasonable timescale; these findings demonstrate the potential to accelerate the medical workflow to facilitate early diagnosis as a triage tool for respiratory diseases which supports improved clinical diagnoses and decision-making.

Light color efficiency-balanced trans-palpebral illumination for widefield fundus photography of the retina and choroid.

A wide-field fundus camera, which can selectively evaluate the retina and choroid, is desirable for better detection and treatment evaluation of eye diseases. Trans-palpebral illumination has been demonstrated for wide-field fundus photography, but its application for true-color retinal imaging is challenging due to the light efficiency delivered through the eyelid and sclera is highly wavelength dependent. This study is to test the feasibility of true-color retinal imaging using efficiency-balanced visible light illumination, and to validate multiple spectral imaging (MSI) of the retina and choroid. 530 nm, 625 nm, 780 nm and 970 nm light emission diodes (LED)s are used to quantitatively evaluate the spectral efficiency of the trans-palpebral illumination. In comparison with 530 nm illumination, the 625 nm, 780 nm and 970 nm light efficiencies are 30.25, 523.05, and 1238.35 times higher. The light efficiency-balanced 530 nm and 625 nm illumination control can be used to produce true-color retinal image with contrast enhancement. The 780 nm light image enhances the visibility of choroidal vasculature, and the 970 nm image is predominated by large veins in the choroid. Without the need of pharmacological pupillary dilation, a 140° eye-angle field of view (FOV) is demonstrated in a snapshot fundus image. In coordination with a fixation target, the FOV can be readily expanded over the equator of the eye to visualize vortex ampullas.