RESUMEN
The ability to recover tissue deformation from visual features is fundamental for many robotic surgery applications. This has been a long-standing research topic in computer vision, however, is still unsolved due to complex dynamics of soft tissues when being manipulated by surgical instruments. The ambiguous pixel correspondence caused by homogeneous texture makes achieving dense and accurate tissue tracking even more challenging. In this paper, we propose a novel self-supervised framework to recover tissue deformations from stereo surgical videos. Our approach integrates semantics, cross-frame motion flow, and long-range temporal dependencies to enable the recovered deformations to represent actual tissue dynamics. Moreover, we incorporate diffeomorphic mapping to regularize the warping field to be physically realistic. To comprehensively evaluate our method, we collected stereo surgical video clips containing three types of tissue manipulation (i.e., pushing, dissection and retraction) from two different types of surgeries (i.e., hemicolectomy and mesorectal excision). Our method has achieved impressive results in capturing deformation in 3D mesh, and generalized well across manipulations and surgeries. It also outperforms current state-of-the-art methods on non-rigid registration and optical flow estimation. To the best of our knowledge, this is the first work on self-supervised learning for dense tissue deformation modeling from stereo surgical videos. Our code will be released.
RESUMEN
Histopathological tissue classification is a fundamental task in computational pathology. Deep learning (DL)-based models have achieved superior performance but centralized training suffers from the privacy leakage problem. Federated learning (FL) can safeguard privacy by keeping training samples locally, while existing FL-based frameworks require a large number of well-annotated training samples and numerous rounds of communication which hinder their viability in real-world clinical scenarios. In this article, we propose a lightweight and universal FL framework, named federated deep-broad learning (FedDBL), to achieve superior classification performance with limited training samples and only one-round communication. By simply integrating a pretrained DL feature extractor, a fast and lightweight broad learning inference system with a classical federated aggregation approach, FedDBL can dramatically reduce data dependency and improve communication efficiency. Five-fold cross-validation demonstrates that FedDBL greatly outperforms the competitors with only one-round communication and limited training samples, while it even achieves comparable performance with the ones under multiple-round communications. Furthermore, due to the lightweight design and one-round communication, FedDBL reduces the communication burden from 4.6 GB to only 138.4 KB per client using the ResNet-50 backbone at 50-round training. Extensive experiments also show the scalability of FedDBL on model generalization to the unseen dataset, various client numbers, model personalization and other image modalities. Since no data or deep model sharing across different clients, the privacy issue is well-solved and the model security is guaranteed with no model inversion attack risk. Code is available at https://github.com/tianpeng-deng/FedDBL.
RESUMEN
Background: The preoperative conversion therapy for advanced hepatocellular carcinoma (HCC) is still being explored. This study reported the potential of combination of transarterial chemoembolization (TACE), hepatic arterial infusion chemotherapy (HAIC), programmed cell death protein-1 (PD-1) inhibitors and lenvatinib as preoperative conversion therapy for nonmetastatic advanced HCC. Methods: This retrospective study gathered data on patients with nonmetastatic advanced HCC who received this combination therapy. We used drug-eluting bead (DEB) instead of conventional iodized oil in TACE. The clinical data, conversion rate, adverse events (AEs) and short-term survival were summarized. A stratified analysis based on whether or not the patient received surgery was conducted. Results: A total of 28 patients were included in the analysis. No grade 4 AEs were observed. The overall objective response rate (ORR) was 64.3%. Ten (35.7%) patients eventually received R0 resection after 2 cycles of combination therapy. Patients succeeding to resection (surgery group) had significantly higher ORR (90.0% vs. 50.0%, P=0.048). The proportion of patients with alpha-fetoprotein (AFP) >1,000 µg/L was significantly lower in surgery group (10.0% vs. 66.7%, P=0.006). After combination therapy, more patients in surgery group experienced significant reduction of >90% in AFP levels (75.0% vs. 23.1%, P=0.03), as well as standardized uptake value (SUV) of 18F-fluorodeoxyglucose (18F-FDG) both in primary tumors and portal vein tumor thrombosis (PVTT) (60.0% vs. 5.6%, P=0.003; 57.1% vs. 8.3%, P=0.04). Of note, 85.7% of PVTT exhibited major pathological response (MPR) in pathological examination although only 28.6% achieved downstage in preoperative imaging examination. MPR was more commonly observed in PVTT than in main tumors (85.7% vs. 20.0%). In non-surgery group, the median overall survival (OS) was 7 months with a 1-year survival rate of 27.8%, while in surgery group, the median OS was not reached and 1-year survival rate was 60.0%. Conclusions: The combination of TACE-HAIC, PD-1 inhibitors and lenvatinib showed its benefit as a preoperative conversion therapy for nonmetastatic advanced HCC. In addition to imaging evaluation, significant reduction of 18F-FDG uptake and AFP can be used as predictors of successful conversion, especially for PVTT.
RESUMEN
Intraoperative imaging techniques for reconstructing deformable tissues in vivo are pivotal for advanced surgical systems. Existing methods either compromise on rendering quality or are excessively computationally intensive, often demanding dozens of hours to perform, which significantly hinders their practical application. In this paper, we introduce Fast Orthogonal Plane (Forplane), a novel, efficient framework based on neural radiance fields (NeRF) for the reconstruction of deformable tissues. We conceptualize surgical procedures as 4D volumes, and break them down into static and dynamic fields comprised of orthogonal neural planes. This factorization discretizes the four-dimensional space, leading to a decreased memory usage and faster optimization. A spatiotemporal importance sampling scheme is introduced to improve performance in regions with tool occlusion as well as large motions and accelerate training. An efficient ray marching method is applied to skip sampling among empty regions, significantly improving inference speed. Forplane accommodates both binocular and monocular endoscopy videos, demonstrating its extensive applicability and flexibility. Our experiments, carried out on two in vivo datasets, the EndoNeRF and Hamlyn datasets, demonstrate the effectiveness of our framework. In all cases, Forplane substantially accelerates both the optimization process (by over 100 times) and the inference process (by over 15 times) while maintaining or even improving the quality across a variety of non-rigid deformations. This significant performance improvement promises to be a valuable asset for future intraoperative surgical applications. The code of our project is now available at https://github.com/Loping151/ForPlane.
RESUMEN
PURPOSE: The healthcare industry has a growing need for realistic modeling and efficient simulation of surgical scenes. With effective models of deformable surgical scenes, clinicians are able to conduct surgical planning and surgery training on scenarios close to real-world cases. However, a significant challenge in achieving such a goal is the scarcity of high-quality soft tissue models with accurate shapes and textures. To address this gap, we present a data-driven framework that leverages emerging neural radiance field technology to enable high-quality surgical reconstruction and explore its application for surgical simulations. METHOD: We first focus on developing a fast NeRF-based surgical scene 3D reconstruction approach that achieves state-of-the-art performance. This method can significantly outperform traditional 3D reconstruction methods, which have failed to capture large deformations and produce fine-grained shapes and textures. We then propose an automated creation pipeline of interactive surgical simulation environments through a closed mesh extraction algorithm. RESULTS: Our experiments have validated the superior performance and efficiency of our proposed approach in surgical scene 3D reconstruction. We further utilize our reconstructed soft tissues to conduct FEM and MPM simulations, showcasing the practical application of our method in data-driven surgical simulations. CONCLUSION: We have proposed a novel NeRF-based reconstruction framework with an emphasis on simulation purposes. Our reconstruction framework facilitates the efficient creation of high-quality surgical soft tissue 3D models. With multiple soft tissue simulations demonstrated, we show that our work has the potential to benefit downstream clinical tasks, such as surgical education.
Asunto(s)
Simulación por Computador , Imagenología Tridimensional , Humanos , Imagenología Tridimensional/métodos , Algoritmos , Cirugía Asistida por Computador/métodosRESUMEN
Aneurysmal subarachnoid hemorrhage is a medical emergency of brain that has high mortality and poor prognosis. Causal effect estimation of treatment strategies on patient outcomes is crucial for aneurysmal subarachnoid hemorrhage treatment decision-making. However, most existing studies on treatment decision-making support of this disease are unable to simultaneously compare the potential outcomes of different treatments for a patient. Furthermore, these studies fail to harmoniously integrate the imaging data with non-imaging clinical data, both of which are useful in clinical scenarios. In this paper, we estimate the causal effect of various treatments on patients with aneurysmal subarachnoid hemorrhage by integrating plain CT with non-imaging clinical data, which is represented using structured tabular data. Specifically, we first propose a novel scheme that uses multi-modality confounders distillation architecture to predict the treatment outcome and treatment assignment simultaneously. With these distilled confounder features, we design an imaging and non-imaging interaction representation learning strategy to use the complementary information extracted from different modalities to balance the feature distribution of different treatment groups. We have conducted extensive experiments using a clinical dataset of 656 subarachnoid hemorrhage cases, which was collected from the Hospital Authority Data Collaboration Laboratory in Hong Kong. Our method shows consistent improvements on the evaluation metrics of treatment effect estimation, achieving state-of-the-art results over strong competitors. Code is released at https://github.com/med-air/TOP-aSAH.
Asunto(s)
Hemorragia Subaracnoidea , Humanos , Hemorragia Subaracnoidea/diagnóstico por imagen , Tomografía Computarizada por Rayos X/métodos , Bases de Datos Factuales , Algoritmos , Sistemas de Apoyo a Decisiones ClínicasRESUMEN
BACKGROUND & AIMS: Benign ulcerative colorectal diseases (UCDs) such as ulcerative colitis, Crohn's disease, ischemic colitis, and intestinal tuberculosis share similar phenotypes with different etiologies and treatment strategies. To accurately diagnose closely related diseases like UCDs, we hypothesize that contextual learning is critical in enhancing the ability of the artificial intelligence models to differentiate the subtle differences in lesions amidst the vastly divergent spatial contexts. METHODS: White-light colonoscopy datasets of patients with confirmed UCDs and healthy controls were retrospectively collected. We developed a Multiclass Contextual Classification (MCC) model that can differentiate among the mentioned UCDs and healthy controls by incorporating the tissue object contexts surrounding the individual lesion region in a scene and spatial information from other endoscopic frames (video-level) into a unified framework. Internal and external datasets were used to validate the model's performance. RESULTS: Training datasets included 762 patients, and the internal and external testing cohorts included 257 patients and 293 patients, respectively. Our MCC model provided a rapid reference diagnosis on internal test sets with a high averaged area under the receiver operating characteristic curve (image-level: 0.950 and video-level: 0.973) and balanced accuracy (image-level: 76.1% and video-level: 80.8%), which was superior to junior endoscopists (accuracy: 71.8%, P < .0001) and similar to experts (accuracy: 79.7%, P = .732). The MCC model achieved an area under the receiver operating characteristic curve of 0.988 and balanced accuracy of 85.8% using external testing datasets. CONCLUSIONS: These results enable this model to fit in the routine endoscopic workflow, and the contextual framework to be adopted for diagnosing other closely related diseases.
Asunto(s)
Inteligencia Artificial , Colitis Ulcerosa , Colonoscopía , Humanos , Colitis Ulcerosa/diagnóstico , Estudios Retrospectivos , Femenino , Masculino , Persona de Mediana Edad , Adulto , Interpretación de Imagen Asistida por Computador/métodos , Curva ROC , Anciano , Reproducibilidad de los Resultados , Colon/patología , Colon/diagnóstico por imagen , Valor Predictivo de las Pruebas , Diagnóstico Diferencial , Grabación en Video , Aprendizaje Automático , Estudios de Casos y ControlesRESUMEN
Background: Gallbladder cancer (GBC) is different from other biliary tract cancers in terms of molecular phenotype and microenvironment. Specific treatments for GBC need to be urgently explored. This study preliminarily investigated the clinical value of hepatic artery infusion chemotherapy (HAIC) combined with bevacizumab plus a programmed death receptor-1 (PD-1) inhibitor for treatment of GBC with hepatic oligometastasis. Methods: We retrospectively collected data on GBC patients with hepatic oligometastasis, who received this combination therapy. The clinical data, conversion rate, treatment response, adverse events (AEs), and short-term survival were summarized. The responses of primary gallbladder lesions and hepatic metastasis, and their effect on prognosis, were investigated. Results: A total of 27 patients were included in the analysis. No grade 4 AEs were observed. The overall objective response rate (ORR) was 55.6% and the disease control rate (DCR) was 85.2%. Median overall survival (OS) time was 15.0 months and the 1-year survival rate was 64.0%. Median progression-free survival (PFS) time was 7.0 months and the 1-year PFS rate was 16.2%. Six patients (22.2%) were successfully converted to resection. Compared with primary gallbladder lesions, it appeared more difficult for patients with hepatic metastasis to achieve remission (ORR: 40.7% vs. 77.8%; P=0.012), but its response appeared to be closely related to the prognosis [median OS: 16.0 months in the complete response (CR) or partial response (PR) group vs. 11.0 months in the stable disease (SD) or progressive disease (PD) group, P=0.070; median PFS: 12.0 months in the CR or PR group vs. 6.5 months in the SD or PD group, P<0.001]. Preoperative CA19-9 of >1,900 U/mL and >5 cm metastatic lesions were associated with an unsatisfactory response, whereas a significant decrease of 18F-fluorodeoxyglucose (18F-FDG) uptake may be a marker of tumor remission. Conclusions: The combination of HAIC, a PD-1 inhibitor, and bevacizumab shows potential for advanced GBC with hepatic oligometastasis. The therapeutic response of hepatic metastasis had a greater influence on prognosis than that of primary gallbladder lesions.
RESUMEN
Automated classification of breast cancer subtypes from digital pathology images has been an extremely challenging task due to the complicated spatial patterns of cells in the tissue micro-environment. While newly proposed graph transformers are able to capture more long-range dependencies to enhance accuracy, they largely ignore the topological connectivity between graph nodes, which is nevertheless critical to extract more representative features to address this difficult task. In this paper, we propose a novel connectivity-aware graph transformer (CGT) for phenotyping the topology connectivity of the tissue graph constructed from digital pathology images for breast cancer classification. Our CGT seamlessly integrates connectivity embedding to node feature at every graph transformer layer by using local connectivity aggregation, in order to yield more comprehensive graph representations to distinguish different breast cancer subtypes. In light of the realistic intercellular communication mode, we then encode the spatial distance between two arbitrary nodes as connectivity bias in self-attention calculation, thereby allowing the CGT to distinctively harness the connectivity embedding based on the distance of two nodes. We extensively evaluate the proposed CGT on a large cohort of breast carcinoma digital pathology images stained by Haematoxylin & Eosin. Experimental results demonstrate the effectiveness of our CGT, which outperforms state-of-the-art methods by a large margin. Codes are released on https://github.com/wang-kang-6/CGT.
Asunto(s)
Algoritmos , Neoplasias de la Mama , Interpretación de Imagen Asistida por Computador , Humanos , Neoplasias de la Mama/diagnóstico por imagen , Neoplasias de la Mama/patología , Neoplasias de la Mama/clasificación , Femenino , Interpretación de Imagen Asistida por Computador/métodos , Mama/diagnóstico por imagen , Mama/patologíaRESUMEN
Importance: Intracerebral hemorrhage (ICH) associated with direct oral anticoagulant (DOAC) use carries extremely high morbidity and mortality. The clinical effectiveness of hemostatic therapy is unclear. Objective: To compare the clinical and radiological outcomes of DOAC-associated ICH treated with prothrombin complex concentrate (PCC) vs conservative management. Design, Setting, and Participants: In this population-based, propensity score-weighted retrospective cohort study, patients who developed DOAC-associated ICH from January 1, 2016, to December 31, 2021, in Hong Kong were identified. The outcomes of patients who received 25 to 50 IU/kg PCC with those who received no hemostatic agents were compared. Data were analyzed from May 1, 2022, to June 30, 2023. Main Outcomes and Measures: The primary outcome was modified Rankin scale of 0 to 3 or returning to baseline functional status at 3 months. Secondary outcomes were mortality at 90 days, in-hospital mortality, and hematoma expansion. Weighted logistic regression was performed to evaluate the association of PCC with study outcomes. In unweighted logistic regression models, factors associated with good neurological outcome and hematoma expansion in DOAC-associated ICH were identified. Results: A total of 232 patients with DOAC-associated ICH, with a mean (SD) age of 77.2 (9.3) years and 101 (44%) female patients, were included. Among these, 116 (50%) received conservative treatment and 102 (44%) received PCC. Overall, 74 patients (31%) patients had good neurological recovery and 92 (39%) died within 90 days. Median (IQR) baseline hematoma volume was 21.7 mL (3.6-66.1 mL). Compared with conservative management, PCC was not associated with improved neurological recovery (adjusted odds ratio [aOR], 0.62; 95% CI, 0.33-1.16; P = .14), mortality at 90 days (aOR, 1.03; 95% CI, 0.70-1.53; P = .88), in-hospital mortality (aOR, 1.11; 95% CI, 0.69-1.79; P = .66), or reduced hematoma expansion (aOR, 0.94; 95% CI, 0.38-2.31; P = .90). Higher baseline hematoma volume, lower Glasgow coma scale, and intraventricular hemorrhage were associated with lower odds of good neurological outcome but not hematoma expansion. Conclusions and Relevance: In this cohort study, Chinese patients with DOAC-associated ICH had large baseline hematoma volumes and high rates of mortality and functional disability. PCC treatment was not associated with improved functional outcome, hematoma expansion, or mortality. Further studies on novel hemostatic agents as well as neurosurgical and adjunctive medical therapies are needed to identify the best management algorithm for DOAC-associated ICH.
Asunto(s)
Factores de Coagulación Sanguínea , Tratamiento Conservador , Hemostáticos , Humanos , Femenino , Anciano , Masculino , Estudios de Cohortes , Estudios Retrospectivos , Factor IX , Hemostáticos/uso terapéutico , Hemorragia Cerebral/inducido químicamente , Hemorragia Cerebral/tratamiento farmacológico , Hematoma/inducido químicamente , Hematoma/tratamiento farmacológico , Anticoagulantes/efectos adversosRESUMEN
In this article, we present a novel and generic data-driven method to servo-control the 3-D shape of continuum and soft robots based on proprioceptive sensing feedback. Developments of 3-D shape perception and control technologies are crucial for continuum and soft robots to perform tasks autonomously in surgical interventions. However, owing to the nonlinear properties of continuum robots, one main difficulty lies in the modeling of them, especially for soft robots with variable stiffness. To address this problem, we propose a versatile learning-based adaptive shape controller by leveraging proprioception of 3-D configuration from fiber Bragg grating (FBG) sensors, which can online estimate the unknown model of continuum robot against unexpected disturbances and exhibit an adaptive behavior to the unmodeled system without priori data exploration. Based on a new composite adaptation algorithm, the asymptotic convergences of the closed-loop system with learning parameters have been proven by Lyapunov theory. To validate the proposed method, we present a comprehensive experimental study using two continuum and soft robots both integrated with multicore FBGs, including a robotic-assisted colonoscope and multisection extensible soft manipulators. The results demonstrate the feasibility, adaptability, and superiority of our controller in various unstructured environments, as well as phantom experiments.
RESUMEN
Imaging markers of cerebral small vessel disease provide valuable information on brain health, but their manual assessment is time-consuming and hampered by substantial intra- and interrater variability. Automated rating may benefit biomedical research, as well as clinical assessment, but diagnostic reliability of existing algorithms is unknown. Here, we present the results of the VAscular Lesions DetectiOn and Segmentation (Where is VALDO?) challenge that was run as a satellite event at the international conference on Medical Image Computing and Computer Aided Intervention (MICCAI) 2021. This challenge aimed to promote the development of methods for automated detection and segmentation of small and sparse imaging markers of cerebral small vessel disease, namely enlarged perivascular spaces (EPVS) (Task 1), cerebral microbleeds (Task 2) and lacunes of presumed vascular origin (Task 3) while leveraging weak and noisy labels. Overall, 12 teams participated in the challenge proposing solutions for one or more tasks (4 for Task 1-EPVS, 9 for Task 2-Microbleeds and 6 for Task 3-Lacunes). Multi-cohort data was used in both training and evaluation. Results showed a large variability in performance both across teams and across tasks, with promising results notably for Task 1-EPVS and Task 2-Microbleeds and not practically useful results yet for Task 3-Lacunes. It also highlighted the performance inconsistency across cases that may deter use at an individual level, while still proving useful at a population level.
Asunto(s)
Enfermedades de los Pequeños Vasos Cerebrales , Imagen por Resonancia Magnética , Humanos , Imagen por Resonancia Magnética/métodos , Reproducibilidad de los Resultados , Enfermedades de los Pequeños Vasos Cerebrales/diagnóstico por imagen , Hemorragia Cerebral , ComputadoresRESUMEN
Recent advancements in artificial intelligence have witnessed human-level performance; however, AI-enabled cognitive assistance for therapeutic procedures has not been fully explored nor pre-clinically validated. Here we propose AI-Endo, an intelligent surgical workflow recognition suit, for endoscopic submucosal dissection (ESD). Our AI-Endo is trained on high-quality ESD cases from an expert endoscopist, covering a decade time expansion and consisting of 201,026 labeled frames. The learned model demonstrates outstanding performance on validation data, including cases from relatively junior endoscopists with various skill levels, procedures conducted with different endoscopy systems and therapeutic skills, and cohorts from international multi-centers. Furthermore, we integrate our AI-Endo with the Olympus endoscopic system and validate the AI-enabled cognitive assistance system with animal studies in live ESD training sessions. Dedicated data analysis from surgical phase recognition results is summarized in an automatically generated report for skill assessment.
Asunto(s)
Endometriosis , Resección Endoscópica de la Mucosa , Animales , Femenino , Humanos , Resección Endoscópica de la Mucosa/educación , Resección Endoscópica de la Mucosa/métodos , Inteligencia Artificial , Flujo de Trabajo , Endoscopía , AprendizajeRESUMEN
Background: No investigations have thoroughly explored the feasibility of combining magnetic resonance (MR) images and deep-learning methods for predicting the progression of knee osteoarthritis (KOA). We thus aimed to develop a potential deep-learning model for predicting OA progression based on MR images for the clinical setting. Methods: A longitudinal case-control study was performed using data from the Foundation for the National Institutes of Health (FNIH), composed of progressive cases [182 osteoarthritis (OA) knees with both radiographic and pain progression for 24-48 months] and matched controls (182 OA knees not meeting the case definition). DeepKOA was developed through 3-dimensional (3D) DenseNet169 to predict KOA progression over 24-48 months based on sagittal intermediate-weighted turbo-spin echo sequences with fat-suppression (SAG-IW-TSE-FS), sagittal 3D dual-echo steady-state water excitation (SAG-3D-DESS-WE) and its axial and coronal multiplanar reformation, and their combined MR images with patient-level labels at baseline, 12, and 24 months to eventually determine the probability of progression. The classification performance of the DeepKOA was evaluated using 5-fold cross-validation. An X-ray-based model and traditional models that used clinical variables via multilayer perceptron were built. Combined models were also constructed, which integrated clinical variables with DeepKOA. The area under the curve (AUC) was used as the evaluation metric. Results: The performance of SAG-IW-TSE-FS in predicting OA progression was similar or higher to that of other single and combined sequences. The DeepKOA based on SAG-IW-TSE-FS achieved an AUC of 0.664 (95% CI: 0.585-0.743) at baseline, 0.739 (95% CI: 0.703-0.775) at 12 months, and 0.775 (95% CI: 0.686-0.865) at 24 months. The X-ray-based model achieved an AUC ranging from 0.573 to 0.613 at 3 time points. However, adding clinical variables to DeepKOA did not improve performance (P>0.05). Initial visualizations from gradient-weighted class activation mapping (Grad-CAM) indicated that the frequency with which the patellofemoral joint was highlighted increased as time progressed, which contrasted the trend observed in the tibiofemoral joint. The meniscus, the infrapatellar fat pad, and muscles posterior to the knee were highlighted to varying degrees. Conclusions: This study initially demonstrated the feasibility of DeepKOA in the prediction of KOA progression and identified the potential responsible structures which may enlighten the future development of more clinically practical methods.
RESUMEN
OBJECTIVES: Whole-body magnetic resonance imaging (WB-MRI) has been demonstrated to be efficient and cost-effective for cancer staging. The study aim was to develop a machine learning (ML) algorithm to improve radiologists' sensitivity and specificity for metastasis detection and reduce reading times. MATERIALS AND METHODS: A retrospective analysis of 438 prospectively collected WB-MRI scans from multicenter Streamline studies (February 2013-September 2016) was undertaken. Disease sites were manually labeled using Streamline reference standard. Whole-body MRI scans were randomly allocated to training and testing sets. A model for malignant lesion detection was developed based on convolutional neural networks and a 2-stage training strategy. The final algorithm generated lesion probability heat maps. Using a concurrent reader paradigm, 25 radiologists (18 experienced, 7 inexperienced in WB-/MRI) were randomly allocated WB-MRI scans with or without ML support to detect malignant lesions over 2 or 3 reading rounds. Reads were undertaken in the setting of a diagnostic radiology reading room between November 2019 and March 2020. Reading times were recorded by a scribe. Prespecified analysis included sensitivity, specificity, interobserver agreement, and reading time of radiology readers to detect metastases with or without ML support. Reader performance for detection of the primary tumor was also evaluated. RESULTS: Four hundred thirty-three evaluable WB-MRI scans were allocated to algorithm training (245) or radiology testing (50 patients with metastases, from primary 117 colon [n = 117] or lung [n = 71] cancer). Among a total 562 reads by experienced radiologists over 2 reading rounds, per-patient specificity was 86.2% (ML) and 87.7% (non-ML) (-1.5% difference; 95% confidence interval [CI], -6.4%, 3.5%; P = 0.39). Sensitivity was 66.0% (ML) and 70.0% (non-ML) (-4.0% difference; 95% CI, -13.5%, 5.5%; P = 0.344). Among 161 reads by inexperienced readers, per-patient specificity in both groups was 76.3% (0% difference; 95% CI, -15.0%, 15.0%; P = 0.613), with sensitivity of 73.3% (ML) and 60.0% (non-ML) (13.3% difference; 95% CI, -7.9%, 34.5%; P = 0.313). Per-site specificity was high (>90%) for all metastatic sites and experience levels. There was high sensitivity for the detection of primary tumors (lung cancer detection rate of 98.6% with and without ML [0.0% difference; 95% CI, -2.0%, 2.0%; P = 1.00], colon cancer detection rate of 89.0% with and 90.6% without ML [-1.7% difference; 95% CI, -5.6%, 2.2%; P = 0.65]). When combining all reads from rounds 1 and 2, reading times fell by 6.2% (95% CI, -22.8%, 10.0%) when using ML. Round 2 read-times fell by 32% (95% CI, 20.8%, 42.8%) compared with round 1. Within round 2, there was a significant decrease in read-time when using ML support, estimated as 286 seconds (or 11%) quicker ( P = 0.0281), using regression analysis to account for reader experience, read round, and tumor type. Interobserver variance suggests moderate agreement, Cohen κ = 0.64; 95% CI, 0.47, 0.81 (with ML), and Cohen κ = 0.66; 95% CI, 0.47, 0.81 (without ML). CONCLUSIONS: There was no evidence of a significant difference in per-patient sensitivity and specificity for detecting metastases or the primary tumor using concurrent ML compared with standard WB-MRI. Radiology read-times with or without ML support fell for round 2 reads compared with round 1, suggesting that readers familiarized themselves with the study reading method. During the second reading round, there was a significant reduction in reading time when using ML support.
Asunto(s)
Neoplasias del Colon , Neoplasias Pulmonares , Humanos , Imagen por Resonancia Magnética/métodos , Estudios Retrospectivos , Imagen de Cuerpo Entero/métodos , Pulmón , Neoplasias Pulmonares/diagnóstico por imagen , Neoplasias del Colon/diagnóstico por imagen , Sensibilidad y Especificidad , Pruebas Diagnósticas de RutinaRESUMEN
This paper presents an effective and general data augmentation framework for medical image segmentation. We adopt a computationally efficient and data-efficient gradient-based meta-learning scheme to explicitly align the distribution of training and validation data which is used as a proxy for unseen test data. We improve the current data augmentation strategies with two core designs. First, we learn class-specific training-time data augmentation (TRA) effectively increasing the heterogeneity within the training subsets and tackling the class imbalance common in segmentation. Second, we jointly optimize TRA and test-time data augmentation (TEA), which are closely connected as both aim to align the training and test data distribution but were so far considered separately in previous works. We demonstrate the effectiveness of our method on four medical image segmentation tasks across different scenarios with two state-of-the-art segmentation models, DeepMedic and nnU-Net. Extensive experimentation shows that the proposed data augmentation framework can significantly and consistently improve the segmentation performance when compared to existing solutions. Code is publicly available at https://github.com/ZerojumpLine/JCSAugment.
RESUMEN
Magnetic resonance imaging (MRI) possesses the unique versatility to acquire images under a diverse array of distinct tissue contrasts, which makes multicontrast super-resolution (SR) techniques possible and needful. Compared with single-contrast MRI SR, multicontrast SR is expected to produce higher quality images by exploiting a variety of complementary information embedded in different imaging contrasts. However, existing approaches still have two shortcomings: 1) most of them are convolution-based methods and, hence, weak in capturing long-range dependencies, which are essential for MR images with complicated anatomical patterns and 2) they ignore to make full use of the multicontrast features at different scales and lack effective modules to match and aggregate these features for faithful SR. To address these issues, we develop a novel multicontrast MRI SR network via transformer-empowered multiscale feature matching and aggregation, dubbed McMRSR ++ . First, we tame transformers to model long-range dependencies in both reference and target images at different scales. Then, a novel multiscale feature matching and aggregation method is proposed to transfer corresponding contexts from reference features at different scales to the target features and interactively aggregate them Furthermore, a texture-preserving branch and a contrastive constraint are incorporated into our framework for enhancing the textural details in the SR images. Experimental results on both public and clinical in vivo datasets show that McMRSR ++ outperforms state-of-the-art methods under peak signal to noise ratio (PSNR), structure similarity index measure (SSIM), and root mean square error (RMSE) metrics significantly. Visual results demonstrate the superiority of our method in restoring structures, demonstrating its great potential to improve scan efficiency in clinical practice.