Benchmarking PathCLIP for Pathology Image Analysis.

Accurate image classification and retrieval are of importance for clinical diagnosis and treatment decision-making. The recent contrastive language-image pre-training (CLIP) model has shown remarkable proficiency in understanding natural images. Drawing inspiration from CLIP, pathology-dedicated CLIP (PathCLIP) has been developed, utilizing over 200,000 image and text pairs in training. While the performance the PathCLIP is impressive, its robustness under a wide range of image corruptions remains unknown. Therefore, we conduct an extensive evaluation to analyze the performance of PathCLIP on various corrupted images from the datasets of osteosarcoma and WSSS4LUAD. In our experiments, we introduce eleven corruption types including brightness, contrast, defocus, resolution, saturation, hue, markup, deformation, incompleteness, rotation, and flipping at various settings. Through experiments, we find that PathCLIP surpasses OpenAI-CLIP and the pathology language-image pre-training (PLIP) model in zero-shot classification. It is relatively robust to image corruptions including contrast, saturation, incompleteness, and orientation factors. Among the eleven corruptions, hue, markup, deformation, defocus, and resolution can cause relatively severe performance fluctuation of the PathCLIP. This indicates that ensuring the quality of images is crucial before conducting a clinical test. Additionally, we assess the robustness of PathCLIP in the task of image-to-image retrieval, revealing that PathCLIP performs less effectively than PLIP on osteosarcoma but performs better on WSSS4LUAD under diverse corruptions. Overall, PathCLIP presents impressive zero-shot classification and retrieval performance for pathology images, but appropriate care needs to be taken when using it.

Localization of contrast-enhanced breast lesions in ultrafast screening MRI using deep convolutional neural networks.

OBJECTIVES: To develop a deep learning-based method for contrast-enhanced breast lesion detection in ultrafast screening MRI. MATERIALS AND METHODS: A total of 837 breast MRI exams of 488 consecutive patients were included. Lesion's location was independently annotated in the maximum intensity projection (MIP) image of the last time-resolved angiography with stochastic trajectories (TWIST) sequence for each individual breast, resulting in 265 lesions (190 benign, 75 malignant) in 163 breasts (133 women). YOLOv5 models were fine-tuned using training sets containing the same number of MIP images with and without lesions. A long short-term memory (LSTM) network was employed to help reduce false positive predictions. The integrated system was then evaluated on test sets containing enriched uninvolved breasts during cross-validation to mimic the performance in a screening scenario. RESULTS: In five-fold cross-validation, the YOLOv5x model showed a sensitivity of 0.95, 0.97, 0.98, and 0.99, with 0.125, 0.25, 0.5, and 1 false positive per breast, respectively. The LSTM network reduced 15.5% of the false positive prediction from the YOLO model, and the positive predictive value was increased from 0.22 to 0.25. CONCLUSIONS: A fine-tuned YOLOv5x model can detect breast lesions on ultrafast MRI with high sensitivity in a screening population, and the output of the model could be further refined by an LSTM network to reduce the amount of false positive predictions. CLINICAL RELEVANCE STATEMENT: The proposed integrated system would make the ultrafast MRI screening process more effective by assisting radiologists in prioritizing suspicious examinations and supporting the diagnostic workup. KEY POINTS: • Deep convolutional neural networks could be utilized to automatically pinpoint breast lesions in screening MRI with high sensitivity. • False positive predictions significantly increased when the detection models were tested on highly unbalanced test sets with more normal scans. • Dynamic enhancement patterns of breast lesions during contrast inflow learned by the long short-term memory networks helped to reduce false positive predictions.

Masked Conditional Variational Autoencoders for Chromosome Straightening.

Karyotyping is of importance for detecting chromosomal aberrations in human disease. However, chromosomes easily appear curved in microscopic images, which prevents cytogeneticists from analyzing chromosome types. To address this issue, we propose a framework for chromosome straightening, which comprises a preliminary processing algorithm and a generative model called masked conditional variational autoencoders (MC-VAE). The processing method utilizes patch rearrangement to address the difficulty in erasing low degrees of curvature, providing reasonable preliminary results for the MC-VAE. The MC-VAE further straightens the results by leveraging chromosome patches conditioned on their curvatures to learn the mapping between banding patterns and conditions. During model training, we apply a masking strategy with a high masking ratio to train the MC-VAE with eliminated redundancy. This yields a non-trivial reconstruction task, allowing the model to effectively preserve chromosome banding patterns and structure details in the reconstructed results. Extensive experiments on three public datasets with two stain styles show that our framework surpasses the performance of state-of-the-art methods in retaining banding patterns and structure details. Compared to using real-world bent chromosomes, the use of high-quality straightened chromosomes generated by our proposed method can improve the performance of various deep learning models for chromosome classification by a large margin. Such a straightening approach has the potential to be combined with other karyotyping systems to assist cytogeneticists in chromosome analysis.

Preparing CT imaging datasets for deep learning in lung nodule analysis: Insights from four well-known datasets.

BACKGROUND: Deep learning is an important means to realize the automatic detection, segmentation, and classification of pulmonary nodules in computed tomography (CT) images. An entire CT scan cannot directly be used by deep learning models due to image size, image format, image dimensionality, and other factors. Between the acquisition of the CT scan and feeding the data into the deep learning model, there are several steps including data use permission, data access and download, data annotation, and data preprocessing. This paper aims to recommend a complete and detailed guide for researchers who want to engage in interdisciplinary lung nodule research of CT images and Artificial Intelligence (AI) engineering. METHODS: The data preparation pipeline used the following four popular large-scale datasets: LIDC-IDRI (Lung Image Database Consortium image collection), LUNA16 (Lung Nodule Analysis 2016), NLST (National Lung Screening Trial) and NELSON (The Dutch-Belgian Randomized Lung Cancer Screening Trial). The dataset preparation is presented in chronological order. FINDINGS: The different data preparation steps before deep learning were identified. These include both more generic steps and steps dedicated to lung nodule research. For each of these steps, the required process, necessity, and example code or tools for actual implementation are provided. DISCUSSION AND CONCLUSION: Depending on the specific research question, researchers should be aware of the various preparation steps required and carefully select datasets, data annotation methods, and image preprocessing methods. Moreover, it is vital to acknowledge that each auxiliary tool or code has its specific scope of use and limitations. This paper proposes a standardized data preparation process while clearly demonstrating the principles and sequence of different steps. A data preparation pipeline can be quickly realized by following these proposed steps and implementing the suggested example codes and tools.

Prediction of histologic types in solid lung lesions using preoperative contrast-enhanced CT.

OBJECTIVES: This study aimed to develop and validate a predicting model for the histologic classification of solid lung lesions based on preoperative contrast-enhanced CT. METHODS: A primary dataset of 1012 patients from Tianjin Medical University Cancer Institute and Hospital (TMUCIH) was randomly divided into a development cohort (708) and an internal validation cohort (304). Patients from the Second Hospital of Shanxi Medical University (SHSMU) were set as an external validation cohort (212). Two clinical factors (age, gender) and twenty-one characteristics on contrast-enhanced CT were used to construct a multinomial multivariable logistic regression model for the classification of seven common histologic types of solid lung lesions. The area under the receiver operating characteristic curve was used to assess the diagnostic performance of the model in the development and validation cohorts, separately. RESULTS: Multivariable analysis showed that two clinical factors and twenty-one characteristics on contrast-enhanced CT were predictive in lung lesion histologic classification. The mean AUC of the proposed model for histologic classification was 0.95, 0.94, and 0.92 in the development, internal validation, and external validation cohort, respectively. When determining the malignancy of lung lesions based on histologic types, the mean AUC of the model was 0.88, 0.86, and 0.90 in three cohorts. CONCLUSIONS: We demonstrated that by utilizing both clinical and CT characteristics on contrast-enhanced CT images, the proposed model could not only effectively stratify histologic types of solid lung lesions, but also enabled accurate assessment of lung lesion malignancy. Such a model has the potential to avoid unnecessary surgery for patients and to guide clinical decision-making for preoperative treatment. KEY POINTS: • Clinical and CT characteristics on contrast-enhanced CT could be used to differentiate histologic types of solid lung lesions. • Predicting models using preoperative contrast-enhanced CT could accurately assessment of tumor malignancy based on predicted histologic types.

Survival prediction for stage I-IIIA non-small cell lung cancer using deep learning.

BACKGROUND AND PURPOSE: The aim of this study was to develop and evaluate a prediction model for 2-year overall survival (OS) in stage I-IIIA non-small cell lung cancer (NSCLC) patients who received definitive radiotherapy by considering clinical variables and image features from pre-treatment CT-scans. MATERIALS AND METHODS: NSCLC patients who received stereotactic radiotherapy were prospectively collected at the UMCG and split into a training and a hold out test set including 189 and 81 patients, respectively. External validation was performed on 228 NSCLC patients who were treated with radiation or concurrent chemoradiation at the Maastro clinic (Lung1 dataset). A hybrid model that integrated both image and clinical features was implemented using deep learning. Image features were learned from cubic patches containing lung tumours extracted from pre-treatment CT scans. Relevant clinical variables were selected by univariable and multivariable analyses. RESULTS: Multivariable analysis showed that age and clinical stage were significant prognostic clinical factors for 2-year OS. Using these two clinical variables in combination with image features from pre-treatment CT scans, the hybrid model achieved a median AUC of 0.76 [95 % CI: 0.65-0.86] and 0.64 [95 % CI: 0.58-0.70] on the complete UMCG and Maastro test sets, respectively. The Kaplan-Meier survival curves showed significant separation between low and high mortality risk groups on these two test sets (log-rank test: p-value < 0.001, p-value = 0.012, respectively) CONCLUSION: We demonstrated that a hybrid model could achieve reasonable performance by utilizing both clinical and image features for 2-year OS prediction. Such a model has the potential to identify patients with high mortality risk and guide clinical decision making.

Using deep learning to safely exclude lesions with only ultrafast breast MRI to shorten acquisition and reading time.

OBJECTIVES: To investigate the feasibility of automatically identifying normal scans in ultrafast breast MRI with artificial intelligence (AI) to increase efficiency and reduce workload. METHODS: In this retrospective analysis, 837 breast MRI examinations performed on 438 women from April 2016 to October 2019 were included. The left and right breasts in each examination were labelled normal (without suspicious lesions) or abnormal (with suspicious lesions) based on final interpretation. Maximum intensity projection (MIP) images of each breast were then used to train a deep learning model. A high sensitivity threshold was calculated based on the detection trade - off (DET) curve on the validation set. The performance of the model was evaluated by receiver operating characteristic analysis of the independent test set. The sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) with the high sensitivity threshold were calculated. RESULTS: The independent test set consisted of 178 examinations of 149 patients (mean age, 44 years ± 14 [standard deviation]). The trained model achieved an AUC of 0.81 (95% CI: 0.75-0.88) on the independent test set. Applying a threshold of 0.25 yielded a sensitivity of 98% (95% CI: 90%; 100%), an NPV of 98% (95% CI: 89%; 100%), a workload reduction of 15.7%, and a scan time reduction of 16.6%. CONCLUSION: This deep learning model has a high potential to help identify normal scans in ultrafast breast MRI and thereby reduce radiologists' workload and scan time. KEY POINTS: • Deep learning in TWIST may eliminate the necessity of additional sequences for identifying normal breasts during MRI screening. • Workload and scanning time reductions of 15.7% and 16.6%, respectively, could be achieved with the cost of 1 (1 of 55) false negative prediction.

Outstanding negative prediction performance of solid pulmonary nodule volume AI for ultra-LDCT baseline lung cancer screening risk stratification.

OBJECTIVE: To evaluate performance of AI as a standalone reader in ultra-low-dose CT lung cancer baseline screening, and compare it to that of experienced radiologists. METHODS: 283 participants who underwent a baseline ultra-LDCT scan in Moscow Lung Cancer Screening, between February 2017-2018, and had at least one solid lung nodule, were included. Volumetric nodule measurements were performed by five experienced blinded radiologists, and independently assessed using an AI lung cancer screening prototype (AVIEW LCS, v1.0.34, Coreline Soft, Co. ltd, Seoul, Korea) to automatically detect, measure, and classify solid nodules. Discrepancies were stratified into two groups: positive-misclassification (PM); nodule classified by the reader as a NELSON-plus /EUPS-indeterminate/positive nodule, which at the reference consensus read was < 100 mm3, and negative-misclassification (NM); nodule classified as a NELSON-plus /EUPS-negative nodule, which at consensus read was ≥ 100 mm3. RESULTS: 1149 nodules with a solid-component were detected, of which 878 were classified as solid nodules. For the largest solid nodule per participant (n = 283); 61 [21.6 %; 53 PM, 8 NM] discrepancies were reported for AI as a standalone reader, compared to 43 [15.1 %; 22 PM, 21 NM], 36 [12.7 %; 25 PM, 11 NM], 29 [10.2 %; 25 PM, 4 NM], 28 [9.9 %; 6 PM, 22 NM], and 50 [17.7 %; 15 PM, 35 NM] discrepancies for readers 1, 2, 3, 4, and 5 respectively. CONCLUSION: Our results suggest that through the use of AI as an impartial reader in baseline lung cancer screening, negative-misclassification results could exceed that of four out of five experienced radiologists, and radiologists' workload could be drastically diminished by up to 86.7%.

Performance of a deep learning-based lung nodule detection system as an alternative reader in a Chinese lung cancer screening program.

OBJECTIVE: To evaluate the performance of a deep learning-based computer-aided detection (DL-CAD) system in a Chinese low-dose CT (LDCT) lung cancer screening program. MATERIALS AND METHODS: One-hundred-and-eighty individuals with a lung nodule on their baseline LDCT lung cancer screening scan were randomly mixed with screenees without nodules in a 1:1 ratio (total: 360 individuals). All scans were assessed by double reading and subsequently processed by an academic DL-CAD system. The findings of double reading and the DL-CAD system were then evaluated by two senior radiologists to derive the reference standard. The detection performance was evaluated by the Free Response Operating Characteristic curve, sensitivity and false-positive (FP) rate. The senior radiologists categorized nodules according to nodule diameter, type (solid, part-solid, non-solid) and Lung-RADS. RESULTS: The reference standard consisted of 262 nodules ≥ 4 mm in 196 individuals; 359 findings were considered false positives. The DL-CAD system achieved a sensitivity of 90.1% with 1.0 FP/scan for detection of lung nodules regardless of size or type, whereas double reading had a sensitivity of 76.0% with 0.04 FP/scan (P = 0.001). The sensitivity for detection of nodules ≥ 4 - ≤ 6 mm was significantly higher with DL-CAD than with double reading (86.3% vs. 58.9% respectively; P = 0.001). Sixty-three nodules were only identified by the DL-CAD system, and 27 nodules only found by double reading. The DL-CAD system reached similar performance compared to double reading in Lung-RADS 3 (94.3% vs. 90.0%, P = 0.549) and Lung-RADS 4 nodules (100.0% vs. 97.0%, P = 1.000), but showed a higher sensitivity in Lung-RADS 2 (86.2% vs. 65.4%, P < 0.001). CONCLUSIONS: The DL-CAD system can accurately detect pulmonary nodules on LDCT, with an acceptable false-positive rate of 1 nodule per scan and has higher detection performance than double reading. This DL-CAD system may assist radiologists in nodule detection in LDCT lung cancer screening.

Prognostic value of desmoplastic stromal reaction, tumor budding and tumor-stroma ratio in stage II colorectal cancer.

Background: Existing high-risk factors are insufficient to accurately predict the postoperative recurrence risk of stage II colorectal cancer (CRC). The discovery of additional prognostic markers may be the key to improving the current status of stage II CRC treatment. The present study aimed to evaluate the relationship among desmoplastic reaction (DR), tumor budding (TBd), the tumor-stroma ratio (TSR) and their prognostic value for relapse-free survival (RFS). Methods: In this study, 207 patients with histologically confirmed stage II CRC from January 2012 to August 2018 were retrospectively reviewed from a single center; the cohort was divided into subgroups based on low or high TSR, and low, intermediate or high DR and TBd. Kaplan-Meier curve analysis and log-rank test were applied to examine RFS among subgroups. Univariate and multivariate Cox proportional hazards analyses were used to identify independent factors associated with RFS, and a nomogram was subsequently developed. Results: Abnormal CA242, CEA, T4 stage, presence of hypertension, internal obstruction or perforation (IOP), lymphovascular or/and perineural invasion (PNI), number of nodes examined less than 12, low-frequency microsatellite instability (MSI-L), higher Ki-67 and immature DR were associated with a lower RFS. In multivariable analysis, DR (HR =2.111; 95% CI: 1.184-3.766; P=0.011), LVI (HR =1.919; 95% CI: 1.004-3.669; P=0.049) and PNI (HR =2.724; 95% CI: 1.362-5.448; P=0.005) were prognostic factors for RFS. On this basis, a nomogram that integrated DR and clinicopathologic predictors for predicting RFS passed the calibration and had an area under the curve of 0.826. Conclusions: The prognostic significance of DR outperformed TBd and TSR, therefore, we recommend adding DR as a biomarker in routine pathological reports. The novel nomogram combining these factors may be used as a reliable and effective tool for the prediction of RFS in stage II CRC, thus helping optimize therapeutic regimens under cooperation of oncologists and surgeons. Further multicentric studies are required for validation of this novel, simple and cost-effective prognostic model.

Deep convolutional neural networks for multiplanar lung nodule detection: Improvement in small nodule identification.

PURPOSE: Early detection of lung cancer is of importance since it can increase patients' chances of survival. To detect nodules accurately during screening, radiologists would commonly take the axial, coronal, and sagittal planes into account, rather than solely the axial plane in clinical evaluation. Inspired by clinical work, the paper aims to develop an accurate deep learning framework for nodule detection by a combination of multiple planes. METHODS: The nodule detection system is designed in two stages, multiplanar nodule candidate detection, multiscale false positive (FP) reduction. At the first stage, a deeply supervised encoder-decoder network is trained by axial, coronal, and sagittal slices for the candidate detection task. All possible nodule candidates from the three different planes are merged. To further refine results, a three-dimensional multiscale dense convolutional neural network that extracts multiscale contextual information is applied to remove non-nodules. In the public LIDC-IDRI dataset, 888 computed tomography scans with 1186 nodules accepted by at least three of four radiologists are selected to train and evaluate our proposed system via a tenfold cross-validation scheme. The free-response receiver operating characteristic curve is used for performance assessment. RESULTS: The proposed system achieves a sensitivity of 94.2% with 1.0 FP/scan and a sensitivity of 96.0% with 2.0 FPs/scan. Although it is difficult to detect small nodules (i.e., <6 mm), our designed CAD system reaches a sensitivity of 93.4% (95.0%) of these small nodules at an overall FP rate of 1.0 (2.0) FPs/scan. At the nodule candidate detection stage, results show that the system with a multiplanar method is capable to detect more nodules compared to using a single plane. CONCLUSION: Our approach achieves good performance not only for small nodules but also for large lesions on this dataset. This demonstrates the effectiveness of our developed CAD system for lung nodule detection.

Deep learning-based pulmonary nodule detection: Effect of slab thickness in maximum intensity projections at the nodule candidate detection stage.

BACKGROUND AND OBJECTIVE: To investigate the effect of the slab thickness in maximum intensity projections (MIPs) on the candidate detection performance of a deep learning-based computer-aided detection (DL-CAD) system for pulmonary nodule detection in CT scans. METHODS: The public LUNA16 dataset includes 888 CT scans with 1186 nodules annotated by four radiologists. From those scans, MIP images were reconstructed with slab thicknesses of 5 to 50 mm (at 5 mm intervals) and 3 to 13 mm (at 2 mm intervals). The architecture in the nodule candidate detection part of the DL-CAD system was trained separately using MIP images with various slab thicknesses. Based on ten-fold cross-validation, the sensitivity and the F2 score were determined to evaluate the performance of using each slab thickness at the nodule candidate detection stage. The free-response receiver operating characteristic (FROC) curve was used to assess the performance of the whole DL-CAD system that took the results combined from 16 MIP slab thickness settings. RESULTS: At the nodule candidate detection stage, the combination of results from 16 MIP slab thickness settings showed a high sensitivity of 98.0% with 46 false positives (FPs) per scan. Regarding a single MIP slab thickness of 10 mm, the highest sensitivity of 90.0% with 8 FPs/scan was reached before false positive reduction. The sensitivity increased (82.8% to 90.0%) for slab thickness of 1 to 10 mm and decreased (88.7% to 76.6%) for slab thickness of 15-50 mm. The number of FPs was decreasing with increasing slab thickness, but was stable at 5 FPs/scan at a slab thickness of 30 mm or more. After false positive reduction, the DL-CAD system, utilizing 16 MIP slab thickness settings, had the sensitivity of 94.4% with 1 FP/scan. CONCLUSIONS: The utilization of multi-MIP images could improve the performance at the nodule candidate detection stage, even for the whole DL-CAD system. For a single slab thickness of 10 mm, the highest sensitivity for pulmonary nodule detection was reached at the nodule candidate detection stage, similar to the slab thickness usually applied by radiologists.

A Subsolid Nodules Imaging Reporting System (SSN-IRS) for Classifying 3 Subtypes of Pulmonary Adenocarcinoma.

OBJECTIVES: To develop an imaging reporting system for the classification of 3 adenocarcinoma subtypes of computed tomography (CT)-detected subsolid pulmonary nodules (SSNs) in clinical patients. METHODS: Between November 2011 and October 2017, 437 pathologically confirmed SSNs were retrospectively identified. SSNs were randomly divided 2:1 into a training group (291 cases) and a testing group (146 cases). CT-imaging characteristics were analyzed using multinomial univariable and multivariable logistic regression analysis to identify discriminating factors for the 3 adenocarcinoma subtypes (pre-invasive lesions, minimally invasive adenocarcinoma, and invasive adenocarcinoma). These factors were used to develop a classification and regression tree model. Finally, an SSN Imaging Reporting System (SSN-IRS) was constructed based on the optimized classification model. For validation, the classification performance was evaluated in the testing group. RESULTS: Of the CT-derived characteristics of SSNs, qualitative density (nonsolid or part-solid), core (non-core or core), semantic features (pleural indentation, vacuole sign, vascular invasion), and diameter of solid component (≤6 mm or >6 mm), were the most important factors for the SSN-IRS. The total sensitivity, specificity, and diagnostic accuracy of the SSN-IRS was 89.0% (95% confidence interval [CI], 84.8%-92.4%), 74.6% (95% CI, 70.8%-78.1%), and 79.4% (95% CI, 76.5%-82.0%) in the training group and 84.9% (95% CI, 78.1%-90.3%), 68.5% (95% CI, 62.8%-73.8%), and 74.0% (95% CI, 69.6%-78.0%) in the testing group, respectively. CONCLUSIONS: The SSN-IRS can classify 3 adenocarcinoma subtypes using CT-based characteristics of subsolid pulmonary nodules. This classification tool can help clinicians to make follow-up recommendations or decisions for surgery in clinical patients with SSNs.

Automatic Pulmonary Nodule Detection in CT Scans Using Convolutional Neural Networks Based on Maximum Intensity Projection.

Accurate pulmonary nodule detection is a crucial step in lung cancer screening. Computer-aided detection (CAD) systems are not routinely used by radiologists for pulmonary nodule detection in clinical practice despite their potential benefits. Maximum intensity projection (MIP) images improve the detection of pulmonary nodules in radiological evaluation with computed tomography (CT) scans. Inspired by the clinical methodology of radiologists, we aim to explore the feasibility of applying MIP images to improve the effectiveness of automatic lung nodule detection using convolutional neural networks (CNNs). We propose a CNN-based approach that takes MIP images of different slab thicknesses (5 mm, 10 mm, 15 mm) and 1 mm axial section slices as input. Such an approach augments the two-dimensional (2-D) CT slice images with more representative spatial information that helps discriminate nodules from vessels through their morphologies. Our proposed method achieves sensitivity of 92.7% with 1 false positive per scan and sensitivity of 94.2% with 2 false positives per scan for lung nodule detection on 888 scans in the LIDC-IDRI dataset. The use of thick MIP images helps the detection of small pulmonary nodules (3 mm-10 mm) and results in fewer false positives. Experimental results show that utilizing MIP images can increase the sensitivity and lower the number of false positives, which demonstrates the effectiveness and significance of the proposed MIP-based CNNs framework for automatic pulmonary nodule detection in CT scans. The proposed method also shows the potential that CNNs could gain benefits for nodule detection by combining the clinical procedure.

Comparison of Veterans Affairs, Mayo, Brock classification models and radiologist diagnosis for classifying the malignancy of pulmonary nodules in Chinese clinical population.

BACKGROUND: Several classification models based on Western population have been developed to help clinicians to classify the malignancy probability of pulmonary nodules. However, the diagnostic performance of these Western models in Chinese population is unknown. This paper aimed to compare the diagnostic performance of radiologist evaluation of malignancy probability and three classification models (Mayo Clinic, Veterans Affairs, and Brock University) in Chinese clinical pulmonology patients. METHODS: This single-center retrospective study included clinical patients from Tianjin Medical University Cancer Institute and Hospital with new, CT-detected pulmonary nodules in 2013. Patients with a nodule with diameter of 4-25 mm, and histological diagnosis or 2-year follow-up were included. Analysis of area under the receiver operating characteristic curve (AUC), decision curve analysis (DCA) and threshold of decision analysis was used to evaluate the diagnostic performance of radiologist diagnosis and the three classification models, with histological diagnosis or 2-year follow-up as the reference. RESULTS: In total, 277 patients (286 nodules) were included. Two hundred and seven of 286 nodules (72.4%) in 203 patients were malignant. AUC of the Mayo model (0.77; 95% CI: 0.72-0.82) and Brock model (0.77; 95% CI: 0.72-0.82) were similar to radiologist diagnosis (0.78; 95% CI: 0.73-0.83; P=0.68, P=0.71, respectively). The diagnostic performance of the VA model (AUC: 0.66) was significantly lower than that of radiologist diagnosis (P=0.003). A three-class classifying threshold analysis and DCA showed that the radiologist evaluation had higher discriminatory power for malignancy than the three classification models. CONCLUSIONS: In a cohort of Chinese clinical pulmonology patients, radiologist evaluation of lung nodule malignancy probability demonstrated higher diagnostic performance than Mayo, Brock, and VA classification models. To optimize nodule diagnosis and management, a new model with more radiological characteristics could be valuable.