Local Large Language Models for Complex Structured Tasks.

This paper introduces an approach that combines the language reasoning capabilities of large language models (LLMs) with the benefits of local training to tackle complex language tasks. The authors demonstrate their approach by extracting structured condition codes from pathology reports. The proposed approach utilizes local, fine-tuned LLMs to respond to specific generative instructions and provide structured outputs. Over 150k uncurated surgical pathology reports containing gross descriptions, final diagnoses, and condition codes were used. Different model architectures were trained and evaluated, including LLaMA, BERT, and LongFormer. The results show that the LLaMA-based models significantly outperform BERT-style models across all evaluated metrics. LLaMA models performed especially well with large datasets, demonstrating their ability to handle complex, multi-label tasks. Overall, this work presents an effective approach for utilizing LLMs to perform structured generative tasks on domain-specific language in the medical domain.

Ki-67 Testing in Breast Cancer: Assessing Variability With Scoring Methods and Specimen Types and the Potential Subsequent Impact on Therapy Eligibility.

Abemaciclib was originally FDA approved for patients with ER-positive/HER2-negative breast cancer with Ki-67 expression ≥20%. However, there were no guidelines provided on which specimen to test or which scoring method to use. We performed a comprehensive study evaluating the variation in Ki-67 expression in breast specimens from 50 consecutive patients who could have been eligible for abemaciclib therapy. Three pathologists with breast expertise each performed a blinded review with 3 different manual scoring methods [estimated (EST), unweighted (UNW), and weighted (WT) (WT recommended by the International Ki-67 in Breast Cancer Working Group)]. Quantitative image analysis (QIA) using the HALO platform was also performed. Three different specimen types [core needle biopsy (CNB) (n=63), resection (RES) (n=52), and axillary lymph node metastasis (ALN) (n=50)] were evaluated for each patient. The average Ki-67 for all specimens was 14.68% for EST, 14.46% for UNW, 14.15% for WT, and 11.15% for QIA. For the manual methods, the range between the lowest and highest Ki-67 for each specimen between the 3 pathologists was 8.44 for EST, 5.94 for WT, and 5.93 for UNW. The WT method limited interobserver variability with ICC1=0.959 (EST ICC1=0.922 and UNW=0.949). Using the aforementioned cutoff of Ki-67 ≥20% versus <20% to determine treatment eligibility, the averaged EST method yields 20 of 50 patients (40%) who would have been treatment-eligible, versus 15 (30%) for the UNW, 17 (34%) for the WT, and 12 (24%) for the QIA. There was no statistically significant difference in Ki-67 among the 3 specimen types. The average Ki-67 difference was 4.36 for CNB vs RES, 6.95 for CNB versus ALN, and RES versus ALN (P=0.93, 0.99, and 0.94, respectively). Our study concludes that further refinement in Ki-67 scoring is advisable to reduce clinically significant variation.

Utility of artificial intelligence in a binary classification of soft tissue tumors.

Soft tissue tumors (STTs) pose diagnostic and therapeutic challenges due to their rarity, complexity, and morphological overlap. Accurate differentiation between benign and malignant STTs is important to set treatment directions, however, this task can be difficult. The integration of machine learning and artificial intelligence (AI) models can potentially be helpful in classifying these tumors. The aim of this study was to investigate AI and machine learning tools in the classification of STT into benign and malignant categories. This study consisted of three components: (1) Evaluation of whole-slide images (WSIs) to classify STT into benign and malignant entities. Five specialized soft tissue pathologists from different medical centers independently reviewed 100 WSIs, representing 100 different cases, with limited clinical information and no additional workup. The results showed an overall concordance rate of 70.4% compared to the reference diagnosis. (2) Identification of cell-specific parameters that can distinguish benign and malignant STT. Using an image analysis software (QuPath) and a cohort of 95 cases, several cell-specific parameters were found to be statistically significant, most notably cell count, nucleus/cell area ratio, nucleus hematoxylin density mean, and cell max caliper. (3) Evaluation of machine learning library (Scikit-learn) in differentiating benign and malignant STTs. A total of 195 STT cases (156 cases in the training group and 39 cases in the validation group) achieved approximately 70% sensitivity and specificity, and an AUC of 0.68. Our limited study suggests that the use of WSI and AI in soft tissue pathology has the potential to enhance diagnostic accuracy and identify parameters that can differentiate between benign and malignant STTs. We envision the integration of AI as a supportive tool to augment the pathologists' diagnostic capabilities.

Automated Curation and AI Workflow Management System for Digital Pathology.

Digital pathology applications present several challenges, including the processing, storage, and distribution of gigapixel images across distributed computational resources and viewing stations. Individual slides must be available for interactive review, and large repositories must be programmatically accessible for dataset and model building. We present a platform to manage and process multi-modal pathology data (images and case information) across multiple locations. Using an agent-based system coupled with open-source automated machine learning and review tools allows not only dynamic load-balancing and cross-network operation but also the development of research and clinical AI models using the data managed by the platform. The platform presented covers end-to-end AI workflow from data acquisition and curation through model training and evaluation allowing for sharing and review. We conclude with a case study of colon and prostate cancer model development utilizing the presented system.