Risk and tropism of central nervous system (CNS) metastases in patients with stage II and III cutaneous melanoma.

BACKGROUND: Recent data suggest that patients with stage III melanoma are at high risk for developing central nervous system (CNS) metastases. Because a subset of patients with stage II melanoma experiences worse survival outcomes than some patients with stage III disease, the authors investigated the risk of CNS metastasis in stage II melanoma to inform surveillance guidelines for this population. METHODS: The authors examined clinicopathologic data prospectively collected from 1054 patients who had cutaneous melanoma. The χ2 test, the cumulative incidence, and Cox multivariable regression analyses were performed to evaluate the association between baseline characteristics and the development of CNS metastases. RESULTS: Patients with stage III melanoma had a higher rate of developing brain metastases than those with stage II melanoma (100 of 468 patients [21.4%] vs. 82 of 586 patients [14.0%], respectively; p = .002). However, patients who had stage IIC melanoma had a significantly higher rate of isolated first recurrences in the CNS compared with those who had stage III disease (12.1% vs. 3.6%; p = .002). The risk of ever developing brain metastases was similarly elevated for patients who had stage IIC disease (hazard ratio [HR], 3.16; 95% CI, 1.77-5.66), stage IIIB disease (HR, 2.83; 95% CI, 1.63-4.91), and stage IIIC disease (HR, 2.93; 95% CI, 1.81-4.74), and the risk was highest in patients who had stage IIID disease (HR, 8.59; 95% CI: 4.11-17.97). CONCLUSIONS: Patients with stage IIC melanoma are at elevated risk for first recurrence in the CNS. Surveillance strategies that incorporate serial neuroimaging should be considered for these individuals until more accurate predictive markers can be identified.

Generalization in Clinical Prediction Models: The Blessing and Curse of Measurement Indicator Variables.

OBJECTIVE: Specific factors affecting generalizability of clinical prediction models are poorly understood. Our main objective was to investigate how measurement indicator variables affect external validity in clinical prediction models for predicting onset of vasopressor therapy. DESIGN: We fit logistic regressions on retrospective cohorts to predict vasopressor onset using two classes of variables: seemingly objective clinical variables (vital signs and laboratory measurements) and more subjective variables denoting recency of measurements. SETTING: Three cohorts from two tertiary-care academic hospitals in geographically distinct regions, spanning general inpatient and critical care settings. PATIENTS: Each cohort consisted of adult patients (age greater than or equal to 18 yr at time of hospitalization), with lengths of stay between 6 and 600 hours, and who did not receive vasopressors in the first 6 hours of hospitalization or ICU admission. Models were developed on each of the three derivation cohorts and validated internally on the derivation cohort and externally on the other two cohorts. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: The prevalence of vasopressors was 0.9% in the general inpatient cohort and 12.4% and 11.5% in the two critical care cohorts. Models utilizing both classes of variables performed the best in-sample, with C-statistics for predicting vasopressor onset in 4 hours of 0.862 (95% CI, 0.844-0.879), 0.822 (95% CI, 0.793-0.852), and 0.889 (95% CI, 0.880-0.898). Models solely using the subjective variables denoting measurement recency had poor external validity. However, these practice-driven variables helped adjust for differences between the two hospitals and led to more generalizable models using clinical variables. CONCLUSIONS: We developed and externally validated models for predicting the onset of vasopressors. We found that practice-specific features denoting measurement recency improved local performance and also led to more generalizable models if they are adjusted for during model development but discarded at validation. The role of practice-specific features such as measurement indicators in clinical prediction modeling should be carefully considered if the goal is to develop generalizable models.

The myth of generalisability in clinical research and machine learning in health care.

An emphasis on overly broad notions of generalisability as it pertains to applications of machine learning in health care can overlook situations in which machine learning might provide clinical utility. We believe that this narrow focus on generalisability should be replaced with wider considerations for the ultimate goal of building machine learning systems that are useful at the bedside.

Machine learning for early detection of sepsis: an internal and temporal validation study.

OBJECTIVE: Determine if deep learning detects sepsis earlier and more accurately than other models. To evaluate model performance using implementation-oriented metrics that simulate clinical practice. MATERIALS AND METHODS: We trained internally and temporally validated a deep learning model (multi-output Gaussian process and recurrent neural network [MGP-RNN]) to detect sepsis using encounters from adult hospitalized patients at a large tertiary academic center. Sepsis was defined as the presence of 2 or more systemic inflammatory response syndrome (SIRS) criteria, a blood culture order, and at least one element of end-organ failure. The training dataset included demographics, comorbidities, vital signs, medication administrations, and labs from October 1, 2014 to December 1, 2015, while the temporal validation dataset was from March 1, 2018 to August 31, 2018. Comparisons were made to 3 machine learning methods, random forest (RF), Cox regression (CR), and penalized logistic regression (PLR), and 3 clinical scores used to detect sepsis, SIRS, quick Sequential Organ Failure Assessment (qSOFA), and National Early Warning Score (NEWS). Traditional discrimination statistics such as the C-statistic as well as metrics aligned with operational implementation were assessed. RESULTS: The training set and internal validation included 42 979 encounters, while the temporal validation set included 39 786 encounters. The C-statistic for predicting sepsis within 4 h of onset was 0.88 for the MGP-RNN compared to 0.836 for RF, 0.849 for CR, 0.822 for PLR, 0.756 for SIRS, 0.619 for NEWS, and 0.481 for qSOFA. MGP-RNN detected sepsis a median of 5 h in advance. Temporal validation assessment continued to show the MGP-RNN outperform all 7 clinical risk score and machine learning comparisons. CONCLUSIONS: We developed and validated a novel deep learning model to detect sepsis. Using our data elements and feature set, our modeling approach outperformed other machine learning methods and clinical scores.

Assessing Quality of Surgical Real-World Data from an Automated Electronic Health Record Pipeline.

BACKGROUND: Significant analysis errors can be caused by nonvalidated data quality of electronic health records data. To determine surgical data fitness, a framework of foundational and study-specific data analyses was adapted and assessed using conformance, completeness, and plausibility analyses. STUDY DESIGN: Electronic health records-derived data from a cohort of 241,695 patients undergoing 412,182 procedures from October 1, 2014 to August 31, 2018 at 3 hospital sites was evaluated. Data quality analyses tested CPT codes, medication administrations, vital signs, provider notes, labs, orders, diagnosis codes, medication lists, and encounters. RESULTS: Foundational checks showed that all encounters had procedures within the inclusion period, all admission dates occurred before discharge dates, and race was missing for 1% of patients. All procedures had associated CPT codes, 69% had recorded blood pressure, pulse, temperature, respiration rate, and oxygen saturation. After curation, all medication matched RxNorm medication naming standards, 84% of procedures had current outpatient medication lists, and 15% of procedures had missing procedure notes. Study-specific checks temporally validated CPT codes, intraoperative medication doses were in conventional units, and of the 13,500 patients who received blood pressure medication intraoperatively, 93% had a systolic blood pressure >140 mmHg. All procedure notes were completed within less than 30 days of the procedure and 93% of patients after total knee arthroplasty had postoperative physical therapy notes. All patients with postoperative troponin-T lab values ≥0.10 ng/mL had more than 1 ECG with relevant diagnoses. Postoperative opioid prescription decreased by 8.8% and nonopioid use increased by 8.8%. CONCLUSIONS: High levels of conformance, completeness, and clinical plausability demonstrate higher quality of real-world data fitness and low levels demonstrate less-fit-for-use data.

Full depth measurement of tenofovir transport in rectal mucosa using confocal Raman spectroscopy and optical coherence tomography.

The prophylactic activity of antiretroviral drugs applied as microbicides against sexually transmitted HIV is dependent upon their concentrations in infectable host cells. Within mucosal sites of infection (e.g., vaginal and rectal mucosa), those cells exist primarily in the stromal layer of the tissue. Traditional pharmacokinetic studies of these drugs have been challenged by poor temporal and spatial specificity. Newer techniques to measure drug concentrations, involving Raman spectroscopy, have been limited by laser penetration depth into tissue. Utilizing confocal Raman spectroscopy (RS) in conjunction with optical coherence tomography (OCT), a new lateral imaging assay enabled concentration distributions to be imaged with spatial and temporal specificity throughout the full depth of a tissue specimen. The new methodology was applied in rectal tissue using a clinical rectal gel formulation of 1% tenofovir (TFV). Confocal RS revealed diffusion-like behavior of TFV through the tissue specimen, with significant partitioning of the drug at the interface between the stromal and adipose tissue layers. This has implications for drug delivery to infectable tissue sites. The new assay can be applied to rigorously analyze microbicide transport and delineate fundamental transport parameters of the drugs (released from a variety of delivery vehicles) throughout the mucosa, thus informing microbicide product design.

Label-Free Measurements of Tenofovir Diffusion Coefficients in a Microbicide Gel Using Raman Spectroscopy.

Confocal Raman spectroscopy was implemented in a new label-free technique to quantify molecular diffusion coefficients within gels. A leading anti-HIV drug, tenofovir, was analyzed in a clinical microbicide gel. The gel was tested undiluted, and in 10%-50% wt/wt dilutions with vaginal fluid simulant to capture the range of conditions likely occurring in vivo. The concentration distributions of tenofovir in gel over time and space were measured and input to a mathematical diffusion model to deduce diffusion coefficients. These were 3.16 ± 0.11 × 10-6 cm2/s in undiluted gel, and increased by 11%-46% depending on the extent of dilution. Results were interpreted with respect to traditional release rate measurements in devices such as Franz cells. This comparison highlighted an advantage of our assay in that it characterizes the diffusive barrier within the gel material itself; in contrast, release rate in the traditional assay is affected by external conditions, such as drug partitioning at the gel/liquid sink interface. This new assay is relevant to diffusion in polymeric hydrogels over pharmacologically relevant length scales, for example, those characteristic of topical drug delivery. Resulting transport parameters are salient measures of drug delivery potential, and serve as inputs to computational models of drug delivery performance.