Development of a Human Evaluation Framework and Correlation with Automated Metrics for Natural Language Generation of Medical Diagnoses.

In the evolving landscape of clinical Natural Language Generation (NLG), assessing abstractive text quality remains challenging, as existing methods often overlook generative task complexities. This work aimed to examine the current state of automated evaluation metrics in NLG in healthcare. To have a robust and well-validated baseline with which to examine the alignment of these metrics, we created a comprehensive human evaluation framework. Employing ChatGPT-3.5-turbo generative output, we correlated human judgments with each metric. None of the metrics demonstrated high alignment; however, the SapBERT score-a Unified Medical Language System (UMLS)- showed the best results. This underscores the importance of incorporating domain-specific knowledge into evaluation efforts. Our work reveals the deficiency in quality evaluations for generated text and introduces our comprehensive human evaluation framework as a baseline. Future efforts should prioritize integrating medical knowledge databases to enhance the alignment of automated metrics, particularly focusing on refining the SapBERT score for improved assessments.

Alcohol Withdrawal Severity Measures for Identifying Patients Requiring High-Intensity Care.

OBJECTIVES: Alcohol withdrawal syndrome (AWS) may progress to require high-intensity care. Approaches to identify hospitalized patients with AWS who received higher level of care have not been previously examined. This study aimed to examine the utility of Clinical Institute Withdrawal Assessment Alcohol Revised (CIWA-Ar) for alcohol scale scores and medication doses for alcohol withdrawal management in identifying patients who received high-intensity care. DESIGN: A multicenter observational cohort study of hospitalized adults with alcohol withdrawal. SETTING: University of Chicago Medical Center and University of Wisconsin Hospital. PATIENTS: Inpatient encounters between November 2008 and February 2022 with a CIWA-Ar score greater than 0 and benzodiazepine or barbiturate administered within the first 24 hours. The primary composite outcome was patients who progressed to high-intensity care (intermediate care or ICU). INTERVENTIONS: None. MAIN RESULTS: Among the 8742 patients included in the study, 37.5% (n = 3280) progressed to high-intensity care. The odds ratio for the composite outcome increased above 1.0 when the CIWA-Ar score was 24. The sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) at this threshold were 0.12 (95% CI, 0.11-0.13), 0.95 (95% CI, 0.94-0.95), 0.58 (95% CI, 0.54-0.61), and 0.64 (95% CI, 0.63-0.65), respectively. The OR increased above 1.0 at a 24-hour lorazepam milligram equivalent dose cutoff of 15 mg. The sensitivity, specificity, PPV, and NPV at this threshold were 0.16 (95% CI, 0.14-0.17), 0.96 (95% CI, 0.95-0.96), 0.68 (95% CI, 0.65-0.72), and 0.65 (95% CI, 0.64-0.66), respectively. CONCLUSIONS: Neither CIWA-Ar scores nor medication dose cutoff points were effective measures for identifying patients with alcohol withdrawal who received high-intensity care. Research studies for examining outcomes in patients who deteriorate with AWS will require better methods for cohort identification.

Improving immunoassay detection accuracy of anti-SARS-CoV-2 antibodies through dual modality validation.

A novel test strategy is proposed with dual-modality detection techniques for COVID-19 antibody detection. The full-length S protein of SARS-CoV-2 was chemically immobilized on a glass surface to capture anti-SARS-CoV-2 IgG in patient serum and was detected through either Electrochemical Impedance Spectroscopy (EIS) or fluorescence imaging with labeled secondary antibodies. Gold nanoparticles conjugated with protein G were used as the probe and the bound GNP-G was detected through EIS measurements. Anti-human-IgG conjugated with the fluorescent tag Alexa Fluor 488 was used as the probe for fluorescence imaging. Clinical SARS-CoV-2 IgG positive serum and negative controls were used to validate both modalities. For fluorescence-based detection, a high sensitivity was noticed with a quantification range of 0.01-0.1 A.U.C. and a LOD of 0.004 A.U.C. This study demonstrates the possibility of utilizing different measurement techniques in conjunction for improved COVID-19 serology testing.

The Addition of United States Census-Tract Data Does Not Improve the Prediction of Substance Misuse.

Predictors from the structured data in the electronic health record (EHR) have previously been used for case-identification in substance misuse. We aim to examine the added benefit from census-tract data, a proxy for socioeconomic status, to improve identification. A cohort of 186,611 hospitalizations was derived between 2007 and 2017. Reference labels included alcohol misuse only, opioid misuse only, and both alcohol and opioid misuse. Baseline models were created using 24 EHR variables, and enhanced models were created with the addition of 48 census-tract variables from the United States American Community Survey. The absolute net reclassification index (NRI) was applied to measure the benefit in adding census-tract variables to baseline models. The baseline models already had good calibration and discrimination. Adding census-tract variables provided negligible improvement to sensitivity and specificity and NRI was less than 1% across substance groups. Our results show the census-tract added minimal value to prediction models.

Validation of an alcohol misuse classifier in hospitalized patients.

BACKGROUND: Current modes of identifying alcohol misuse in hospitalized patients rely on self-report questionnaires and diagnostic codes that have limitations, including low sensitivity. Information in the clinical notes of the electronic health record (EHR) may further augment the identification of alcohol misuse. Natural language processing (NLP) with supervised machine learning has been successful at analyzing clinical notes and identifying cases of alcohol misuse in trauma patients. METHODS: An alcohol misuse NLP classifier, previously developed on trauma patients who completed the Alcohol Use Disorders Identification Test, was validated in a cohort of 1000 hospitalized patients at a large, tertiary health system between January 1, 2007 and September 1, 2017. The clinical notes were processed using the clinical Text Analysis and Knowledge Extraction System. The National Institute on Alcohol Abuse and Alcoholism (NIAAA) guidelines for alcohol misuse were used during annotation of the medical records in our validation dataset. RESULTS: The alcohol misuse classifier had an area under the receiver operating characteristic curve of 0.91 (95% CI 0.90-0.93) in the cohort of hospitalized patients. The sensitivity, specificity, positive predictive value, and negative predictive value were 0.88 (95% CI 0.85-0.90), 0.78 (95% CI 0.74-0.82), 0.85 (95% CI 0.82-0.87), and 0.82 (95% CI 0.78-0.86), respectively. The Hosmer-Lemeshow Test (p = 0.13) demonstrates good model fit. Additionally, there was a dose-dependent response in alcohol consumption behaviors across increasing strata of predicted probabilities for alcohol misuse. CONCLUSION: The alcohol misuse NLP classifier had good discrimination and test characteristics in hospitalized patients. An approach using the clinical notes with NLP and supervised machine learning may better identify alcohol misuse cases than conventional methods solely relying on billing diagnostic codes.

Natural language processing and machine learning to identify alcohol misuse from the electronic health record in trauma patients: development and internal validation.

Objective: Alcohol misuse is present in over a quarter of trauma patients. Information in the clinical notes of the electronic health record of trauma patients may be used for phenotyping tasks with natural language processing (NLP) and supervised machine learning. The objective of this study is to train and validate an NLP classifier for identifying patients with alcohol misuse. Materials and Methods: An observational cohort of 1422 adult patients admitted to a trauma center between April 2013 and November 2016. Linguistic processing of clinical notes was performed using the clinical Text Analysis and Knowledge Extraction System. The primary analysis was the binary classification of alcohol misuse. The Alcohol Use Disorders Identification Test served as the reference standard. Results: The data corpus comprised 91 045 electronic health record notes and 16 091 features. In the final machine learning classifier, 16 features were selected from the first 24 hours of notes for identifying alcohol misuse. The classifier's performance in the validation cohort had an area under the receiver-operating characteristic curve of 0.78 (95% confidence interval [CI], 0.72 to 0.85). Sensitivity and specificity were at 56.0% (95% CI, 44.1% to 68.0%) and 88.9% (95% CI, 84.4% to 92.8%). The Hosmer-Lemeshow goodness-of-fit test demonstrates the classifier fits the data well (P = .17). A simpler rule-based keyword approach had a decrease in sensitivity when compared with the NLP classifier from 56.0% to 18.2%. Conclusions: The NLP classifier has adequate predictive validity for identifying alcohol misuse in trauma centers. External validation is needed before its application to augment screening.

A broader active site in Pyrococcus horikoshii CoA disulfide reductase accommodates larger substrates and reveals evidence of subunit asymmetry.

Within the family of pyridine nucleotide disulfide oxidoreductase (PNDOR), enzymes are a group of single-cysteine containing FAD-dependent reductases that utilize a tightly bound coenzyme A to assist in the NAD(P)H-dependent reduction of di-, per-, and polysulfide substrates in bacteria and archaea. For many of these homodimeric enzymes, it has proved difficult to determine the substrate specificity and metabolic function based on sequence and genome analysis alone. Coenzyme A-disulfide reductase (CoADR) isolated from Pyrococcus horikoshii (phCoADR) reduces Co-A per- and polysulfides, but, unlike other highly homologous members of this group, is a poor CoA disulfide reductase. The phCoADR structure has a narrower access channel for CoA substrates, which suggested that this restriction might be responsible for the enzyme's poor activity toward the bulky CoA disulfide substrate. To test this hypothesis, the substrate channel was widened by making four mutations along the channel wall (Y65A, Y66A, P67G, and H367G). The structure of the quadruple mutant shows a widened substrate channel, which is supported by a fourfold increase in kcat for the NAD(P)H-dependent reduction of CoA disulfide and enhanced activity toward the substrate at lower temperatures. Anaerobic titrations of the enzyme with NADH revealed a half-site reactivity not observed with the wild-type enzyme in which one subunit of the enzyme could be fully reduced to an EH4 state, while the other remained in an EH2 or EH2·NADH state. These results suggest that for these closely related enzymes, substrate channel morphology is an important determinant of substrate specificity, and homology modeling will be the preferred technique for predicting function among PNDORs.

Preparation of concentrated stable fenofibrate suspensions via liquid antisolvent precipitation.

A major challenge in achieving size stability for relatively high concentration of fine particles from poorly water-soluble drug fenofibrate (FNB) is addressed through T-mixing based liquid antisolvent precipitation in the presence of ultrasonication and judicious use of stabilizers. Multiple stabilizers were screened in a batch mode prior to their continuous formation via T-mixing. In both cases, the stable suspensions maintained their size after 2 days of storage at room temperature, with the smallest particle size of d50: ∼1.2 µm was achieved through a combination of HPMC with SDS or PF-68. The influence of processing parameters, such as sonication energy, sonication probe insert depth and solvent/antisolvent flow rate, on the particle size distribution (PSD) in T-mixing were investigated, to identify optimum processing conditions. Optimal operating and formulation conditions also allowed increase in the drug loading from 0.32% to 4% (w/v), while keeping the median size 2.5 µm. Interestingly, the primary particles observed in the SEM were spherical and under 100 nm in diameter, indicating agglomeration. It was shown that the stabilized particles could be centrifuged and did not show size growth upon resuspension, allowing for increase in the drug loading up to 27% (w/v), which is a significant novel outcome.

Passivation of high-surface-energy sites of milled ibuprofen crystals via dry coating for reduced cohesion and improved flowability.

Ibuprofen micronization with dry coating is investigated to examine its influence on passivation/stabilization of high-surface-energy sites and reduced cohesion. A fluid energy mill was used to micronize ibuprofen particles down to 5-28 µm with or without simultaneous nanosilica coating. Powder flow property and dispersibility were characterized using FT4 powder tester and Rodos/Helos laser diffraction particle sizer. Surface energy was characterized using a next generation inverse gas chromatography instrument. Uncoated micronized ibuprofen showed an increased Lifshitz-van der Waals (LW) dispersion component of surface energy with increasing milling intensity. In contrast, dry-coated milled powders showed a significant reduction in the LW component, whereas physical mixture of uncoated micronized ibuprofen and silica exhibited no reduction in surface energy, indicating that dry coating is necessary for the passivation of high-energy sites of ibuprofen created during micronization. Surface energy of pure micronized ibuprofen was highly heterogeneous, whereas dry-coated ibuprofen had greatly reduced heterogeneity. Micronization with dry coating also improved flowability and bulk density as compared with pure active pharmaceutical ingredient micronization without coating, or just blending with silica. Overall, dry coating leads to decreased cohesion and improved flowability because of reduced LW dispersive component of surface energy and creating nanoscale surface roughness.

Simultaneous micronization and surface modification for improvement of flow and dissolution of drug particles.

Simultaneous micronization and surface modification of drug particles is considered in order to mitigate disadvantages of micronization, e.g., agglomeration, poor flowability, marginal increase in surface area and low bulk density. Particles of ibuprofen (102 µm), a model drug, pre-blended with hydrophilic nano-silica, are micronized down to 10 and 5 µm in a continuous fluid energy mill (FEM) to obtain fine surface modified particles. The solid feeding rate and the grinding pressure are shown as critical parameters for achieving the desired particle size and size distribution. The powder properties were characterized via SEM, laser scattering, powder rheometer with shear-cell, and dissolution test. Significant improvement in flow properties and dissolution rate was observed when micronization accompanied surface modification. Additionally, co-grinding with water-soluble polymer during micronization led to further increase in bulk density and more enhanced dissolution rate improvement, which is attributed to improved wettability. XRD, DSC and Raman were used to examine crystallinity, indicating minimal detectable physical transformation with FEM processed ibuprofen. The surface modified, micronized powders also showed improved dispersion, higher bulk densities (>0.4 g/ml), reduced electrostatic, and higher flowability (FFC ≥ 6) compared to just micronized powder (0.19 g/ml, FFC=1.0), indicating they may be used in high drug loaded formulations amenable to direct compression.

Diastereo- and enantioselective synthesis of alpha,beta-epoxyketones using aqueous NaOCl in conjunction with dihydrocinchonidine derived phase-transfer catalysis at room temperature. Scope and limitations.

In this paper we present studies into the scope and limitations of asymmetric PTC epoxidation of enones and the oxidation-epoxidation of allylic alcohols using aqueous NaOCl in conjunction with a dihydrocinchonidine derived quaternary ammonium salt catalyst.

Asymmetric epoxidation via phase-transfer catalysis: direct conversion of allylic alcohols into alpha,beta-epoxyketones.

Studies into the use of a chiral phase-transfer catalyst in conjunction with sodium hypochlorite to effect the enantio-selective formation of alpha,beta-epoxyketones from allylic alcohols are described.