High Rates of Missed HIV Testing Among Oral PrEP Users in the United States From 2018-2021: A National Assessment on Compliance With HIV Testing Recommendations of the CDC PrEP Guidelines.

Background: The US Centers for Disease Control and Prevention recommends HIV testing every 3 months in oral PrEP users. We performed a national assessment of HIV testing compliance among oral PrEP users. Methods: We analyzed 408 910 PrEP prescriptions issued to 39 809 PrEP users using a national insurance claims database that contained commercial and Medicaid claims. We identified PrEP use based on pharmacy claims and outpatient diagnostic coding. We evaluated the percentage of PrEP prescription refills without HIV testing (identified by CPT codes) within the prior 3, 6, and 12 months using time to event methods. We performed subgroup and multivariate analyses by age, gender, race, insurance type, and geography. Results: Of 39 809 persons, 36 197 were commercially insured, 3612 were Medicaid-insured, and 96% identified as male; the median age (interquartile range) was 34 (29-44) years, and the Medicaid-insured PrEP users were 24% Black/African American, 44% White, and 9% Hispanic/Latinx. Within the prior 3, 6, and 12 months, respectively, the percentage of PrEP prescription fills in individuals without HIV Ag/Ab testing was 34.3% (95% CI, 34.2%-34.5%), 23.8% (95% CI, 23.7%-23.9%), and 16.6% (95% CI, 16.4%-16.7%), and the percentage without any type of HIV test was 25.8% (95% CI, 25.6%-25.9%), 14.6% (95% CI, 14.5%-14.7%), and 7.8% (95% CI, 7.7%-7.9%). Conclusions: Approximately 1 in 3 oral PrEP prescriptions were filled in persons who had not received an HIV Ag/Ab test within the prior 3 months, with evidence of health disparities. These findings inform clinical PrEP monitoring efforts and compliance with national HIV testing guidance to monitor PrEP users.

Using machine learning to develop smart reflex testing protocols.

OBJECTIVE: Reflex testing protocols allow clinical laboratories to perform second line diagnostic tests on existing specimens based on the results of initially ordered tests. Reflex testing can support optimal clinical laboratory test ordering and diagnosis. In current clinical practice, reflex testing typically relies on simple "if-then" rules; however, this limits the opportunities for reflex testing since most test ordering decisions involve more complexity than traditional rule-based approaches would allow. Here, using the analyte ferritin as an example, we propose an alternative machine learning-based approach to "smart" reflex testing. METHODS: Using deidentified patient data, we developed a machine learning model to predict whether a patient getting CBC testing will also have ferritin testing ordered. We evaluate applications of this model to reflex testing by assessing its performance in comparison to possible rule-based approaches. RESULTS: Our underlying machine learning models performed moderately well in predicting ferritin test ordering (AUC=0.731 in reference to actual ordering) and demonstrated promising potential to underlie key clinical applications. In contrast, none of the many traditionally framed, rule-based, hypothetical reflex protocols we evaluated offered sufficient agreement with actual ordering to be clinically feasible. Using chart review, we further demonstrated that the strategic deployment of our model could avoid important ferritin test ordering errors. CONCLUSIONS: Machine learning may provide a foundation for new types of reflex testing with enhanced benefits for clinical diagnosis.

Using Machine Learning to Develop Smart Reflex Testing Protocols.

Objective: Reflex testing protocols allow clinical laboratories to perform second line diagnostic tests on existing specimens based on the results of initially ordered tests. Reflex testing can support optimal clinical laboratory test ordering and diagnosis. In current clinical practice, reflex testing typically relies on simple "if-then" rules; however, this limits their scope since most test ordering decisions involve more complexity than a simple rule will allow. Here, using the analyte ferritin as an example, we propose an alternative machine learning-based approach to "smart" reflex testing with a wider scope and greater impact than traditional rule-based approaches. Methods: Using patient data, we developed a machine learning model to predict whether a patient getting CBC testing will also have ferritin testing ordered, consider applications of this model to "smart" reflex testing, and evaluate the model by comparing its performance to possible rule-based approaches. Results: Our underlying machine learning models performed moderately well in predicting ferritin test ordering and demonstrated greater suitability to reflex testing than rule-based approaches. Using chart review, we demonstrate that our model may improve ferritin test ordering. Finally, as a secondary goal, we demonstrate that ferritin test results are missing not at random (MNAR), a finding with implications for unbiased imputation of missing test results. Conclusions: Machine learning may provide a foundation for new types of reflex testing with enhanced benefits for clinical diagnosis and laboratory utilization management.

Artificial Intelligence in the Clinical Laboratory: An Overview with Frequently Asked Questions.

This article provides an overview of machine learning fundamentals and some applications of machine learning to clinical laboratory diagnostics and patient management. A key goal of this article is to provide a basic foundation in clinical machine learning for readers with clinical laboratory experience that will set them up for more in-depth study of the topic and/or to become a better collaborator with computational colleagues in the development and deployment of machine learning-based solutions.

Real evidence to assess clinical testing interference risk (REACTIR): A strategy using real world data to assess the prevalence of interfering substances in patients undergoing clinical laboratory testing.

INTRODUCTION: Laboratory test interferences can cause spurious test results and patient harm. Knowing the frequency of various interfering substances in patient populations likely to be tested with a particular laboratory assay may inform test development, test utilization and strategies to mitigate interference risk. METHODS: We developed REACTIR (Real Evidence to Assess Clinical Testing Interference Risk), an approach using real world data to assess the prevalence of various interfering substances in patients tested with a particular type of assay. REACTIR uses administrative real world data to identify and subgroup patient cohorts tested with a particular laboratory test and evaluate interference risk. RESULTS: We demonstrate the application REACTIR to point of care (POC) blood glucose testing. We found that exposure to several substances with the potential to interfere in POC blood glucose tests, including N-acetyl cysteine (NAC) and high dose vitamin C was uncommon in most patients undergoing POC glucose tests with several key exceptions, such as burn patients receiving high dose IV-vitamin C or acetaminophen overdose patients receiving NAC. CONCLUSIONS: Findings from REACTIR may support risk mitigation strategies including targeted clinician education and clinical decision support. Likewise, adaptations of REACTIR to premarket assay development may inform optimal assay design and assessment.

Use of machine learning to predict clinical decision support compliance, reduce alert burden, and evaluate duplicate laboratory test ordering alerts.

OBJECTIVES: While well-designed clinical decision support (CDS) alerts can improve patient care, utilization management, and population health, excessive alerting may be counterproductive, leading to clinician burden and alert fatigue. We sought to develop machine learning models to predict whether a clinician will accept the advice provided by a CDS alert. Such models could reduce alert burden by targeting CDS alerts to specific cases where they are most likely to be effective. MATERIALS AND METHODS: We focused on a set of laboratory test ordering alerts, deployed at 8 hospitals within the Partners Healthcare System. The alerts notified clinicians of duplicate laboratory test orders and advised discontinuation. We captured key attributes surrounding 60 399 alert firings, including clinician and patient variables, and whether the clinician complied with the alert. Using these data, we developed logistic regression models to predict alert compliance. RESULTS: We identified key factors that predicted alert compliance; for example, clinicians were less likely to comply with duplicate test alerts triggered in patients with a prior abnormal result for the test or in the context of a nonvisit-based encounter (eg, phone call). Likewise, differences in practice patterns between clinicians appeared to impact alert compliance. Our best-performing predictive model achieved an area under the receiver operating characteristic curve (AUC) of 0.82. Incorporating this model into the alerting logic could have averted more than 1900 alerts at a cost of fewer than 200 additional duplicate tests. CONCLUSIONS: Deploying predictive models to target CDS alerts may substantially reduce clinician alert burden while maintaining most or all the CDS benefit.

Development of a "meta-model" to address missing data, predict patient-specific cancer survival and provide a foundation for clinical decision support.

OBJECTIVE: Like most real-world data, electronic health record (EHR)-derived data from oncology patients typically exhibits wide interpatient variability in terms of available data elements. This interpatient variability leads to missing data and can present critical challenges in developing and implementing predictive models to underlie clinical decision support for patient-specific oncology care. Here, we sought to develop a novel ensemble approach to addressing missing data that we term the "meta-model" and apply the meta-model to patient-specific cancer prognosis. MATERIALS AND METHODS: Using real-world data, we developed a suite of individual random survival forest models to predict survival in patients with advanced lung cancer, colorectal cancer, and breast cancer. Individual models varied by the predictor data used. We combined models for each cancer type into a meta-model that predicted survival for each patient using a weighted mean of the individual models for which the patient had all requisite predictors. RESULTS: The meta-model significantly outperformed many of the individual models and performed similarly to the best performing individual models. Comparisons of the meta-model to a more traditional imputation-based method of addressing missing data supported the meta-model's utility. CONCLUSIONS: We developed a novel machine learning-based strategy to underlie clinical decision support and predict survival in cancer patients, despite missing data. The meta-model may more generally provide a tool for addressing missing data across a variety of clinical prediction problems. Moreover, the meta-model may address other challenges in clinical predictive modeling including model extensibility and integration of predictive algorithms trained across different institutions and datasets.

Iron studies and transferrin, a source of test ordering confusion highly amenable to clinical decision support.

INTRODUCTION: An important cause of laboratory test misordering and overutilization is clinician confusion between tests with similar sounding names or similar indications. We identified an area of test ordering confusion with iron studies that involves total iron binding capacity (TIBC), transferrin, and transferrin saturation. We observed concurrent ordering of direct transferrin along with TIBC at many hospitals within our health system and suspected this was unnecessary. METHODS: We extracted patient test results for transferrin, TIBC and other biomarkers. Using these data, we evaluated both patterns of test utilization and test result concordance. We implemented a clinical decision support (CDS) alert to discourage unnecessary orders for direct transferrin. RESULTS: Using linear regression, we were able to predict transferrin from either TIBC alone or TIBC with other analytes with a high degree of accuracy, demonstrating that in most cases, direct transferrin in combination with TIBC provides little if any additional diagnostic information beyond TIBC alone. The CDS alert proved highly effective in reducing transferrin test utilization at four different hospitals. CONCLUSIONS: Concurrent ordering of direct transferrin and TIBC should usually be avoided. Removal of transferrin or TIBC from the test menu or implementation of CDS may improve utilization of these tests.

Use of Routine Complete Blood Count Results to Rule Out Anaplasmosis Without the Need for Specific Diagnostic Testing.

BACKGROUND: Anaplasmosis presents with fever, headache, and laboratory abnormalities including leukopenia and thrombocytopenia. Polymerase chain reaction (PCR) is the preferred diagnostic but is overutilized. We determined if routine laboratory tests could exclude anaplasmosis, improving PCR utilization. METHODS: Anaplasma PCR results from a 3-year period, with associated complete blood count (CBC) and liver function test results, were retrospectively reviewed. PCR rejection criteria, based on white blood cell (WBC) and platelet (PLT) counts, were developed and prospectively applied in a mock stewardship program. If rejection criteria were met, a committee mock-refused PCR unless the patient was clinically unstable or immunocompromised. RESULTS: WBC and PLT counts were the most actionable routine tests for excluding anaplasmosis. Retrospective review demonstrated that rejection criteria of WBC ≥11 000 cells/µL or PLT ≥300 000 cells/µL would have led to PCR refusal in 428 of 1685 true-negative cases (25%) and 3 of 66 true-positive cases (5%) involving clinically unstable or immunocompromised patients. In the prospective phase, 155 of 663 PCR requests (23%) met rejection criteria and were reviewed by committee, which endorsed refusal in 110 of 155 cases (71%) and approval in 45 (29%), based on clinical criteria. PCR was negative in all 45 committee-approved cases. Only 1 of 110 mock-refused requests yielded a positive PCR result; this patient was already receiving doxycycline at the time of testing. CONCLUSIONS: A CBC-based stewardship algorithm would reduce unnecessary Anaplasma PCR testing, without missing active cases. Although the prospectively evaluated screening approach involved medical record review, this was unnecessary to prevent errors and could be replaced by a rejection comment specifying clinical situations that might warrant overriding the algorithm.

Laboratory Blood-Based Testing for Non-Lyme Disease Tick-Borne Infections at a National Reference Laboratory.

OBJECTIVES: We evaluated trends in non-Lyme disease tick-borne disease (NLTBI) testing at a national reference laboratory. METHODS: Testing data performed at Quest Diagnostics during 2010 to 2016 were analyzed nationally and at the state level. RESULTS: Testing and positivity for most NLTBIs increased dramatically from 2010 through 2016 based on testing from a large reference laboratory. The number of positive cases, though not as stringent as criteria for public health reporting, generally exceeds that reported by the Centers for Disease Control and Prevention. The frequency of NLTBI in the US is seasonal but testing activity and positive test results are observed throughout all months of the year. Positive results for NLTBI testing mostly originated from a limited number of states, indicating the geographic concentration and distribution of NLTBIs reported in this study. CONCLUSIONS: This report provides an important complementary source of data to best understand trends in and spread of NLTBI.

Validation and Implementation of an Ordering Alert to Improve the Efficiency of Monoclonal Gammopathy Evaluation.

OBJECTIVES: To evaluate the use of a provider ordering alert to improve laboratory efficiency and reduce costs. METHODS: We conducted a retrospective study to assess the use of an institutional reflex panel for monoclonal gammopathy evaluation. We then created a clinical decision support (CDS) alert to educate and encourage providers to change their less-efficient orders to the reflex panel. RESULTS: Our retrospective analysis demonstrated that an institutional reflex panel could be safely substituted for a less-efficient and higher-cost panel. The implemented CDS alert resulted in 79% of providers changing their high-cost order panel to an order panel based on the reflex algorithm. CONCLUSIONS: The validated decision support alert demonstrated high levels of provider acceptance and directly led to operational and cost savings within the laboratory. Furthermore, these studies highlight the value of laboratory involvement with CDS efforts to provide agile and targeted provider ordering assistance.

Machine Learning Models Improve the Diagnostic Yield of Peripheral Blood Flow Cytometry.

OBJECTIVES: Peripheral blood flow cytometry (PBFC) is useful for evaluating circulating hematologic malignancies (HM) but has limited diagnostic value for screening. We used machine learning to evaluate whether clinical history and CBC/differential parameters could improve PBFC utilization. METHODS: PBFC cases with concurrent/recent CBC/differential were split into training (n = 626) and test (n = 159) cohorts. We classified PBFC results with abnormal blast/lymphoid populations as positive and used two models to predict results. RESULTS: Positive PBFC results were seen in 58% and 21% of training cases with and without prior HM (P < .001). % neutrophils, absolute lymphocyte count, and % blasts/other cells differed significantly between positive and negative PBFC groups (areas under the curve [AUC] > 0.7). Among test cases, a decision tree model achieved 98% sensitivity and 65% specificity (AUC = 0.906). A logistic regression model achieved 100% sensitivity and 54% specificity (AUC = 0.919). CONCLUSIONS: We outline machine learning-based triaging strategies to decrease unnecessary utilization of PBFC by 35% to 40%.

Early Prediction of Acute Kidney Injury in Critical Care Setting Using Clinical Notes and Structured Multivariate Physiological Measurements.

The onset of acute kidney injury (AKI) during an intensive care unit (ICU) admission is associated with increased morbidity and mortality. Developing novel methods to identify early AKI onset is of critical importance in preventing or reducing AKI complications. We built and applied multiple machine learning models to integrate clinical notes and structured physiological measurements and estimate the risk of new AKI onset using the MIMIC-III database. From the clinical notes, we generated clinically meaningful word representations and embeddings. Four supervised learning classifiers and mixed-feature deep learning architecture were used to construct prediction models. The best configurations consistently utilized both structured and unstructured clinical features and yielded competitive AUCs above 0.83. Our work suggests that integrating structured and unstructured clinical features can be effectively applied to assist clinicians in identifying the risk of incident AKI onset in critically-ill patients upon admission to the ICU.

Machine Learning and Other Emerging Decision Support Tools.

Emerging applications of machine learning and artificial intelligence offer the opportunity to discover new clinical knowledge through secondary exploration of existing patient medical records. This new knowledge may in turn offer a foundation to build new types of clinical decision support (CDS) that provide patient-specific insights and guidance across a wide range of clinical questions and settings. This article will provide an overview of these emerging approaches to CDS, discussing both existing technologies as well as challenges that health systems and informaticists will need to address to allow these emerging approaches to reach their full potential.

Laboratory Blood-Based Testing for Lyme Disease at a National Reference Laboratory.

OBJECTIVES: We evaluated trends in Lyme disease (LD) testing at a national reference laboratory. METHODS: LD screening enzyme immunoassay and Western blot testing data performed at Quest Diagnostics during 2010 to 2016 were analyzed nationally and at the state level. RESULTS: Overall, 593,800 (11.3%) results were positive of 5,255,636 tests. There was an increase in the rate of positivity over the last 2 years of the study and an increase in the number of positive tests in 2016. Positive tests were observed in all 50 states and the District of Columbia. New York had the most positive tests, whereas Connecticut had the highest positivity rate when normalized to state populations. Some states with historically low rates of LD (eg, Texas, Florida, and California) showed significant increases in testing and positivity rates over time. CONCLUSIONS: LD testing and positivity have increased in recent years, including in states not historically associated with the disease.

Order Indication Solicitation to Assess Clinical Laboratory Test Utilization: D-Dimer Order Patterns as an Illustrative Case.

BACKGROUND: A common challenge in the development of laboratory clinical decision support (CDS) and laboratory utilization management (UM) initiatives stems from the fact that many laboratory tests have multiple potential indications, limiting the ability to develop context-specific alerts. As a potential solution, we designed a CDS alert that asks the ordering clinician to provide the indication for testing, using D-dimer as an exemplar. Using data collected over a nearly 3-year period, we sought to determine whether the indication capture was a useful feature within the CDS alert and whether it provided actionable intelligence to guide the development of an UM strategy. METHODS: We extracted results and ordering data for D-dimer testing performed in our laboratory over a 35-month period. We analyzed order patterns by clinical indication, hospital service, and length of hospitalization. RESULTS: Our final data set included 13,971 result-order combinations and indeed provided actionable intelligence regarding test utilization patterns. For example, pulmonary embolism was the most common emergency department indication (86%), while disseminated intravascular coagulation was the most common inpatient indication (56%). D-dimer positivity rates increased with the duration of hospitalization and our data suggested limited utility for ordering this test in the setting of suspected venous thromboembolic disease in admitted patients. In addition, we found that D-dimer was ordered for unexpected indications including the assessment of stroke, dissection, and extracorporeal membrane oxygenation. CONCLUSIONS: Indication capture within a CDS alert and correlation with result data can provide insight into order patterns which can be used to develop future CDS strategies to guide appropriate test use by clinical indication.