Piloting a Clinical Decision Support Tool to Identify Patients With Social Needs and Provide Navigation Services and Referral to Community-Based Organizations: Protocol for a Randomized Controlled Trial.

BACKGROUND: Social needs and social determinants of health (SDOH) significantly outrank medical care when considering the impact on a person's length and quality of life, resulting in poor health outcomes and worsening life expectancy. Integrating social needs and SDOH data along with clinical risk information within operational clinical decision support (CDS) systems built into electronic health records (EHRs) is an effective approach to addressing health-related social needs. To achieve this goal, applied research is needed to develop EHR-integrated CDS tools and closed-loop referral systems and implement and test them in the digital and clinical workflows at health care systems and collaborating community-based organizations (CBOs). OBJECTIVE: This study aims to describe the protocol for a mixed methods study including a randomized controlled trial and a qualitative phase assessing the feasibility, acceptability, and effectiveness of an EHR-integrated digital platform to identify patients with social needs and provide navigation services and closed-loop referrals to CBOs to address their social needs. METHODS: The randomized controlled trial will enroll and randomize adult patients living in socioeconomically challenged neighborhoods in Baltimore City receiving care at a single academic health care institution in the 3-month intervention (using the digital platform) or the 3-month control (standard-of-care assessment and addressing of social needs) arms (n=295 per arm). To evaluate the feasibility and acceptability of the digital platform and its impact on the clinical and digital workflow and patient care, we will conduct focus groups with the care teams in the health care system (eg, clinical providers, social workers, and care managers) and collaborating CBOs. The outcomes will be the acceptability, feasibility, and effectiveness of the CDS tool and closed-loop referral system. RESULTS: This clinical trial opened to enrollment in June 2023 and will be completed in March 2025. Initial results are expected to be published in spring 2025. We will report feasibility outcome measures as weekly use rates of the digital platform. The acceptability outcome measure will be the provider's and patient's responses to the truthfulness of a statement indicating a willingness to use the platform in the future. Effectiveness will be measured by tracking a 3-month change in identified social needs and provided navigation services as well as clinical outcomes such as hospitalization and emergency department visits. CONCLUSIONS: The results of this investigation are expected to contribute to our understanding of the use of digital interventions and the implementation of such interventions in digital and clinical workflows to enhance the health care system and CBO ability related to social needs assessment and intervention. These results may inform the construction of a future multi-institutional trial designed to test the effectiveness of this intervention across different health care systems and care settings. TRIAL REGISTRATION: ClinicalTrials.gov NCT05574699; https://clinicaltrials.gov/study/NCT05574699. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/57316.

Validation of an algorithm to prioritize patients for comprehensive medication management in primary care settings.

BACKGROUND: Comprehensive medication management (CMM) programs optimize the effectiveness and safety of patients' medication regimens, but CMM may be underutilized. Whether healthcare claims data can identify patients appropriate for CMM is not well-studied. AIM: Determine the face validity of a claims-based algorithm to prioritize patients who likely need CMM. METHOD: We used claims data to construct patient-level markers of "regimen complexity" and "high-risk for adverse effects," which were combined to define four categories of claims-based CMM-need (very likely, likely, unlikely, very unlikely) among 180 patient records. Three clinicians independently reviewed each record to assess CMM need. We assessed concordance between the claims-based and clinician-review CMM need by calculating percent agreement as well as kappa statistic. RESULTS: Most records identified as 'very likely' (90%) by claims-based markers were identified by clinician-reviewers as needing CMM. Few records within the 'very unlikely' group (5%) were identified by clinician-reviewers as needing CMM. Interrater agreement between CMM-based algorithm and clinician review was moderate in strength (kappa = 0.6, p < 0.001). CONCLUSION: Claims-based pharmacy measures may offer a valid approach to prioritize patients into CMM-need groups. Further testing of this algorithm is needed prior to implementation in clinic settings.

Assessing racial bias in healthcare predictive models: Practical lessons from an empirical evaluation of 30-day hospital readmission models.

OBJECTIVE: Despite increased availability of methodologies to identify algorithmic bias, the operationalization of bias evaluation for healthcare predictive models is still limited. Therefore, this study proposes a process for bias evaluation through an empirical assessment of common hospital readmission models. The process includes selecting bias measures, interpretation, determining disparity impact and potential mitigations. METHODS: This retrospective analysis evaluated racial bias of four common models predicting 30-day unplanned readmission (i.e., LACE Index, HOSPITAL Score, and the CMS readmission measure applied as is and retrained). The models were assessed using 2.4 million adult inpatient discharges in Maryland from 2016 to 2019. Fairness metrics that are model-agnostic, easy to compute, and interpretable were implemented and apprised to select the most appropriate bias measures. The impact of changing model's risk thresholds on these measures was further assessed to guide the selection of optimal thresholds to control and mitigate bias. RESULTS: Four bias measures were selected for the predictive task: zero-one-loss difference, false negative rate (FNR) parity, false positive rate (FPR) parity, and generalized entropy index. Based on these measures, the HOSPITAL score and the retrained CMS measure demonstrated the lowest racial bias. White patients showed a higher FNR while Black patients resulted in a higher FPR and zero-one-loss. As the models' risk threshold changed, trade-offs between models' fairness and overall performance were observed, and the assessment showed all models' default thresholds were reasonable for balancing accuracy and bias. CONCLUSIONS: This study proposes an Applied Framework to Assess Fairness of Predictive Models (AFAFPM) and demonstrates the process using 30-day hospital readmission model as the example. It suggests the feasibility of applying algorithmic bias assessment to determine optimized risk thresholds so that predictive models can be used more equitably and accurately. It is evident that a combination of qualitative and quantitative methods and a multidisciplinary team are necessary to identify, understand and respond to algorithm bias in real-world healthcare settings. Users should also apply multiple bias measures to ensure a more comprehensive, tailored, and balanced view. The results of bias measures, however, must be interpreted with caution and consider the larger operational, clinical, and policy context.

Effectiveness of telehealth versus in-person care during the COVID-19 pandemic: a systematic review.

In this systematic review, we compared the effectiveness of telehealth with in-person care during the pandemic using PubMed, CINAHL, PsycINFO, and the Cochrane Central Register of Controlled Trials from March 2020 to April 2023. We included English-language, U.S.-healthcare relevant studies comparing telehealth with in-person care conducted after the onset of the pandemic. Two reviewers independently screened search results, serially extracted data, and independently assessed the risk of bias and strength of evidence. We identified 77 studies, the majority of which (47, 61%) were judged to have a serious or high risk of bias. Differences, if any, in healthcare utilization and clinical outcomes between in-person and telehealth care were generally small and/or not clinically meaningful and varied across the type of outcome and clinical area. For process outcomes, there was a mostly lower rate of missed visits and changes in therapy/medication and higher rates of therapy/medication adherence among patients receiving an initial telehealth visit compared with those receiving in-person care. However, the rates of up-to-date labs/paraclinical assessment were also lower among patients receiving an initial telehealth visit compared with those receiving in-person care. Most studies lacked a standardized approach to assessing outcomes. While we refrain from making an overall conclusion about the performance of telehealth versus in-person visits the use of telehealth is comparable to in-person care across a variety of outcomes and clinical areas. As we transition through the COVID-19 era, models for integrating telehealth with traditional care become increasingly important, and ongoing evaluations of telehealth will be particularly valuable.

Development of a hospital coding intensity measure based on sepsis diagnoses.

Significant variation in coding intensity among hospitals has been observed and can lead to reimbursement inequities and inadequate risk adjustment for quality measures. Reliable tools to quantify hospital coding intensity are needed. We hypothesized that coded sepsis rates among patients hospitalized with common infections may serve as a useful surrogate for coding intensity and derived a hospital-level sepsis coding intensity measure using prevalence of "sepsis" primary diagnoses among patients hospitalized with urinary tract infection, cellulitis, and pneumonia. This novel measure was well correlated with the hospital mean number of discharge diagnoses, which has historically been used to quantify hospital-level coding intensity. However, it has the advantage of inferring hospital coding intensity without the strong association with comorbidity that the mean number of discharge diagnoses has. Our measure may serve as a useful tool to compare coding intensity across institutions.

Association of buprenorphine retention and subsequent adverse outcomes following non-fatal overdose: An analysis using statewide linked Maryland databases.

INTRODUCTION: Patients receiving buprenorphine after a non-fatal overdose have lower risk of future nonfatal or fatal overdose, but less is known about the relationship between buprenorphine retention and the risk of adverse outcomes in the post-overdose year. OBJECTIVE: To examine the relationship between the total number of months with an active buprenorphine prescription (retention) and the odds of an adverse outcome within the 12 months following an index non-fatal overdose. MATERIALS AND METHODS: We studied a cohort of people with an index non-fatal opioid overdose in Maryland between July 2016 and December 2020 and at least one filled buprenorphine prescription in the 12-month post-overdose observation period. We used individually linked Maryland prescription drug and hospital admissions data. Multivariable logistic regression models were used to examine buprenorphine retention and associated odds of experiencing a second non-fatal overdose, all-cause emergency department visits, and all-cause hospitalizations. RESULTS: Of 5439 people, 25% (n=1360) experienced a second non-fatal overdose, 78% had an (n=4225) emergency department visit, and 37% (n=2032) were hospitalized. With each additional month of buprenorphine, the odds of experiencing another non-fatal overdose decreased by 4.7%, all-cause emergency department visits by 5.3%, and all-cause hospitalization decreased by 3.9% (p<.0001, respectively). Buprenorphine retention for at least nine months was a critical threshold for reducing overdose risk versus shorter buprenorphine retention. CONCLUSIONS: Buprenorphine retention following an index non-fatal overdose event significantly decreases the risk of future overdose, emergency department use, and hospitalization even among people already on buprenorphine.

Development of a Social Risk Score in the Electronic Health Record to Identify Social Needs Among Underserved Populations: Retrospective Study.

BACKGROUND: Patients with unmet social needs and social determinants of health (SDOH) challenges continue to face a disproportionate risk of increased prevalence of disease, health care use, higher health care costs, and worse outcomes. Some existing predictive models have used the available data on social needs and SDOH challenges to predict health-related social needs or the need for various social service referrals. Despite these one-off efforts, the work to date suggests that many technical and organizational challenges must be surmounted before SDOH-integrated solutions can be implemented on an ongoing, wide-scale basis within most US-based health care organizations. OBJECTIVE: We aimed to retrieve available information in the electronic health record (EHR) relevant to the identification of persons with social needs and to develop a social risk score for use within clinical practice to better identify patients at risk of having future social needs. METHODS: We conducted a retrospective study using EHR data (2016-2021) and data from the US Census American Community Survey. We developed a prospective model using current year-1 risk factors to predict future year-2 outcomes within four 2-year cohorts. Predictors of interest included demographics, previous health care use, comorbidity, previously identified social needs, and neighborhood characteristics as reflected by the area deprivation index. The outcome variable was a binary indicator reflecting the likelihood of the presence of a patient with social needs. We applied a generalized estimating equation approach, adjusting for patient-level risk factors, the possible effect of geographically clustered data, and the effect of multiple visits for each patient. RESULTS: The study population of 1,852,228 patients included middle-aged (mean age range 53.76-55.95 years), White (range 324,279/510,770, 63.49% to 290,688/488,666, 64.79%), and female (range 314,741/510,770, 61.62% to 278,488/448,666, 62.07%) patients from neighborhoods with high socioeconomic status (mean area deprivation index percentile range 28.76-30.31). Between 8.28% (37,137/448,666) and 11.55% (52,037/450,426) of patients across the study cohorts had at least 1 social need documented in their EHR, with safety issues and economic challenges (ie, financial resource strain, employment, and food insecurity) being the most common documented social needs (87,152/1,852,228, 4.71% and 58,242/1,852,228, 3.14% of overall patients, respectively). The model had an area under the curve of 0.702 (95% CI 0.699-0.705) in predicting prospective social needs in the overall study population. Previous social needs (odds ratio 3.285, 95% CI 3.237-3.335) and emergency department visits (odds ratio 1.659, 95% CI 1.634-1.684) were the strongest predictors of future social needs. CONCLUSIONS: Our model provides an opportunity to make use of available EHR data to help identify patients with high social needs. Our proposed social risk score could help identify the subset of patients who would most benefit from further social needs screening and data collection to avoid potentially more burdensome primary data collection on all patients in a target population of interest.

Evaluating Algorithmic Bias in 30-Day Hospital Readmission Models: Retrospective Analysis.

BACKGROUND: The adoption of predictive algorithms in health care comes with the potential for algorithmic bias, which could exacerbate existing disparities. Fairness metrics have been proposed to measure algorithmic bias, but their application to real-world tasks is limited. OBJECTIVE: This study aims to evaluate the algorithmic bias associated with the application of common 30-day hospital readmission models and assess the usefulness and interpretability of selected fairness metrics. METHODS: We used 10.6 million adult inpatient discharges from Maryland and Florida from 2016 to 2019 in this retrospective study. Models predicting 30-day hospital readmissions were evaluated: LACE Index, modified HOSPITAL score, and modified Centers for Medicare & Medicaid Services (CMS) readmission measure, which were applied as-is (using existing coefficients) and retrained (recalibrated with 50% of the data). Predictive performances and bias measures were evaluated for all, between Black and White populations, and between low- and other-income groups. Bias measures included the parity of false negative rate (FNR), false positive rate (FPR), 0-1 loss, and generalized entropy index. Racial bias represented by FNR and FPR differences was stratified to explore shifts in algorithmic bias in different populations. RESULTS: The retrained CMS model demonstrated the best predictive performance (area under the curve: 0.74 in Maryland and 0.68-0.70 in Florida), and the modified HOSPITAL score demonstrated the best calibration (Brier score: 0.16-0.19 in Maryland and 0.19-0.21 in Florida). Calibration was better in White (compared to Black) populations and other-income (compared to low-income) groups, and the area under the curve was higher or similar in the Black (compared to White) populations. The retrained CMS and modified HOSPITAL score had the lowest racial and income bias in Maryland. In Florida, both of these models overall had the lowest income bias and the modified HOSPITAL score showed the lowest racial bias. In both states, the White and higher-income populations showed a higher FNR, while the Black and low-income populations resulted in a higher FPR and a higher 0-1 loss. When stratified by hospital and population composition, these models demonstrated heterogeneous algorithmic bias in different contexts and populations. CONCLUSIONS: Caution must be taken when interpreting fairness measures' face value. A higher FNR or FPR could potentially reflect missed opportunities or wasted resources, but these measures could also reflect health care use patterns and gaps in care. Simply relying on the statistical notions of bias could obscure or underplay the causes of health disparity. The imperfect health data, analytic frameworks, and the underlying health systems must be carefully considered. Fairness measures can serve as a useful routine assessment to detect disparate model performances but are insufficient to inform mechanisms or policy changes. However, such an assessment is an important first step toward data-driven improvement to address existing health disparities.

Prevalence of lifestyle-related behavioral information in claims data in the U.S.

OBJECTIVE: Given their association with varying health risks, lifestyle-related behaviors are essential to consider in population-level disease prevention. Health insurance claims are a key source of information for population health analytics, but the availability of lifestyle information within claims data is unknown. Our goal was to assess the availability and prevalence of data items that describe lifestyle behaviors across several domains within a large U.S. claims database. METHODS: We conducted a retrospective, descriptive analysis to determine the availability of the following claims-derived lifestyle domains: nutrition, eating habits, physical activity, weight status, emotional wellness, sleep, tobacco use, and substance use. To define these domains, we applied a serial review process with three physicians to identify relevant diagnosis and procedure codes within claims for each domain. We used enrollment files and medical claims from a large national U.S. health plan to identify lifestyle relevant codes filed between 2016 and 2020. We calculated the annual prevalence of each claims-derived lifestyle domain and the proportion of patients by count within each domain. RESULTS: Approximately half of all members within the sample had claims information that identified at least one lifestyle domain (2016 = 41.9%; 2017 = 46.1%; 2018 = 49.6%; 2019 = 52.5%; 2020 = 50.6% of patients). Most commonly identified domains were weight status (19.9-30.7% across years), nutrition (13.3-17.8%), and tobacco use (7.9-9.8%). CONCLUSION: Our study demonstrates the feasibility of using claims data to identify key lifestyle behaviors. Additional research is needed to confirm the accuracy and validity of our approach and determine its use in population-level disease prevention.

Patterns of Morbidity Across the Lifespan: A Population Segmentation Framework for Classifying Health Care Needs for All Ages.

BACKGROUND: Classification systems to segment such patients into subgroups for purposes of care management and population analytics should balance administrative simplicity with clinical meaning and measurement precision. OBJECTIVE: To describe and empirically apply a new clinically relevant population segmentation framework applicable to all payers and all ages across the lifespan. RESEARCH DESIGN AND SUBJECTS: Cross-sectional analyses using insurance claims database for 3.31 Million commercially insured and 1.05 Million Medicaid enrollees under 65 years old; and 5.27 Million Medicare fee-for-service beneficiaries aged 65 and older. MEASURES: The "Patient Need Groups" (PNGs) framework, we developed, classifies each person within the entire 0-100+ aged population into one of 11 mutually exclusive need-based categories. For each PNG segment, we documented a range of clinical and resource endpoints, including health care resource use, avoidable emergency department visits, hospitalizations, behavioral health conditions, and social need factors. RESULTS: The PNG categories included: (1) nonuser, (2) low-need child, (3) low-need adult, (4) low-complexity multimorbidity, (5) medium-complexity multimorbidity, (6) low-complexity pregnancy, (7) high-complexity pregnancy, (8) dominant psychiatric/behavioral condition, (9) dominant major chronic condition, (10) high-complexity multimorbidity, and (11) frailty. Each PNG evidenced a characteristic age-related trajectory across the full lifespan. In addition to offering clinically cogent groupings, large percentages (29%-62%) of patients in two pregnancy and high-complexity multimorbidity and frailty PNGs were in a high-risk subgroup (upper 10%) of potential future health care utilization. CONCLUSIONS: The PNG population segmentation approach represents a comprehensive measurement framework that captures and categorizes available electronic health care data to characterize individuals of all ages based on their needs.

Trends and Patterns of Social History Data Collection Within an Electronic Health Record.

There is increased acceptance that social and behavioral determinants of health (SBDH) impact health outcomes, but electronic health records (EHRs) are not always set up to capture the full range of SBDH variables in a systematic manner. The purpose of this study was to explore rates and trends of social history (SH) data collection-1 element of SBDH-in a structured portion of an EHR within a large academic integrated delivery system. EHR data for individuals with at least 1 visit in 2017 were included in this study. Completeness rates were calculated for how often SBDH variable was assessed and documented. Logistic regressions identified factors associated with assessment rates for each variable. A total of 44,166 study patients had at least 1 SH variable present. Tobacco use and alcohol use were the most frequently captured SH variables. Black individuals were more likely to have their alcohol use assessed (odds ratio [OR] 1.21) compared with White individuals, whereas White individuals were more likely to have their "smokeless tobacco use" assessed (OR 0.92). There were also differences between insurance types. Drug use was more likely to be assessed in the Medicaid population for individuals who were single (OR 0.95) compared with the commercial population (OR 1.05). SH variable assessment is inconsistent, which makes use of EHR data difficult to gain better understanding of the impact of SBDH on health outcomes. Standards and guidelines on how and why to collect SBDH information within the EHR are needed.

Impact of the patient-centered medical home on consistently high-cost patients.

OBJECTIVE: To evaluate the effect of a statewide multipayer patient-centered medical home (PCMH) demonstration on patients consistently within the highest ranks of health services expenditure across Maryland. STUDY DESIGN: Post hoc longitudinal analyses of administrative data on privately insured patients of medical homes that participated in the Maryland Multi-Payer PCMH Program (MMPP), matched for comparison to medical homes in a single-payer PCMH program and to non-PCMH practices. METHODS: Consistently high-cost patients (CHPs) were defined as being in the top statewide quintile of payer expenditure over a 2-year baseline period. Using population-averaged generalized linear regression models, we evaluated the odds of CHPs remaining in the highest-cost quintile during the 2-year MMPP implementation period and assessed changes in their utilization patterns. RESULTS: Six percent of included patients were CHPs and accounted for one-third of total expenditure. For CHPs in multipayer PCMHs, estimated odds of remaining in this status after 2 years were lower by 34% (adjusted OR [AOR], 0.66; 95% CI, 0.41-0.90; P = .03) relative to CHPs in non-PCMH practices and higher by 41% (AOR, 1.41; 95% CI, 1.08-1.75; P = .004) compared with CHPs in single-payer PCMHs. Relative to CHPs in non-PCMH practices, CHPs in multipayer PCMHs had inpatient admissions decline by 40% (incidence rate ratio [IRR], 0.60; 95% CI, 0.36-1.00; P = .049) and visits to the attributed primary care provider increase by 21% (IRR, 1.21; 95% CI, 1.05-1.39; P = .01). CONCLUSIONS: Relative to routine primary care, the PCMH model significantly reduces the probability that CHPs remain in this expensive category and enhances continuity of care.

Assessing the Added Value of Vital Signs Extracted from Electronic Health Records in Healthcare Risk Adjustment Models.

Purpose: Patient vital signs are related to specific health risks and outcomes but are underutilized in the prediction of health-care utilization and cost. To measure the added value of electronic health record (EHR) extracted Body Mass Index (BMI) and blood pressure (BP) values in improving healthcare risk and utilization predictions. Patients and Methods: A sample of 12,820 adult outpatients from the Johns Hopkins Health System (JHHS) were identified between 2016 and 2017, having high data quality and recorded values for BMI and BP. We evaluated the added value of BMI and BP in predicting health-care utilization and cost through a retrospective cohort design. BMI, mean arterial pressure (MAP), systolic and diastolic BPs were summarized as annual aggregated values. Concurrent annual BMI and MAP changes were quantified as the difference between maximum and minimum recorded values. Model performance estimates consisted of repeated 10-fold cross validation, compared to base model point estimates for demographic and diagnostic, coded events: (1) patient age and sex, (2) age, sex, and the Charlson weighted index, (3) age, sex and the Johns Hopkins ACG system's DxPM risk score. Results: Both categorical BMI and BP were progressively indicative of disease comorbidity, but not uniformly related to health-care utilization or cost. Annual change in BMI and MAP improved predictions for most concurrent year outcomes when compared to base models. Conclusion: When a healthcare system lacks relevant diagnostic or risk assessment information for a patient, vital signs may be useful for a simple estimation of disease risk, cost and utilization.

Impact of Social Needs in Electronic Health Records and Claims on Health Care Utilization and Costs Risk-Adjustment Models Within Medicaid Population.

Patients enrolled in Medicaid have significantly higher social needs (SNs) than others. Using claims and electronic health records (EHRs) data, managed care organizations (MCOs) could systemically identify high-risk patients with SNs and develop population health management interventions. Impact of SNs on models predicting health care utilization and costs was assessed. This retrospective study included claims and EHRs data on 39,267 patients younger than age 65 years who were continuously enrolled during 2018-2019 in a Medicaid-managed care plan. SN marker was developed suggesting presence of International Classification of Diseases, 10th revision codes in any of the 5 SN domains. Impact of SN marker was compared across demographic and 2 diagnosis-based (ie, Charlson and Adjusted Clinical Groups risk score) prediction models of emergency department (ED) visit and hospitalizations, and total, medical, and pharmacy costs. After combining data sources, prevalence of documented SN marker increased from 11% and 13% to 18% of the study population across claims, EHRs, and both combined, respectively. SN marker improved predictions of demographic models for all utilization and total costs outcomes (area under the curve [AUC] of ED model increased from 0.57 to 0.61 and R2 of total cost model increased from 10.9 to 12.2). In both diagnosis-based models, adding SN marker marginally improved outcomes prediction (AUC of ED model increased from 0.65 to 0.66). This study demonstrated feasibility of using claims and EHRs data to systematically capture SNs and incorporate in prediction models that could enable MCOs and policy makers to adjust and develop effective population health interventions.

Comparing person-level matching algorithms to identify risk across disparate datasets among patients with a controlled substance prescription: retrospective analysis.

Background: The opioid epidemic in the United States has precipitated a need for public health agencies to better understand risk factors associated with fatal overdoses. Matching person-level information stored in public health, medical, and human services datasets can enhance the understanding of opioid overdose risk factors and interventions. Objective: This study compares approximate match versus exact match algorithms to link disparate datasets together for identifying persons at risk from an applied perspective. Methods: This study used statewide prescription drug monitoring program (PDMP), arrest, and mortality data matched at the person-level using an approximate match and 2 exact match algorithms. Impact of matching was assessed by analyzing 3 independent concepts: (1) the prevalence of key risk indicators used by PDMP programs in practice, (2) the prevalence of arrests and fatal opioid overdose, and (3) the performance of a multivariate logistic regression for fatal opioid overdose. The PDMP key risk indicators included (1) multiple provider episodes (MPE), or patients with prescriptions from multiple prescribers and dispensers, (2) high morphine milligram equivalents (MMEs), which represents an opioid's potency relative to morphine, and (3) overlapping opioid and benzodiazepine prescriptions. Results: Prevalence of PDMP-based risk indicators were higher in the approximate match population for MPEs (n = 4893/1 859 445 [0.26%]) and overlapping opioid/benzodiazepines (n = 57 888/1 859 445 [4.71%]), but the exact-basic match population had the highest prevalence of individuals with high MMEs (n = 664/1 910 741 [3.11%]). Prevalence of arrests and deaths were highest for the approximate match population compared with the exact match populations. Model performance was comparable across the 3 matching algorithms (exact-basic validation area under the receiver operating characteristic curve [AUC]: 0.854; approximate validation AUC: 0.847; exact + zip validation AUC: 0.826) but resulted in different cutoff points balancing sensitivity and specificity. Conclusions: Our study illustrates the specific tradeoffs of different matching methods. Further research should be performed to compare matching algorithms and its impact on the prevalence of key risk indicators in an applied setting that can improve understanding of risk within a population.

Claims-based pharmacy markers for comprehensive medication management program case identification: Validation against concurrent and prospective healthcare costs and utilization.

BACKGROUND: Three claims-based pharmacy markers (complex, costly and risky medications) were developed to help automatically identify patients for comprehensive medication management. OBJECTIVE: To evaluate the association between newly-developed markers and healthcare outcomes. METHODS: This was a two-year retrospective cohort study using PharMetrics Plus patient-level administrative claims in 2014 and 2015. We included all claims from 1,541,873 individuals with: (1) 24-month medical and pharmacy enrollment in 2014 and 2015, (2) aged between 18 and 63 in 2014, and (3) known gender. Independent/control variables came from 2014 while outcomes came from 2014 (concurrent analysis) and 2015 (prospective analysis). Three pharmacy markers, separately or together, were added to four base models to predict concurrent and prospective healthcare costs (total, medical, and pharmacy) and utilization (having any hospitalization, having any emergency department visit, and having any readmission). We applied linear regression for costs while logistic regression for utilization. Measures of model performances and coefficients were derived from a 5-fold cross-validation repeated 20 times. RESULTS: Individuals with 1+ complex, risky or costly medication markers had higher comorbidity, healthcare costs and utilization than their counterparts. Nine binary risky category markers performed the best among the three types of risky medication markers; the Medication Complexity Score and three-level complex category both outperformed a simpler complex medication indicator. Adding three novel pharmacy markers separately or together into the base models provided the greatest improvement in explaining pharmacy costs, compared with medical (non-medication) costs. These pharmacy markers also added value in explaining healthcare utilization among the simple base models. CONCLUSIONS: Three claims-based pharmacy indicators had positive associations with healthcare outcomes and added value in predicting them. This initial study suggested that these novel markers can be used by pharmacy case management programs to help identify potential high-risk patients most likely to benefit from clinical pharmacist review and other interventions.

Outcomes of In-Person and Telehealth Ambulatory Encounters During COVID-19 Within a Large Commercially Insured Cohort.

Importance: Since the start of the COVID-19 pandemic, few studies have assessed the association of telehealth with outcomes of care, including patterns of health care use after the initial encounter. Objective: To assess the association of telehealth and in-person visits with outcomes of care during the COVID-19 pandemic. Design, Setting, and Participants: This cohort study assessed continuously enrolled members in private health plans of the Blue Cross and Blue Shield Association from July 1, 2019, to December 31, 2020. Main Outcomes and Measures: Main outcomes were ambulatory encounters per enrollee stratified by characteristics derived from enrollment files, practitioner claims, and community characteristics linked to the enrollee's zip code. Outcomes of care were assessed 14 days after the initial encounters and included follow-up encounters of any kind, emergency department encounters, and hospitalizations after initial telehealth or in-person encounters. Results: In this cohort study of 40 739 915 individuals (mean [SD] age, 35.37 [18.77] years; 20 480 768 [50.3%] female), ambulatory encounters decreased by 1.0% and the number of in-person encounters per enrollee decreased by 17.0% from 2019 to 2020; however, as a proportion of all ambulatory encounters, telehealth encounters increased substantially from 0.6% (n = 236 220) to 14.1% (n = 5 743 718). For members with an initial telehealth encounter for a new acute condition, the adjusted odds ratio was 1.44 (95% CI, 1.42-1.46) for all follow-ups combined and 1.11 (95% CI, 1.06-1.16) for an emergency department encounter. For members with an initial telehealth encounter for a new chronic condition, the adjusted odds ratios were 0.94 (95% CI, 0.92-0.95) for all follow-ups combined and 0.94 (95% CI, 0.90-0.99) for in-patient admissions. Conclusions and Relevance: In this cohort study of 40.7 million commercially insured adults, telehealth accounted for a large share of ambulatory encounters at the peak of the pandemic and remained prevalent after infection rates subsided. Telehealth encounters for chronic conditions had similar rates of follow-up to in-person encounters for these conditions, whereas telehealth encounters for acute conditions seemed to be more likely than in-person encounters to require follow-up. These findings suggest a direction for future work and are relevant to policy makers, payers, and practitioners as they manage the use of telehealth during the COVID-19 pandemic and afterward.

Improving the Prediction of Persistent High Health Care Utilizers: Retrospective Analysis Using Ensemble Methodology.

BACKGROUND: A small proportion of high-need patients persistently use the bulk of health care services and incur disproportionate costs. Population health management (PHM) programs often refer to these patients as persistent high utilizers (PHUs). Accurate PHU prediction enables PHM programs to better align scarce health care resources with high-need PHUs while generally improving outcomes. While prior research in PHU prediction has shown promise, traditional regression methods used in these studies have yielded limited accuracy. OBJECTIVE: We are seeking to improve PHU predictions with an ensemble approach in a retrospective observational study design using insurance claim records. METHODS: We defined a PHU as a patient with health care costs in the top 20% of all patients for 4 consecutive 6-month periods. We used 2013 claims data to predict PHU status in next 24 months. Our study population included 165,595 patients in the Johns Hopkins Health Care plan, with 8359 (5.1%) patients identified as PHUs in 2014 and 2015. We assessed the performance of several standalone machine learning methods and then an ensemble approach combining multiple models. RESULTS: The candidate ensemble with complement naïve Bayes and random forest layers produced increased sensitivity and positive predictive value (PPV; 49.0% and 50.3%, respectively) compared to logistic regression (46.8% and 46.1%, respectively). CONCLUSIONS: Our results suggest that ensemble machine learning can improve prediction of care management needs. Improved PPV implies reduced incorrect referral of low-risk patients. With the improved sensitivity/PPV balance of this approach, resources may be directed more efficiently to patients needing them most.

Comparing and validating medication complexity from insurance claims against electronic health records.

BACKGROUND: Patient effort to comply with complex medication instructions is known to be related to nonadherence and subsequent medical complications or health care costs. A widely used Medication Regimen Complexity Index (MRCI) has been used with electronic health records (EHRs) to identify patients who could benefit from pharmacist intervention. A similar claims-derived measure may be better suited for clinical decision support, since claims offer a more complete view of patient care and health utilization. OBJECTIVE: To define and validate a novel insurance claims-based medication complexity score (MCS) patterned after the widely used MRCI, derived from EHRs. METHODS: Insurance claims and EHR data were provided by HealthPartners (N = 54,988) (Bloomington, Minnesota) and The Johns Hopkins Health System (N = 28,589) (Baltimore, Maryland) for years 2013 and 2017, respectively. Yearly measures of medication complexity were developed for each patient and evaluated with one another using rank correlation within different clinical subgroupings. Indicators for the presence of individually complex prescriptions were also developed and assessed using exact agreement. Complexity measures were then correlated with select covariates to further validate the concordance between MCS and MRCI with respect to clinical metrics. These included demographic, comorbidity, and health care utilization markers. Prescribed medications in each system's EHR were coded using the previously validated MRCI weighting rules. Insurance claims for retail pharmacy medications were coded using our novel MCS, which closely followed MRCI scoring rules. RESULTS: EHR-based MRCI and claims-based MCS were significantly correlated with one another for most clinical subgroupings. Likewise, both measures were correlated with several covariates, including count of active medications and chronic conditions. The MCS was, in most cases, more associated with key health covariates than was MRCI, although both were consistently significant. We found that the highest correlation between MCS and MRCI is obtained with patients who have similar counts of pharmacy records between EHRs and claims (HealthPartners: P = 0.796; Johns Hopkins Health System: P = 0.779). CONCLUSIONS: The findings suggest good correspondence between MCS and MRCI and that claims data represent a useful resource for assessing medication complexity. Claims data also have major practical advantages, such as interoperability across health care systems, although they lack the detailed clinical context of EHRs. DISCLOSURES: The Johns Hopkins University holds the copyright to the Adjusted Clinical Groups (ACG) system and receives royalties from the global distribution of the ACG system. This revenue supports a portion of the authors' salary. No additional or external funding supported this work. The authors have no conflict of interest to disclose.

Development and assessment of a natural language processing model to identify residential instability in electronic health records' unstructured data: a comparison of 3 integrated healthcare delivery systems.

OBJECTIVE: To evaluate whether a natural language processing (NLP) algorithm could be adapted to extract, with acceptable validity, markers of residential instability (ie, homelessness and housing insecurity) from electronic health records (EHRs) of 3 healthcare systems. MATERIALS AND METHODS: We included patients 18 years and older who received care at 1 of 3 healthcare systems from 2016 through 2020 and had at least 1 free-text note in the EHR during this period. We conducted the study independently; the NLP algorithm logic and method of validity assessment were identical across sites. The approach to the development of the gold standard for assessment of validity differed across sites. Using the EntityRuler module of spaCy 2.3 Python toolkit, we created a rule-based NLP system made up of expert-developed patterns indicating residential instability at the lead site and enriched the NLP system using insight gained from its application at the other 2 sites. We adapted the algorithm at each site then validated the algorithm using a split-sample approach. We assessed the performance of the algorithm by measures of positive predictive value (precision), sensitivity (recall), and specificity. RESULTS: The NLP algorithm performed with moderate precision (0.45, 0.73, and 1.0) at 3 sites. The sensitivity and specificity of the NLP algorithm varied across 3 sites (sensitivity: 0.68, 0.85, and 0.96; specificity: 0.69, 0.89, and 1.0). DISCUSSION: The performance of this NLP algorithm to identify residential instability in 3 different healthcare systems suggests the algorithm is generally valid and applicable in other healthcare systems with similar EHRs. CONCLUSION: The NLP approach developed in this project is adaptable and can be modified to extract types of social needs other than residential instability from EHRs across different healthcare systems.