Identifying Optimal Methods for Addressing Confounding Bias When Estimating the Effects of State-level Policies.

BACKGROUND: Policy evaluation studies that assess how state-level policies affect health-related outcomes are foundational to health and social policy research. The relative ability of newer analytic methods to address confounding, a key source of bias in observational studies, has not been closely examined. METHODS: We conducted a simulation study to examine how differing magnitudes of confounding affected the performance of 4 methods used for policy evaluations: (1) the two-way fixed effects difference-in-differences model; (2) a 1-period lagged autoregressive model; (3) augmented synthetic control method; and (4) the doubly robust difference-in-differences approach with multiple time periods from Callaway-Sant'Anna. We simulated our data to have staggered policy adoption and multiple confounding scenarios (i.e., varying the magnitude and nature of confounding relationships). RESULTS: Bias increased for each method: (1) as confounding magnitude increases; (2) when confounding is generated with respect to prior outcome trends (rather than levels), and (3) when confounding associations are nonlinear (rather than linear). The autoregressive model and augmented synthetic control method had notably lower root mean squared error than the two-way fixed effects and Callaway-Sant'Anna approaches for all scenarios; the exception is nonlinear confounding by prior trends, where Callaway-Sant'Anna excels. Coverage rates were unreasonably high for the augmented synthetic control method (e.g., 100%), reflecting large model-based standard errors and wide confidence intervals in practice. CONCLUSIONS: In our simulation study, no single method consistently outperformed the others, but a researcher's toolkit should include all methodologic options. Our simulations and associated R package can help researchers choose the most appropriate approach for their data.

The association between care integration and care quality.

OBJECTIVE: The study aims to analyze the relationship between care integration and care quality, and to examine if the relationship varies by patient risk. DATA SOURCES AND STUDY SETTING: The key independent variables used validated measures derived from a provider survey of functional (i.e., administrative and clinical systems) and social (i.e., patient integration, professional cooperation, professional coordination) integration. Survey responses represented data from a stratified sample of 59 practice sites from 17 health systems. Dependent variables included three quality measures constructed from patient-level Medicare data: colorectal cancer screening among patients at risk, patient-level 30-day readmission, and a practice-level Healthcare Effectiveness Data and Information Set (HEDIS) composite measure of publicly reported, individual measures of ambulatory clinical quality performance. DATA COLLECTION/EXTRACTION METHOD: We obtained quality- and beneficiary-level covariate data for the 41,966 Medicare beneficiaries served by the 59 practices in our survey sample. STUDY DESIGN: We estimated hierarchical linear models to examine the association between care integration and care quality and the moderating effect of patients' clinical risk score. We graphically visualized the moderating effects at ±1 standard deviation of our z-standardized independent and moderating variables and performed simple slope tests. PRINCIPAL FINDINGS: Our analyses uncovered a strong positive relationship between social integration, specifically patient integration, and the quality of care a patient receives (e.g., a 1-point increase in a practice's patient integration was associated with 0.31-point higher HEDIS composite score, p < 0.01). Further, we documented positive and significant associations between aspects of social and functional integration on quality of care based on patient risk. CONCLUSIONS: The findings suggest social integration matters for improving the quality of care and that the relationship of integration to quality is not uniform for all patients. Policymakers and practitioners considering structural integrations of health systems should direct attention beyond structure to consider the potential for social integration to impact outcomes and how that might be achieved.

Just in Reach Pay for Success: Impact Evaluation and Cost Analysis of a Permanent Supportive Housing Program.

Discharging individuals from jails and prisons who may be poorly equipped for independent living-such as those with a history of chronic health conditions, including serious mental illness-is likely to reinforce a pattern of homelessness and recidivism. Permanent supportive housing (PSH)-which combines a long-term housing subsidy with supportive services-has been proposed as a mechanism to intervene directly on this relationship between housing and health. In Los Angeles County, jail has become a default housing and services provider to unhoused individuals with serious mental health issues. In 2017, the county initiated the Just in Reach Pay for Success (JIR PFS) project, which provided PSH as an alternative to jail for individuals with a history of homelessness and chronic behavioral or physical health conditions. The authors of this study assessed whether the project led to changes in use of several county services, including justice, health, and homeless services. The authors examined changes in county service use, before and after incarceration, by JIR PFS participants and a comparison control group and found that use of jail services was significantly reduced after JIR PFS PSH placement, while the use of mental health and other services increased. The researchers assess that the net cost of the program is highly uncertain but that it may pay for itself in terms of reducing the use of other county services and therefore provide a cost-neutral means of addressing homelessness among individuals with chronic health conditions involved with the justice system in Los Angeles County.

Implementation and 12-Month Health Service Utilization and Cost Outcomes from a Managed Care Health Plan's Permanent Supportive Housing Program.

Homelessness, which refers to the lack of a fixed, regular, and adequate nighttime residence, is a pervasive public health issue. This article presents results from an implementation and outcome study of an ongoing permanent supportive housing (PSH) program-including service utilization and associated costs review-operated by a large not-for-profit Medicaid and Medicare managed care plan serving more than 1 million members in the Inland Empire area of Southern California. This PSH program combines a long-term housing subsidy with intensive case management services for adult plan members experiencing homelessness who have one or more chronic physical or behavioral health conditions and represent high utilizers of inpatient health care. The aim of this research was to determine whether programmatic costs incurred by the health plan supporting the PSH program were partially or fully offset by decreased costs attributable to health care utilization within the health system. The evaluation used a quasi-experimental research design with an observational control group. The authors differentiated the program's effect during the transitional period-that is, after program enrollment and prior to housing placement-from its effect during the period after members were housed. In addition, the authors present participant flow through the key program milestones (e.g., referral, enrollment, housing placement, program exit) and describe health care utilization and associated costs for members who exited the program. Finally, they report the PSH programmatic expenditures relative to the changes in health care costs to provide an overall picture of the intervention's benefits and costs to the health plan.

MACRA Palliative Care Quality Measure Development-Testing Summary Report: Measure Name: Receiving Desired Help for Pain.

Palliative care has expanded rapidly in the past 20 years, especially in the ambulatory (office) setting, and there is growing consensus regarding the need to systematically measure and incentivize high-quality care. The Centers for Medicare & Medicaid Services entered a cooperative agreement with the American Academy of Hospice and Palliative Medicine (AAHPM) as part of the Medicare Access and CHIP Reauthorization Act of 2015 to develop two patient-reported measures of ambulatory palliative care experience: Feeling Heard and Understood and Receiving Desired Help for Pain. Under contract to AAHPM, RAND Health Care researchers developed and tested both measures over a three-year project period. Researcher efforts included identifying, developing, testing, and validating appropriate patient-reported data elements for each measure; developing and fielding a survey instrument to collect necessary data in a national beta field test with 44 ambulatory palliative care programs; and collecting and analyzing data about measure reliability and validity to establish measure performance and final specifications. Further, the authors elicited provider and program perspectives on the use and value of the performance measures and their implementation and elicited the perspectives of patients from racial and ethnic minorities to understand their experience of ambulatory palliative care and optimal approaches to measurement. In this study, the authors present results from their test of the Receiving Desired Help for Pain performance measure, which they demonstrate to be a reliable and valid measure that is ready for use in quality improvement and quality payment programs.

MACRA Palliative Care Quality Measure Development-Testing Summary Report: Measure Name: Feeling Heard and Understood.

Palliative care has expanded rapidly in the past 20 years, especially in the ambulatory (office) setting, and there is growing consensus regarding the need to systematically measure and incentivize high-quality care. The Centers for Medicare & Medicaid Services entered a cooperative agreement with the American Academy of Hospice and Palliative Medicine (AAHPM) as part of the Medicare Access and CHIP Reauthorization Act of 2015 to develop two patient-reported measures of ambulatory palliative care experience: Feeling Heard and Understood and Receiving Desired Help for Pain. Under contract to AAHPM, RAND Health Care researchers developed and tested both measures over a three-year project period. Researcher efforts included identifying, developing, testing, and validating appropriate patient-reported data elements for each measure; developing and fielding a survey instrument to collect necessary data in a national beta field test with 44 ambulatory palliative care programs; and collecting and analyzing data about measure reliability and validity to establish measure performance and final specifications. Further, the authors elicited provider and program perspectives on the use and value of the performance measures and their implementation and elicited the perspectives of patients from racial and ethnic minorities to understand their experience of ambulatory palliative care and optimal approaches to measurement. In this study, the authors present results from their test of the Feeling Heard and Understood performance measure, which they demonstrate to be a reliable and valid measure that is ready for use in quality improvement and quality payment programs.

Experiences with Telehealth for Outpatient Palliative Care: Findings from a Mixed-Methods Study of Patients and Providers across the United States.

Background: The COVID-19 pandemic introduced a rapid adoption and scale-up of telehealth for palliative care services in the United Sates. Objectives: To examine and compare in-person versus telehealth experience among outpatient palliative care programs and patients. Design: Mixed-methods study (1) comparing patient experience survey data received between September 2020 and February 2021 from patients who received only in-person care versus those who received only telehealth and (2) qualitative interviews with outpatient palliative care providers. Data for this study were collected as part of a larger effort to develop quality measures for outpatient palliative care in the United States. Setting/Subjects: Outpatient palliative care patients and programs. Measurements: We measured patients' experiences of "feeling heard and understood" by their palliative care provider and team and their overall rating of their provider and team. We also conducted in-depth semistructured interviews with 47 palliative care providers across 25 outpatient palliative care programs. Results: Of 1753 patient experience surveys, 26% reflected telehealth only versus 74% in-person only. Patients in both groups reported highly positive experiences; there were no differences in "feeling heard and understood" or the overall ratings of the provider and team between the telehealth-only and in-person-only groups. Palliative care program leaders described the benefits and challenges of telehealth, including increased efficiency, the ability to incorporate family members, and challenges conducting a physical examination. Conclusion: Data from this study provide preliminary evidence of overall positive experiences of telehealth for outpatient palliative care among patients and providers; future research is needed to examine the sustainability of telehealth for palliative care.

Distinguishing High-Performing Health Systems Using a Composite of Publicly Reported Measures of Ambulatory Care.

BACKGROUND: Payers and policymakers are rewarding high-performing health care providers on the basis of summaries of overall quality performance, and the methods they use for measuring performance are increasingly important. OBJECTIVE: To develop and evaluate a measure that ranks health care systems by ambulatory care quality. DESIGN: Systems were ranked using a composite model that summarizes individual measures of quality, accounts for their correlation, and does not require health care systems to report every measure. The composite measure's suitability was evaluated by examining whether it captured the quality indicated by component measures (validity), whether differences in rank between health care systems were larger than statistical noise (reliability), and whether year-to-year changes in rank were small (stability). SETTING: California and Minnesota, 2014 to 2016. PARTICIPANTS: 55 health care systems. MEASUREMENTS: Publicly reported measures of ambulatory care quality. RESULTS: The composite measure was valid in that it was broadly representative of the component measures and was not dominated by any single measure. The measure was reliable because the ranks for 93% of California systems and 80% of Minnesota systems were unlikely to be more than 2 places lower or higher. The measure was stable because fewer than half of systems changed ranks by more than 2 ranks from year to year. LIMITATION: The analysis is limited to available measures of ambulatory care quality and includes only 2 states. CONCLUSION: This composite measure uses publicly reported data to produce valid, reliable, and stable ranks of ambulatory care quality for health care systems in Minnesota and California, and this approach could be used in other applications. PRIMARY FUNDING SOURCE: Agency for Healthcare Research and Quality.