Social risk adjustment in the hospital readmission reduction program: Pitfalls of peer grouping, measurement challenges, and potential solutions.

OBJECTIVE: To examine the limitations of peer grouping and associated challenges measuring social risk in Medicare's Hospital Readmission Reduction Program (HRRP). Under peer grouping, hospitals are divided into quintiles based on the proportion of a hospital's Medicare inpatients with Medicaid ("dual share"). This approach was implemented to address concerns that the HRRP unfairly penalized hospitals that disproportionately serve disadvantaged patients. DATA: Public data on hospitals in the HRRP. DESIGN: We examined the relationship between hospital dual share and readmission rates within peer groups; changes in hospitals' peer group assignments, readmission rates, and penalties; and the relationship between state Medicaid eligibility rules and peer groups. DATA COLLECTION: Public data on hospital characteristics and readmission rates for 3119 hospitals from 2019 to 2020. PRINCIPAL FINDINGS: The proportion of dual inpatients among hospitals of the same peer group varied by as much as 69 percentage points (ppt). Within peer groups, a one ppt increase in dual share was associated with a 0.01 ppt increase in the difference from the median readmission rate (p < 0.001). From 2019 to 2020, 8.8% of hospitals switched peer groups. Compared to hospitals that did not switch, those moving to a lower peer group had a higher mean penalty in 2020 (0.096 ppt; p = 0.006); those moving to a higher group had a lower mean penalty (-0.06 ppt; p = 0.079). However, changes in penalties did not correspond to changes in readmission rates. Hospitals in states with higher Medicaid income eligibility limits were more likely to be in higher peer groups. CONCLUSIONS: Peer grouping is limited in the extent to which it accounts for differences in hospitals' patient populations, and it may not fully insulate hospitals from penalties linked to changes in patient mix. These problems arise from the construction of peer groups and the measure of social risk used to define them.

Neighborhood disadvantage, health status, and health care utilization after blood or marrow transplant: BMTSS report.

Living in a disadvantaged neighborhood is associated with poor health outcomes. Blood or Marrow Transplant (BMT) survivors remain at risk of chronic health conditions requiring anticipatory management. We hypothesized that among BMT survivors, neighborhood disadvantage was associated with poor self-reported routine health care utilization and health. We leveraged data from BMTSS - a retrospective cohort study examining long-term outcomes among individuals surviving ≥2 y following BMT at three institutions between 1974 and 2014. Participants in this analysis completed the BMTSS survey (sociodemographics; chronic health conditions; time since routine check-up; self-reported health). The Area Deprivation Index (ADI) represented neighborhood disadvantage; this composite indicator of 17 census measures is a percentile rank (0 = least deprived to 100 = most deprived). Multivariable ordered logit regression adjusted for clinical factors and individual-level sociodemographics, modeling associations between ADI, time since routine check-up, and self-reported health. Among 2,857 survivors, median ADI was 24 (interquartile range: 10-46). Adjusting for self-reported individual-level socioeconomic indicators and chronic health conditions, patients in more disadvantaged neighborhoods had higher odds of reporting longer intervals since routine check-up (ORADI_continuous = 1.007, P < .001) and poorer health status (controlling for time since check-up; ORADI_continuous = 1.005, P = .003). Compared with patients living in the least disadvantaged neighborhood (ADI = 1), patients in the most disadvantaged neighborhood (ADI = 100), had twice the odds (ORADI = 1.007^99 = 2.06) of reporting no routine visits and 1.65-times the odds of reporting poor health (ORADI = 1.005^99 = 1.65). In BMT survivors, access to health care and health status are associated with area disadvantage. These findings may inform strategies to address long-term care coordination and retention for vulnerable survivors.

Changes in Health Care Access by Race, Income, and Medicaid Expansion During the COVID-19 Pandemic.

BACKGROUND: The intersecting crises of the COVID-19 pandemic, job losses, and concomitant loss of employer-sponsored health insurance may have disproportionately affected health care access within minorized and lower-socioeconomic status communities. OBJECTIVE: To describe changes in access to care during the COVID-19 pandemic, stratified by race/ethnicity, household income, and state Medicaid expansion status. RESEARCH DESIGN: We used interrupted time series and difference-in-differences regression models, controlling for respondent characteristics and preexisting trends. SUBJECTS: Data were extracted for all adults aged 18-64 surveyed in the 2015-2020 Behavioral Risk Factor Surveillance System (N=1,731,699) from all 50 states and the District of Columbia. MEASURES: Our outcomes included indicators for whether respondents had any health insurance coverage or avoided seeking care because of cost within the prior year. The primary exposure was the onset of the COVID-19 pandemic in the United States in March 2020. RESULTS: The pandemic was associated with a 1.2 percentage point (pp) decline in uninsurance for Medicaid expansion states (95% CI, -1.8, -0.6); these reductions were concentrated among respondents who were Black, multiracial, or low income. The rates of uninsurance were generally stable in nonexpansion states. The rates of avoided care because of cost fell by 3.5 pp in Medicaid expansion states (95% CI, -3.9, -3.1), and by 3.6 pp (95% CI, 4.3-2.9) in nonexpansion states. These declines were concentrated among respondents who were Hispanic, Other Race, or low income. CONCLUSIONS: Our findings reinforce the value of Medicaid expansion as one tool to improve access to health insurance and care for marginalized and vulnerable populations.

Survival in the Real World: A National Analysis of Patients Treated for Early-Stage Breast Cancer.

PURPOSE: Many patient population groups are not proportionally represented in clinical trials, including patients of color, at age extremes, or with comorbidities. It is therefore unclear how treatment outcomes may differ for these patients compared with those who are well-represented in trials. METHODS: This retrospective cohort study included women diagnosed with stage I-III breast cancer between 2005 and 2015 in the national CancerLinQ Discovery electronic medical record-based data set. Patients with comorbidities or concurrent cancer were considered unrepresented in clinical trials. Non-White patients and/or those age < 45 or ≥ 70 years were considered under-represented. Patients who were White, age 45-69 years, and without comorbidities were considered well-represented. Cox proportional hazards models were used to evaluate 5-year mortality by representation group and patient characteristics, adjusting for cancer stage, subtype, chemotherapy, and diagnosis year. RESULTS: Of 11,770 included patients, 48% were considered well-represented in trials, 45% under-represented, and 7% unrepresented. Compared with well-represented patients, unrepresented patients had almost three times the hazard of 5-year mortality (adjusted hazard ratio [aHR], 2.71; 95% CI, 2.08 to 3.52). There were no significant differences in the hazard of 5-year mortality for under-represented patients compared with well-represented patients (aHR, 1.19; 95% CI, 0.98 to 1.45). However, among under-represented patients, those age < 45 years had a lower hazard of 5-year mortality (aHR, 0.63; 95% CI, 0.48 to 0.84) and those age ≥ 70 years had a higher hazard of 5-year mortality (aHR, 2.21; 95% CI, 1.76 to 2.77) compared with those age 45-69 years. CONCLUSION: More than half of the patients were under-represented or unrepresented in clinical trials, because of age, comorbidity, or race. Some of these groups experienced poorer survival compared with those well-represented in trials. Trialists should ensure that study participants reflect the disease population to support evidence-based decision making for all individuals with cancer.

Variation in Restrictive Abortion Policies and Adverse Birth Outcomes in the United States from 2005 to 2015.

BACKGROUND: Since 2011, U.S. states have enacted more than 400 policies restricting abortion access. As structural determinants, abortion policies have the potential to influence maternal and child health access, outcomes, and equity through multiple mechanisms. Limited research has examined their implications for birth outcomes. METHODS: We created a state-level abortion restrictiveness index composed of 18 restrictive abortion policies and evaluated the association between this index and individual-level probabilities of preterm birth (PTB) and low birthweight (LBW) within the United States and by Census Region, using data from the 2005-2015 National Center for Health Statistics Period Linked Live Birth-Infant Death Files. We used logistic multivariable regression modeling, adjusting for individual- and state-level factors and state and year fixed effects. RESULTS: Among 2,500,000 live births, 269,253 (12.0%) were PTBs and 182,960 (8.1%) were LBW. On average from 2005 to 2015, states had approximately seven restrictive abortion policies enacted, with more policies enacted in the Midwest and South. Nationally, relationships between state restrictiveness indices and adverse birth outcomes were insignificant. Regional analyses revealed that a 1 standard deviation increase in a state's restrictiveness index was associated with a 2% increase in PTB in the Midwest (marginal effect [ME], 0.25; 95% confidence interval [CI], 0.04-0.45; p < .01), a 15% increase in LBW in the Northeast (ME, 1.24; 95% CI, 0.12-2.35; p < .05), and a 2% increase in LBW in the West (ME, 0.12; 95% CI, 0.01-0.25; p < .05). CONCLUSION: Variation in restrictive abortion policy environments may have downstream implications for birth outcomes, and increases in abortion restrictions were associated with adverse birth outcomes in three out of four Census Regions.

Racial/ethnic and educational inequities in restrictive abortion policy variation and adverse birth outcomes in the United States.

BACKGROUND: To examine racial/ethnic and educational inequities in the relationship between state-level restrictive abortion policies and adverse birth outcomes from 2005 to 2015 in the United States. METHODS: Using a state-level abortion restrictiveness index comprised of 18 restrictive abortion policies, we conducted a retrospective longitudinal analysis examining whether race/ethnicity and education level moderated the relationship between the restrictiveness index and individual-level probabilities of preterm birth (PTB) and low birthweight (LBW). Data were obtained from the 2005-2015 National Center for Health Statistics Period Linked Live Birth-Infant Death Files and analyzed with linear probability models adjusted for individual- and state-level characteristics and state and year fixed-effects. RESULTS: Among 2,250,000 live births, 269,253 (12.0%) were PTBs and 182,960 (8.1%) were LBW. On average, states had approximately seven restrictive abortion policies enacted from 2005 to 2015. Black individuals experienced increased probability of PTB with additional exposure to restrictive abortion policies compared to non-Black individuals. Similarly, those with less than a college degree experienced increased probability of LBW with additional exposure to restrictive abortion policies compared to college graduates. For all analyses, inequities worsened as state environments grew increasingly restrictive. CONCLUSION: Findings demonstrate that Black individuals at all educational levels and those with fewer years of education disproportionately experienced adverse birth outcomes associated with restrictive abortion policies. Restrictive abortion policies may compound existing racial/ethnic, socioeconomic, and intersecting racial/ethnic and socioeconomic perinatal and infant health inequities.

Clinical trial representativeness and treatment intensity in a real-world sample of women with early stage breast cancer.

PURPOSE: The extent to which evidence-based treatments are applied to populations not well represented in early stage breast cancer (EBC) trials remains unknown. This study evaluated treatment intensity for patients traditionally well represented, underrepresented, and unrepresented in clinical trials. METHODS: This retrospective cohort study used real-world data to evaluate the intensity (high or low) of EBC chemotherapy by patient characteristics (age, race and ethnicity, presence of comorbidity) denoting clinical trial representation status (well represented, underrepresented, unrepresented) for patients diagnosed from 2011 to 2020. Odds ratios (OR) from a logistic regression model was used to evaluate the association between receipt of high-intensity chemotherapy and clinical trial representation status characteristics adjusting for cancer stage and subtype. RESULTS: Of 970 patients with EBC, 41% were characterized as well represented, 45% as underrepresented, and 13% as unrepresented in clinical trials. In adjusted models, patients aged ≥ 70 versus 45-69 had lower odds of receiving a high-intensity treatment (OR 0.40, 95% CI 0.26-0.60), while those aged < 45 versus 45-69 had higher odds of receiving high-intensity treatment (OR 1.82, 95% CI 1.10-3.01). In predicted estimates, the proportion of patients receiving a high-intensity treatment was 87% for patients aged < 45, 79% for patients aged 45-69, and 60% for patients aged ≥ 70. CONCLUSION: 59% of the EBC population is not well represented in clinical trials. Age was associated with differential treatment intensity. Widening clinical trial eligibility criteria should be considered to better understand survival outcomes, toxicity effects, and ultimately make evidence-based treatment decisions using a more diverse sample.

Quantifying treatment preferences and their association with financial toxicity in women with breast cancer.

BACKGROUND: The objective of the current study was to understand treatment preferences and their association with financial toxicity in Patient Advocate Foundation clients with breast cancer. METHODS: This choice-based conjoint analysis used data from a nationwide sample of women with breast cancer who received assistance from the Patient Advocate Foundation. Choice sets created from 13 attributes of 3 levels each elicited patient preferences and trade-offs. Latent class analysis segmented respondents into distinct preference archetypes. The Comprehensive Score for Financial Toxicity (COST) tool captured financial toxicity. Adjusted generalized linear models estimated COST score differences by preference archetype. RESULTS: Of 220 respondents (for a response rate of 10%), the median age was 58 years (interquartile range, 49-66 years); 28% of respondents were Black, indigenous, or people of color; and approximately 60% had household incomes <$40,000. The majority of respondents were diagnosed with early-stage cancer (91%), 38% had recurrent disease, and 61% were receiving treatment. Treatment choice was most affected by preferences related to affordability and impact on activities of daily living. Two distinct treatment preference archetypes emerged. The "cost-prioritizing group" (75% of respondents) was most concerned about affordability, impact on activities of daily living, and burdening care partners. The "functional independence-prioritizing group" (25% of respondents) was most concerned about their ability to work, physical side effects, and interference with life events. COST scores were found to be similar between the archetypes in adjusted models (cost-prioritizing group COST score, 12 [95% confidence interval, 9-14]; and functional independence-prioritizing COST score, 11 [95% confidence interval, 9-13]). CONCLUSIONS: Patients with breast cancer prioritized affordability or maintaining functional independence when making treatment decisions. Because of this variability, preference evaluation during treatment decision making could optimize patients' treatment experiences.

Treatment Decision Making and Financial Toxicity in Women With Metastatic Breast Cancer.

INTRODUCTION: Oncologists have increasingly been proponents of shared decision making (SDM) to enhance patient outcomes and reduce unnecessary health care spending. However, its effect on patient out-of-pocket costs is unknown. This study investigated the relationship between patient preferences for SDM and financial toxicity (FT) in patients with metastatic breast cancer (MBC). PATIENTS AND METHODS: This cross-sectional study utilized surveys of women aged ≥ 18 with MBC who received care at two academic hospitals in Alabama from 2017 to 2019. Patients self-reported their SDM preference (Control Preferences Scale) and FT (Comprehensive Score for Financial Toxicity [COST] tool; 11-item scale, with lower scores indicating worse FT). Effect sizes were calculated using the proportion of variance explained (R2) or Cramer's V. Differences in FT by SDM preference were estimated using mixed models clustered by site and treating medical oncologist. RESULTS: In 95 women with MBC, 44% preferred SDM, 29% preferred provider-driven decision making, and 27% preferred patient-driven decision making. Patients preferring SDM were more often college educated (53% vs. 39%; V = 0.12) with an income greater than $40,000/y (55% vs. 43%; V = 0.18). Overall median COST was 22 (interquartile range, 16-29). After adjusting for patient demographic and clinical characteristics, patients preferring patient-driven decision making trended toward worse FT (COST 17: 95% confidence interval, 12-22) compared to those preferring SDM (COST 19: 95% confidence interval, 15-23) and those preferring provider-driven decision making (COST 22: 95% confidence interval, 17-27). CONCLUSION: Patients preferring more patient-driven decision making reported worse FT, although differences did not reach statistical significance. Further research is needed to understand this relationship.

Oncologic Services Through Project Access and Other Safety Net Care Coordination Programs.

PURPOSE: Little is known about the provision of oncologic services by Project Access safety net care coordination programs. MATERIALS AND METHODS: Information on safety net care coordination program locations, health services, and patient eligibility was obtained via program Web sites and calls. For programs not offering oncologic care, program directors were interviewed to identify oncologic care barriers. RESULTS: Web sites of 29 safety net care coordination programs in 22 states were identified; 62% (n = 18) offered oncologic services. Programs were in 65% (n = 11) of states that did not expand Medicaid. Of those offering oncologic services, 83% (n = 15) offered free chemotherapy, and 93% (n = 27) of all programs offered oncologic imaging. Program director interviews revealed costs, longitudinal care, and multiple-physician buy-in as barriers limiting oncologic care. CONCLUSION: Third-party care coordination centers provide a novel and potentially unrecognized approach to increasing oncology service access. Further research should identify strategies to overcome the relative lack of oncologic care offerings.

Health Insurance Literacy and Financial Hardship in Women Living With Metastatic Breast Cancer.

PURPOSE: In patients with metastatic breast cancer (MBC), low health insurance literacy may be associated with adverse material conditions, psychological response, and coping behaviors because of financial hardship (FH). This study explored the relationship between health insurance literacy and FH in women with MBC. METHODS: This cross-sectional study used data collected from 84 women receiving MBC treatment at 2 southeastern cancer centers. Low health insurance literacy was defined as not knowing premium or deductible costs. FH was defined by lifestyle changes as a result of medical expenses, financial toxicity, and medical care modifications attributable to cost. Mean differences were calculated using Cramer's V. Associations between health insurance literacy and FH were estimated with adjusted linear models. RESULTS: Half of the surveyed patients had low health insurance literacy, 26% were underinsured, 45% had private insurance, 39% had Medicare, and 15% had Medicaid. Patients with low health insurance literacy more often reported borrowing money (19% v 4%; V = 0.35); an inability to pay for basic necessities like food, heat, or rent (10% v 4%; V = 0.18); and skipping a procedure (8% v 1%; V = 0.21), medical test (7% v 0%; V = 0.30), or treatment (4% v 0%; V = 0.20) compared with patients with high health insurance literacy. Median Comprehensive Score for Financial Toxicity was 23 (interquartile range, 17-29). In adjusted models, health insurance literacy was not associated with financial toxicity. CONCLUSION: Low health insurance literacy was common in women receiving MBC treatment. Additional research to increase health insurance literacy could lessen undesirable material FH and unnecessary behavioral FH associated with cancer-related care.

Differential Impact of Hospital and Community Factors on Medicare Readmission Penalties.

OBJECTIVE: To identify hospital/county characteristics and sources of regional heterogeneity associated with readmission penalties. DATA SOURCES/STUDY SETTING: Acute care hospitals under the Hospital Readmissions Reduction Program from fiscal years 2013 to 2018 were linked to data from the Annual Hospital Association, Centers for Medicare and Medicaid Services, Medicare claims, Hospital Compare, Nursing Home Compare, Area Resource File, Health Inequity Project, and Long-term Care Focus. The final sample contained 3,156 hospitals in 1,504 counties. DATA COLLECTION/EXTRACTION METHODS: Data sources were combined using Medicare hospital identifiers or Federal Information Processing Standard codes. STUDY DESIGN: A two-level hierarchical model with correlated random effects, also known as the Mundlak correction, was employed with hospitals nested within counties. PRINCIPAL FINDINGS: Over a third of the variation in readmission penalties was attributed to the county level. Patient sociodemographics and the surrounding access to and quality of care were significantly associated with penalties. Hospital measures of Medicare volume, percentage dual-eligible and Black patients, and patient experience were correlated with unobserved area-level factors that also impact penalties. CONCLUSIONS: As the readmission risk adjustment does not include any community-level characteristics or geographic controls, the resulting endogeneity bias has the potential to disparately penalize certain hospitals.

Health Expenditure Concentration and Characteristics of High-Cost Enrollees in CHIP.

Devising effective cost-containment strategies in public insurance programs requires understanding the distribution of health care spending and characteristics of high-cost enrollees. The aim was to characterize high-cost enrollees in a state's public insurance program and determine whether expenditure inequality changes over time, or with changes in cost-sharing policies or program eligibility. We use 1999-2011 claims and enrollment data from the Alabama Children's Health Insurance Program, ALL Kids. All children enrolled in ALL Kids were included in our study, including multiple years of enrollment (N = 1,031,600 enrollee-months). We examine the distribution of costs over time, whether this distribution changes after increases in cost sharing and expanded eligibility, patient characteristics that predict high-cost status, and examine health services used by high-cost children to identify what is preventable. The top 10% (1%) of enrollees account for about 65.5% (24.7%) of total program costs. Inpatient and outpatient costs are the largest components of costs incurred by high-cost utilizers. Non-urgent emergency department costs are a relatively small portion. Average expenditure increases over time, particularly after expanded eligibility, and the share of costs incurred by the top 10% and 1% increases slightly. Multivariable logistic regression results indicate that infants and older teens, Caucasian children, and those with chronic conditions are more likely to be high-cost utilizers. Increased cost sharing does not reduce cost concentration or average expenditure among high-cost utilizers. These findings suggest that identifying and targeting potentially preventable costs among high-cost utilizers are called for to help reduce costs in public insurance programs.

Infections present on admission compared with hospital-acquired infections in acute ischemic stroke patients.

BACKGROUND: To date, few studies have assessed the influence of infections present on admission (POA) compared with hospital-acquired infections (HAIs) on neurologic deterioration (ND) and other outcome measures in acute ischemic stroke (AIS). METHODS: Patients admitted with AIS to our stroke center (July 2010 to December 2010) were retrospectively assessed. The following infections were assessed: urinary tract infection, pneumonia, and bacteremia. Additional chart review was performed to determine whether the infection was POA or HAI. We assessed the relationship between infections in ischemic stroke patients and several outcome measures including ND and poor functional outcome. A mediation analysis was performed to assess the indirect effects of HAI, ND, and poor functional outcome. RESULTS: Of the 334 patients included in this study, 77 had any type of infection (23 POA). After adjusting for age, National Institutes of Health Stroke Scale at baseline, glucose on admission, and intravenous tissue plasminogen activator, HAI remained a significant predictor of ND (odds ratio [OR]=8.8, 95% confidence interval [CI]: 4.2-18.7, P<.0001) and poor functional outcome (OR=41.7, 95% CI: 5.2-337.9, P=.005), whereas infections POA were no longer associated with ND or poor functional outcome. In an adjusted analysis, we found that 57% of the effect from HAI infections on poor functional outcome is because of mediation through ND (P<.0001). CONCLUSIONS: Our data suggests that HAI in AIS patients increases the odds of experiencing ND and subsequently increases the odds of being discharged with significant disability. This mediated effect suggests a preventable cause of ND that can thereby decrease the odds of poor functional outcomes after an AIS.

Variation in surgical site infection monitoring and reporting by state.

OBJECTIVE: Surgical site infections (SSIs) are common, costly, and often preventable. There are no national requirements for measuring or reporting hospital SSI rates and state-level monitoring occurs with little coordination between states. We designed a study to describe the current status of SSI reporting in the United States. METHODS: We reviewed SSI monitoring and reporting legislation in all 50 states in September 2010. Data collected included whether SSI monitoring and reporting legislation exists, if public reporting is required, how the data are accessible, and for which procedures SSI data are reported. RESULTS: Twenty-one (42%) states have legislation for SSI monitoring and reporting. All 21 of these states require public release of findings. Of the states with legislation, eight (38%) currently have SSI data available publicly. A range of two to seven procedures were reported for SSI measurement by individual states. Eighteen (86%) states use state agency websites to make their data publicly available. CONCLUSION: There is wide variation in state monitoring and reporting of SSI rates. Standardized reporting may be needed so that consumers can make informed health choices based on quality metrics.

Reducing medical errors and adverse events.

Medical errors account for ∼98,000 deaths per year in the United States. They increase disability and costs and decrease confidence in the health care system. We review several important types of medical errors and adverse events. We discuss medication errors, healthcare-acquired infections, falls, handoff errors, diagnostic errors, and surgical errors. We describe the impact of these errors, review causes and contributing factors, and provide an overview of strategies to reduce these events. We also discuss teamwork/safety culture, an important aspect in reducing medical errors.

Variation in public reporting of central line-associated bloodstream infections by state.

Central line-associated bloodstream infections (CLABSIs) are common, costly, and largely preventable. Consumers who want high-quality care should have access to CLABSI rates to make health care decisions. The authors searched state health department Web sites for publicly available CLABSI data. Fourteen states, all with mandatory CLABSI monitoring laws, had publicly available data. The authors identified significant variation in the presentation of infection rates, methods of risk adjustment, locations and care settings reported, time span of data collection, and time lag to reporting. The wide variation in availability and content of information illustrates the need for standardized CLABSI monitoring and reporting mechanisms.

Commentary: Reducing diagnostic errors: another role for checklists?

Diagnostic errors are a widespread problem, although the true magnitude is unknown because they cannot currently be measured validly. These errors have received relatively little attention despite alarming estimates of associated harm and death. One promising intervention to reduce preventable harm is the checklist. This intervention has proven successful in aviation, in which situations are linear and deterministic (one alarm goes off and a checklist guides the flight crew to evaluate the cause). In health care, problems are multifactorial and complex. A checklist has been used to reduce central-line-associated bloodstream infections in intensive care units. Nevertheless, this checklist was incorporated in a culture-based safety program that engaged and changed behaviors and used robust measurement of infections to evaluate progress. In this issue, Ely and colleagues describe how three checklists could reduce the cognitive biases and mental shortcuts that underlie diagnostic errors, but point out that these tools still need to be tested. To be effective, they must reduce diagnostic errors (efficacy) and be routinely used in practice (effectiveness). Such tools must intuitively support how the human brain works, and under time pressures, clinicians rarely think in conditional probabilities when making decisions. To move forward, it is necessary to accurately measure diagnostic errors (which could come from mapping out the diagnostic process as the medication process has done and measuring errors at each step) and pilot test interventions such as these checklists to determine whether they work.