Features of health care interventions associated with reduced services and spending.

OBJECTIVES: This study examines 14 independent and diverse health care interventions funded under the second round of Health Care Innovation Awards by CMS to determine if any organizational, model, or implementation features were strongly associated with the programs' estimated impacts on total expenditures, hospitalizations, or emergency department visits. STUDY DESIGN: We estimated program impacts using awardee-specific difference-in-differences models based on Medicare and Medicaid enrollment and claims data for treatment and matched comparison groups from 2012 to 2018. METHODS: We used 2 analytic approaches to identify program features associated with favorable impacts. The first method identified program characteristics that were common among programs that had estimated reductions in costs and service use and uncommon among those that did not. The second approach compared median impacts among awardees with a given distinguishing feature with median impacts among awardees that lacked the characteristic. RESULTS: Of the 23 program features examined, 7 were associated with favorable estimated impacts: 3 intervention components (behavioral health, telehealth, and health information technology) and 4 program design and organizational characteristics (having prior experience implementing similar programs, targeting patients with substantial nonmedical needs in addition to medical problems, being focused on individual patient care rather than transforming provider practice, and using nonclinical staff as frontline providers of the intervention). CONCLUSIONS: Innovative health care service delivery models with 2 or more of these 7 identified features were more likely than programs without them to reduce Medicare and Medicaid beneficiaries' needs for costly health care services.

Making Evidence Actionable: Interactive Dashboards, Bayes, and Health Care Innovation.

The results of many large-scale federal or multi-site evaluations are typically compiled into long reports which end up sitting on policymaker's shelves. Moreover, the information policymakers need from these reports is often buried in the report, may not be remembered, understood, or readily accessible to the policymaker when it is needed. This is not a new challenge for evaluators, and advances in statistical methodology, while they have created greater opportunities for insight, may compound the challenge by creating multiple lenses through which evidence can be viewed. The descriptive evidence from traditional frequentist models, while familiar, are frequently misunderstood, while newer Bayesian methods provide evidence which is intuitive, but less familiar. These methods are complementary but presenting both increases the amount of evidence stakeholders and policymakers may find useful. In response to these challenges, we developed an interactive dashboard that synthesizes quantitative and qualitative data and allows users to access the evidence they want, when they want it, allowing each user a customized, and customizable view into the data collected for one large-scale federal evaluation. This offers the opportunity for policymakers to select the specifics that are most relevant to them at any moment, and also apply their own risk tolerance to the probabilities of various outcomes.

Restrictive Transfusion Strategy and Clinical Decision Support Practices for Reducing RBC Transfusion Overuse.

OBJECTIVES: Assess support for the effectiveness of two separate practices, restrictive transfusion strategy and computerized physician order entry/clinical decision support (CPOE/CDS) tools, in decreasing RBC transfusions in adult surgical and nonsurgical patients. METHODS: Following the Centers for Disease Control and Prevention Laboratory Medicine Best Practice (LMBP) Systematic Review (A-6) method, studies were assessed for quality and evidence of effectiveness in reducing the percentage of patients transfused and/or units of blood transfused. RESULTS: Twenty-five studies on restrictive transfusion practice and seven studies on CPOE/CDS practice met LMBP inclusion criteria. The overall strength of the body of evidence of effectiveness for restrictive transfusion strategy and CPOE/CDS was rated as high. CONCLUSIONS: Based on these procedures, adherence to an institutional restrictive transfusion strategy and use of CPOE/CDS tools for hemoglobin alerts or reminders of the institution's restrictive transfusion policies are effective in reducing RBC transfusion overuse.

Reducing red blood cell transfusion in orthopedic and cardiac surgeries with Antifibrinolytics: A laboratory medicine best practice systematic review and meta-analysis.

OBJECTIVES: To evaluate the effectiveness of antifibrinolytics tranexamic acid (TA), ε-aminocaproic acid (EACA), and aprotinin to decrease overuse of red blood cell transfusions in adult surgical and non-surgical patients. METHODS: This review followed the Centers for Disease Control and Prevention (CDC) Laboratory Medicine Best Practice (LMBP™) Systematic Review (A-6) method. Eligible studies were assessed for evidence of effectiveness of TA or EACA in reducing the number of patients transfused or the number of whole blood transfusions. RESULTS: Seventy-two articles met LMBP™ inclusion criteria. Fifty-six studies assessed Topical, Intra-articular Injection, or Intravenous TA, 4 studied EACA, and 12 studied the effectiveness of aprotinin. The overall strength of the body of evidence of effectiveness for each of these practices was rated as high. CONCLUSION: LMBP™ recommends the use of topical, intra-articular injection, or intravenous tranexamic acid and the use of ε-aminocaproic acid for reducing overuse of red blood cell transfusion.

Anemia Management and Audit Feedback Practices for Reducing Overuse of RBC Transfusion: A Laboratory Medicine Best Practice Systematic Review and Meta-Analysis.

Objectives: To evaluate the effectiveness of anemia management and audit with feedback practices in reducing overuse of RBC transfusion. Methods: This review follows the Centers for Disease Control and Prevention's Laboratory Medicine Best Practice Systematic Review (A-6) method. We searched the literature and solicited unpublished studies on practices to reduce overuse of RBC transfusions as measured by reductions in units transfused and proportion of patients transfused. Results: Thirteen studies on preoperative anemia management and three studies on audit feedback practices met inclusion criteria. Strength of evidence was high to moderate for reducing the number of units and proportion of patients transfused. Conclusions: Preoperative anemia management reduces the proportion of patients transfused and units of RBCs transfused. Audit with feedback across cases, physicians, and/or service areas, as part of a continuous quality improvement practice, reduces the proportion of patients and units of RBCs transfused.

Risk Factors for Antisocial Behavior in Low- and Middle-Income Countries: A Systematic Review of Longitudinal Studies.

Violent crime is a major cause of social instability, injury, and death in low- and middle-income countries. Longitudinal studies in high-income countries have provided important evidence on developmental precursors of violence and other antisocial behaviors. However, there may be unique influences or different risk factor effects in other social settings. Extensive searches in seven languages and screening of over 60,000 references identified 39 longitudinal studies of antisocial behavior in low- and middle-income countries. Many risk factors have roughly the same average effects as when studied in high-income countries. Stability of aggression over a 3-year period is almost identical across low- and middle-income countries and high-income countries. Dimensions of comorbid psychopathology such as low self-control, hyperactivity, and sensation seeking are associated with antisocial behavior in low- and middle-income countries, but some early physical health factors have consistently weak or null effects.

Challenges, Opportunities, and Methods for Large-Scale Evaluations.

The logic, theory, and practice of large-scale evaluation were once limited to large federal initiatives. However, with the advent of regularly collected performance measures and the often multisite implementation of quality improvement efforts, there is an opportunity to adopt large-scale evaluation methods in local and regional evaluation efforts. While ineffective programs show little variation in their ineffectiveness, effective programs generally show a range of effects. A central task of large-scale evaluation is to describe and ascribe why the same program, implemented in multiple settings, produces different effects. By its attention to variation attributable to setting, activities, outputs, and by participants and documenting the conditions in which programs achieve greater and lesser success, large-scale evaluation supports the needs of decision-makers when choosing to implement an evidence-based program. In addition to knowing a program is effective, decision-makers want to know whether it is appropriate for their situation and what facilitates or impedes effective implementation and bears on the program's ultimate effectiveness. This article presents the different methods and approaches appropriate for effectively and efficiently constructing and executing a large-scale evaluation that will provide decision-makers the evidence they need for evidence-informed adoption of effective programs.

Profile of Justice-Involved Marijuana and Other Substance Users: Demographics, Health and Health Care, Family, and Justice System Experiences.

Substance users are more likely to have co-occurring health problems, and this pattern is intensified among those involved with the criminal justice system. Interview data for 1977 incarcerated men in 5 states from the Multi-site Family Study on Incarceration, Parenting, and Partnering that was conducted between December 2008 and August 2011 were analyzed to compare pre-incarceration substance use patterns and health outcomes between men who primarily used marijuana, primarily used alcohol, primarily used other drugs, and did not use any illicit substances during that time. Using regression modeling, we examined the influence of substance use patterns on physical and mental health. Primary marijuana users comprised the largest portion of the sample (31.5%), closely followed by nonusers (30.0%), and those who primarily used other drugs (30.0%); primary alcohol users comprised the smallest group (19.6%). The substance user groups differed significantly from the nonuser group on many aspects of physical and mental health. Findings suggest that even among justice-involved men who are not using "hard" drugs, substance use merits serious attention. Expanding the availability of substance use treatment during and after incarceration might help to promote physical and mental health during incarceration and reentry.

Lipoprotein Biomarkers and Risk of Cardiovascular Disease: A Laboratory Medicine Best Practices (LMBP) Systematic Review.

BACKGROUND: Controversy exists about the incremental utility of nontraditional lipid biomarkers [e.g., apolipoprotein (apo) B, apo A-I, and non-HDL-C] in improving cardiovascular disease (CVD) risk prediction when added to a conventional model of traditional risk factors (e.g., total cholesterol, LDL cholesterol, HDL cholesterol, sex, age, smoking status, and blood pressure). Here we present a systematic review that was conducted to assess the use of nontraditional lipid biomarkers including apo B, apo A-I, apo B/A-I ratio, and non-HDL-C in improving CVD risk prediction after controlling for the traditional risk factors in populations at risk for cardiovascular events. CONTENT: This systematic review used the Laboratory Medicine Best Practices (LMBP™) A-6 methods. A total of 9 relevant studies published before and including July 2015 comprised the evidence base for this review. Results from this systematic review indicated that after the adjustment for standard nonlipid and lipid CVD risk factors, nontraditional apolipoprotein biomarkers apo B (overall effect = relative risk: 1.31; 95% CI, 1.22-1.40; 4 studies) and apo B/apo A-I ratio (overall effect = relative risk: 1.31; 95% CI, 1.11-1.38; 7 studies) resulted in significant improvement in long-term CVD risk assessment. SUMMARY: Available evidence showed that nontraditional lipid biomarkers apo B and apo B/apo I ratio can improve the risk prediction for cardiovascular events after controlling for the traditional risk factors for the populations at risk. However, because of insufficient evidence, no conclusions could be made for the effectiveness of apo A-I and non-HDL-C lipid markers to predict the CVD events, indicating a need for more research in this field.

Effectiveness of Practices To Increase Timeliness of Providing Targeted Therapy for Inpatients with Bloodstream Infections: a Laboratory Medicine Best Practices Systematic Review and Meta-analysis.

BACKGROUND: Bloodstream infection (BSI) is a major cause of morbidity and mortality throughout the world. Rapid identification of bloodstream pathogens is a laboratory practice that supports strategies for rapid transition to direct targeted therapy by providing for timely and effective patient care. In fact, the more rapidly that appropriate antimicrobials are prescribed, the lower the mortality for patients with sepsis. Rapid identification methods may have multiple positive impacts on patient outcomes, including reductions in mortality, morbidity, hospital lengths of stay, and antibiotic use. In addition, the strategy can reduce the cost of care for patients with BSIs. OBJECTIVES: The purpose of this review is to evaluate the evidence for the effectiveness of three rapid diagnostic practices in decreasing the time to targeted therapy for hospitalized patients with BSIs. The review was performed by applying the Centers for Disease Control and Prevention's (CDC's) Laboratory Medicine Best Practices Initiative (LMBP) systematic review methods for quality improvement (QI) practices and translating the results into evidence-based guidance (R. H. Christenson et al., Clin Chem 57:816-825, 2011, http://dx.doi.org/10.1373/clinchem.2010.157131). SEARCH STRATEGY: A comprehensive literature search was conducted to identify studies with measurable outcomes. A search of three electronic bibliographic databases (PubMed, Embase, and CINAHL), databases containing "gray" literature (unpublished academic, government, or industry evidence not governed by commercial publishing) (CIHI, NIHR, SIGN, and other databases), and the Cochrane database for English-language articles published between 1990 and 2011 was conducted in July 2011. DATES OF SEARCH: The dates of our search were from 1990 to July 2011. SELECTION CRITERIA: Animal studies and non-English publications were excluded. The search contained the following medical subject headings: bacteremia; bloodstream infection; time factors; health care costs; length of stay; morbidity; mortality; antimicrobial therapy; rapid molecular techniques, polymerase chain reaction (PCR); in situ hybridization, fluorescence; treatment outcome; drug therapy; patient care team; pharmacy service, hospital; hospital information systems; Gram stain; pharmacy service; and spectrometry, mass, matrix-assisted laser desorption-ionization. Phenotypic as well as the following key words were searched: targeted therapy; rapid identification; rapid; Gram positive; Gram negative; reduce(ed); cost(s); pneumoslide; PBP2; tube coagulase; matrix-assisted laser desorption/ionization time of flight; MALDI TOF; blood culture; EMR; electronic reporting; call to provider; collaboration; pharmacy; laboratory; bacteria; yeast; ICU; and others. In addition to the electronic search being performed, a request for unpublished quality improvement data was made to the clinical laboratory community. MAIN RESULTS: Rapid molecular testing with direct communication significantly improves timeliness compared to standard testing. Rapid phenotypic techniques with direct communication likely improve the timeliness of targeted therapy. Studies show a significant and homogeneous reduction in mortality associated with rapid molecular testing combined with direct communication. AUTHORS' CONCLUSIONS: No recommendation is made for or against the use of the three assessed practices of this review due to insufficient evidence. The overall strength of evidence is suggestive; the data suggest that each of these three practices has the potential to improve the time required to initiate targeted therapy and possibly improve other patient outcomes, such as mortality. The meta-analysis results suggest that the implementation of any of the three practices may be more effective at increasing timeliness to targeted therapy than routine microbiology techniques for identification of the microorganisms causing BSIs. Based on the included studies, results for all three practices appear applicable across multiple microorganisms, including methicillin-resistant Staphylococcus aureus (MRSA), methicillin-sensitive S. aureus (MSSA), Candida species, and Enterococcus species.

A Synthetic Comparator Approach to Local Evaluation of School-Based Substance Use Prevention Programming.

We propose a method for creating groups against which outcomes of local pretest-posttest evaluations of evidence-based programs can be judged. This involves assessing pretest markers for new and previously conducted evaluations to identify groups that have high pretest similarity. A database of 802 prior local evaluations provided six summary measures for analysis. The proximity of all groups using these variables is calculated as standardized proximities having values between 0 and 1. Five methods for creating standardized proximities are demonstrated. The approach allows proximity limits to be adjusted to find sufficient numbers of synthetic comparators. Several index cases are examined to assess the numbers of groups available to serve as comparators. Results show that most local evaluations would have sufficient numbers of comparators available for estimating program effects. This method holds promise as a tool for local evaluations to estimate relative effectiveness.

Effectiveness of practices to reduce blood sample hemolysis in EDs: a laboratory medicine best practices systematic review and meta-analysis.

OBJECTIVE: To complete a systematic review of emergency department (ED) practices for reducing hemolysis in blood samples sent to the clinical laboratory for testing. RESULTS: A total of 16 studies met the review inclusion criteria (12 published and 4 unpublished). All 11 studies comparing new straight needle venipuncture with IV starts found a reduction in hemolysis rates, [average risk ratio of 0.16 (95% CI=0.11-0.24)]. Four studies on the effect of venipuncture location showed reduced hemolysis rates for the antecubital site [average risk ratio of 0.45 (95% CI=0.35-0.57]. CONCLUSIONS: Use of new straight needle venipuncture instead of IV starts is effective at reducing hemolysis rates in EDs, and is recommended as an evidence-based best practice. The overall strength of evidence rating is high and the effect size is substantial. Unpublished studies made an important contribution to the body of evidence. When IV starts must be used, observed rates of hemolysis may be substantially reduced by placing the IV at the antecubital site.

Effectiveness of barcoding for reducing patient specimen and laboratory testing identification errors: a Laboratory Medicine Best Practices systematic review and meta-analysis.

OBJECTIVES: This is the first systematic review of the effectiveness of barcoding practices for reducing patient specimen and laboratory testing identification errors. DESIGN AND METHODS: The CDC-funded Laboratory Medicine Best Practices Initiative systematic review methods for quality improvement practices were used. RESULTS: A total of 17 observational studies reporting on barcoding systems are included in the body of evidence; 10 for patient specimens and 7 for point-of-care testing. All 17 studies favored barcoding, with meta-analysis mean odds ratios for barcoding systems of 4.39 (95% CI: 3.05-6.32) and for point-of-care testing of 5.93 (95% CI: 5.28-6.67). CONCLUSIONS: Barcoding is effective for reducing patient specimen and laboratory testing identification errors in diverse hospital settings and is recommended as an evidence-based "best practice." The overall strength of evidence rating is high and the effect size rating is substantial. Unpublished studies made an important contribution comprising almost half of the body of evidence.

Effectiveness of automated notification and customer service call centers for timely and accurate reporting of critical values: a laboratory medicine best practices systematic review and meta-analysis.

OBJECTIVE: To conduct a systematic review of the evidence available in support of automated notification methods and call centers and to acknowledge other considerations in making evidence-based recommendations for best practices in improving the timeliness and accuracy of critical value reporting. DESIGN AND METHODS: This review followed the Laboratory Medicine Best Practices (LMBP) review methods (Christenson, et al. 2011). A broad literature search and call for unpublished submissions returned 196 bibliographic records which were screened for eligibility. 41 studies were retrieved. Of these, 4 contained credible evidence for the timeliness and accuracy of automatic notification systems and 5 provided credible evidence for call centers for communicating critical value information in in-patient care settings. RESULTS: Studies reporting improvement from implementing automated notification findings report mean differences and were standardized using the standard difference in means (d=0.42; 95% CI=0.2-0.62) while studies reporting improvement from implementing call centers generally reported criterion referenced findings and were standardized using odds ratios (OR=22.1; 95% CI=17.1-28.6). CONCLUSIONS: The evidence, although suggestive, is not sufficient to make an LMBP recommendation for or against using automated notification systems as a best practice to improve the timeliness of critical value reporting in an in-patient care setting. Call centers, however, are effective in improving the timeliness of critical value reporting in an in-patient care setting, and meet LMBP criteria to be recommended as an "evidence-based best practice."

Effectiveness of practices to reduce blood culture contamination: a Laboratory Medicine Best Practices systematic review and meta-analysis.

OBJECTIVES: This article is a systematic review of the effectiveness of three practices for reducing blood culture contamination rates: venipuncture, phlebotomy teams, and prepackaged preparation/collection (prep) kits. DESIGN AND METHODS: The CDC-funded Laboratory Medicine Best Practices Initiative systematic review methods for quality improvement practices were used. RESULTS: Studies included as evidence were: 9 venipuncture (vs. versus intravenous catheter), 5 phlebotomy team; and 7 prep kit. All studies for venipuncture and phlebotomy teams favored these practices, with meta-analysis mean odds ratios for venipuncture of 2.69 and phlebotomy teams of 2.58. For prep kits 6 studies' effect sizes were not statistically significantly different from no effect (meta-analysis mean odds ratio 1.12). CONCLUSIONS: Venipuncture and the use of phlebotomy teams are effective practices for reducing blood culture contamination rates in diverse hospital settings and are recommended as evidence-based "best practices" with high overall strength of evidence and substantial effect size ratings. No recommendation is made for or against prep kits based on uncertain improvement.

A national evaluation of Safe Schools/Healthy Students: outcomes and influences.

The Safe Schools/Healthy Students (SS/HS) Initiative has awarded over $2 billion in grants to more than 350 school districts in partnership with local mental health, law enforcement, and juvenile justice agencies. To estimate the impact of grantee characteristics, grant operations, and near-term outcomes in reducing violence and substance use, promoting mental health, and enhancing school safety, logged odds ratios (LORs) were calculated contrasting Year 3 with Baseline performance from grantee-provided data on seven outcome measures. After comparing grantee performance across outcomes and outcomes across grantees, the LORs were entered as dependent variables in a series of meta-regressions in which grantee characteristics, grant operations, and near-term outcomes were tested after controlling for pre-grant characteristics. Findings indicate that the SS/HS Initiative significantly improved most outcomes, that within-grantee performance varied greatly by outcome, and that random-effects meta-regression appreciably decreased the variance available for modeling. The approach demonstrates that the SS/HS Initiative is effective and that locally collected performance data can be used to estimate grantee success in improving youth outcomes.

Laboratory medicine best practices: systematic evidence review and evaluation methods for quality improvement.

OBJECTIVE: To develop methods for systematically reviewing evidence for identifying effective laboratory medicine (LM) practices associated with improved healthcare quality outcomes. RELEVANCE: Although many evidence-evaluation systems have been developed, none are designed to include and rate healthcare quality improvement studies to identify evidence-based practices that improve patient safety and LM quality. METHODS: Validated evidence-based medicine methods established by governmental agencies, the Guide to Community Preventive Services, and others were adapted for the LM field. Key methods modifications included (a) inclusion of quality improvement study designs; (b) mechanisms for inclusion of unpublished evidence, (c) combining of individual ratings of study quality, effect size, and relevance of outcome measures to evaluate consistency of practice evidence; and (d) deriving an overall strength rating to support evidence-based best practice recommendations. The methods follow the process steps of: ask; acquire; appraise; analyze; apply; and assess. Expert panels used the systematic evidence review results on practice effectiveness for improving healthcare quality outcomes consistent with the Institute of Medicine's healthcare quality aims (safe, timely, effective, equitable, efficient, and patient-centered). CONCLUSIONS: Adapting and developing methods from validated systems and applying them to systematically review and evaluate practices in LM by using published and unpublished studies is feasible. With these methods, evidence from quality improvement studies can be systematically synthesized and summarized to identify effective LM practices. Practical and scientifically validated demonstration of a positive impact on outcomes ensures that practitioners, policy makers, and decision makers at all levels have the evidence needed for improving healthcare quality and public health.

The use of one- versus two-tailed tests to evaluate prevention programs.

Investigators have used both one- and two-tailed tests to determine the significance of findings yielded by program evaluations. While the literature that addresses the appropriate use of each type of significance test should be used is historically inconsistent, almost all authorities now agree that one-tailed tests are rarely (if ever) appropriate. A review of 85 published evaluations of school-based drug prevention curricula specified on the National Registry of Effective Programs and Practices revealed that 20% employed one-tailed tests and, within this subgroup, an additional 4% also employed two-tailed tests. The majority of publications either did not specify the type of statistical test employed or used some other criterion such as effect sizes or confidence intervals. Evaluators reported that they used one-tailed tests either because they stipulated the direction of expected findings in advance, or because prior evaluations of similar programs had yielded no negative results. The authors conclude that one-tailed tests should never be used because they introduce greater potential for Type I errors and create an uneven playing field when outcomes are compared across programs. The authors also conclude that the traditional threshold of significance that places α at .05 is arbitrary and obsolete, and that evaluators should consistently report the exact p values they find.

A preliminary study of the population-adjusted effectiveness of substance abuse prevention programming: towards making IOM program types comparable.

The Institute of Medicine distinguishes between programs based on who is targeted: the entire population (universal), those at risk (selective), or persons exhibiting the early stages of use or related problem behavior (indicated). Evaluations suggest that although universal programs can be effective in reducing and preventing substance use, selective and indicated programs are both more effective and have greater cost-benefit ratios. This paper tests these assumptions by comparing the impact of these program types in reducing and preventing substance use at the individual level (i.e., those exposed to intervention services) and in the population (i.e., those exposed and not exposed to intervention services). A meta-analysis was performed on 43 studies of 25 programs to examine program comparability across IOM categories. When examining unadjusted effect sizes at the individual level, universal programs were modestly more successful in reducing tobacco use, but selective and indicated programs were modestly more successful in reducing alcohol and marijuana use. When adjusted to the population level, the average effect sizes for selective and indicated programs were reduced by approximately half. At the population level, universal programs were more successful in reducing tobacco and marijuana use and selective and indicated programs were more successful in reducing alcohol use. Editors' Strategic Implications: the authors' focus on the public health value of a prevention strategy is compelling and provides a model for analyses of other strategies and content areas.