The Aligning Forces for Quality initiative: background and evolution from 2005 to 2015.

OBJECTIVE: The Robert Wood Johnson Foundation's (RWJF's) Aligning Forces for Quality (AF4Q) program was the largest privately funded, community-based quality improvement initiative to date, providing funds and technical assistance (TA) to 16 multi-stakeholder alliances located throughout the United States. This article describes the AF4Q initiative's underlying theory of change, its evolution over time, and the key activities undertaken by alliances. STUDY DESIGN: Descriptive overview of a multi-site, community-based quality improvement initiative. METHODS: We summarized information from program documents, program meetings, observation of alliance activities, and interviews with RWJF staff, TA providers, and AF4Q alliance stakeholders. RESULTS: The AF4Q program was a dynamic initiative, expanding and evolving over time. The underlying theory of change was based on the notion that an aligned, multi-stakeholder approach is superior to independent siloed efforts by stakeholders. Participating alliances developed or strengthened programming to varying degrees in 5 main programmatic areas: (1) measurement and public reporting of healthcare quality, patient experience, cost, and efficiency for ambulatory physician practices and hospitals; (2) efforts to engage consumers in health, healthcare, and alliance governance (consumer engagement); (3) adoption and spread of effective strategies to improve care delivery; (4) advancing healthcare equity; and (5) integration of alliance activities with payment reform initiatives. CONCLUSION: The AF4Q initiative was an ambitious program affecting multiple leverage points in the healthcare system. AF4Q alliances were provided a similar set of expectations, and given financial support and access to substantial TA. There was considerable variation in how alliances addressed the AF4Q programmatic areas, given differences in their composition, market structure, and history.

Summative evaluation results and lessons learned from the Aligning Forces for Quality program.

OBJECTIVE: To report summative evaluation results from the Aligning Forces for Quality (AF4Q) initiative, the Robert Wood Johnson Foundation's (RWJF's) signature effort to improve quality of care from 2005 to 2015. METHODS: This was a longitudinal mixed methods program evaluation (ie, multiphase triangulated evaluation) of 16 grantee "alliances" from across the country, funded by RWJF as part of the AF4Q initiative. Grantees were selected in a nonexperimental manner and were charged with deploying interventions in 5 main programmatic areas to improve health and healthcare in their communities. RESULTS: Except for a small proportion of outcomes, there were no major differences in the rate of longitudinal improvement in AF4Q communities, compared with control communities, on quantitative outcomes related to the Triple Aim. Although the majority of the measures improved in both AF4Q and non-AF4Q communities, there were some exceptions to this improving trend, most noticeably in the cost of care and population health. There was also considerable heterogeneity across communities in terms of programmatic areas and the scale and scope of interventions in these areas. Although a number of AF4Q alliances implemented robust interventions in specific areas, often advancing strategies useful for others in the field, no AF4Q alliance pursued and aligned all 5 AF4Q programmatic areas in a robust way. In addition, whereas all alliances were able to garner the participation of multiple stakeholders initially, sustaining this participation and securing new sources of funding after RWJF support ended proved challenging for many alliances. Conclusion and Policy and Practice Implications: While the AF4Q program did not attain the ambitious community-level changes predicted by its sponsor at the program's outset, it did produce pockets of success on some dimensions for particular alliances. A number of factors explain the less-than-expected impact of the AF4Q initiative on community health and the observed variation in alliance sustainability and intervention strength. These include differing acceptance of the AF4Q initiative's theory of change, variation in the experience and capacity of the alliance communities selected for the program, differences in alliances' local healthcare market context, and the changing programmatic requirements for alliances participating in the AF4Q initiative. The variation in AF4Q program outcomes offers important lessons for those engaged in regional health improvement work.

Building the scaffold to improve health care quality in Western New York.

Many health policy leaders are promoting the community as a place to try out new ideas for improving the quality of health care. Alliances with multiple stakeholders are moving forward with communitywide efforts to improve the quality of care without the benefit of an established evidence base or guiding framework. This article presents a profile of one community's attempt to facilitate and coordinate quality improvement in its geographic area. The P(2) Collaborative of Western New York is one of sixteen sites supported by the Robert Wood Johnson Foundation's national Aligning Forces for Quality initiative. The strategy and vision of the collaborative has evolved as it has tried to capitalize on opportunities and overcome barriers in its work. The article concludes with a discussion of eight tasks that community alliances may consider undertaking when establishing an infrastructure for improving the quality of health care, such as convening area stakeholders to develop a strategy and finding ways to monitor health outcomes at the local level on an ongoing basis.

Midterm observations and recommendations from the evaluation of the AF4Q initiative.

OBJECTIVE: To offer midterm observations and recommendations based on how Aligning Forces for Quality (AF4Q) alliances are faring in their journey toward improving healthcare quality at the community level. STUDY DESIGN: This study used a mixed method design. METHODS: Longitudinal evaluation data to date were analyzed, including results from multiple surveys, qualitative analysis of key informant interviews, review of secondary documents and analysis of secondary data, and ongoing tracking of the activities of the 16 participating alliances. The observations and recommendations are based on consensus achieved by the AF4Q evaluation team investigators after in-depth iterative discussions. RESULTS: Six formative observations are identified and discussed: (1) stakeholder support and participation has been maintained despite changes in economic and political environments; (2) progress on program goals has been slow; (3) the "alignment" in the AF4Q initiative has been slow to materialize; (4) the AF4Q initiative has established a productive network of peer communities; (5) the impact of the AF4Q initiative, and the time to observe impact, vary by community, based on history and context; and (6) sustainability is the major future challenge for the AF4Q initiative. CONCLUSIONS: Multi-stakeholder alliances' efforts to improve quality should be viewed as "pieces of the health reform puzzle" rather than stand-alone solutions. As healthcare reform is challenged politically, alliances can practice the bipartisanship that focuses conversation on what is good for the community and how best to achieve community goals amid a potential sea of change in both federal and state policy and funding.

The Aligning Forces for Quality initiative: background and evolution from 2005 to 2012.

OBJECTIVES: The Robert Wood Johnson Foundation's (RWJF's) Aligning Forces for Quality (AF4Q) initiative is the largest privately funded community-based quality improvement initiative to date, providing funds and technical assistance (TA) to 16 multi-stakeholder alliances located throughout the United States. The objectives of this article are to describe the AF4Q initiative's underlying theory of change, its evolution over time, and the key activities undertaken by alliances. STUDY DESIGN: This is a descriptive overview of a major multi-site, community-based quality improvement initiative. METHODS: A qualitative approach was used with information obtained from program documents, program meetings, observation of alliance activities, and interviews with RWJF staff, TA providers, and AF4Q alliance stakeholders. RESULTS: AF4Q is a dynamic initiative, expanding and evolving over time. Participating alliances are addressing 5 main programmatic areas: (1) measurement and public reporting of healthcare quality and efficiency for ambulatory physician practices and hospitals; (2) efforts to engage consumers as partners in their own care (consumer engagement); (3) adoption and spread of effective quality improvement strategies to improve care; (4) ensuring the equitable receipt of healthcare; and (5) integration of alliance activities with payment reform initiatives. CONCLUSIONS: The AF4Q initiative is an ambitious program affecting multiple leverage points in the healthcare system. AF4Q alliances were provided a similar set of expectations and given access to substantial TA. While participating alliances have made progress in addressing the AF4Q programmatic areas, given differences in the alliances' composition, market structure, and history, there is considerable variation in program implementation.

Barriers and strategies to align stakeholders in healthcare alliances.

OBJECTIVE: To identify barriers to stakeholder alignment and strategies used by 14 multi-stakeholder alliances participating in the Aligning Forces for Quality initiative to overcome these barriers. STUDY DESIGN: The study used a mixed method, comparative case study design. METHODS: Alliances were categorized as more or less highly aligned based on an alignment index constructed from survey responses. Six alliances (top and bottom quartile) were selected for more in-depth qualitative analysis. Semi-structured interviews of key informants were used to identify factors that distinguished more highly aligned alliances from less highly aligned alliances. RESULTS: Market context was one of the most important factors differentiating alliances. More highly aligned alliances had more extensive histories of collaboration, established more credibility in the local community, and were more effective at balancing collaborative initiatives against competitive interests. More highly aligned alliances also took more active approaches to build consensus among stakeholders regarding alliance initiatives, and were able to successfully utilize small decision-making bodies to foster this consensus. In contrast, leadership credibility, leadership stability, and trust were important facilitators of alignment for all alliances, regardless of the level of alignment. These factors intersect and overlap in a multitude of ways to influence stakeholder alignment. CONCLUSIONS: Alignment in an alliance context is critical for leveraging the unique knowledge, skills, and abilities of stakeholders in ways that can build capacity to improve the health of the community in ways that cannot be achieved independently by stakeholders. The findings highlight the need for multifaceted approaches to promote stakeholder alignment.

Approaches to improving healthcare delivery by multi-stakeholder alliances.

OBJECTIVE: Our purposes were: (1) to describe how 14 multi-stakeholder alliances participating in the Aligning Forces for Quality (AF4Q) initiative approached the charge of improving healthcare delivery at the community level between 2006 and 2010; and (2) to offer insights to policy makers and program planners seeking to promote or establish community-wide quality improvement (QI). STUDY DESIGN: This was a qualitative study. METHODS: A total of 84 semi-structured interviews were conducted with AF4Q alliance leaders between 2006 and 2010, and an iterative coding process was used to identify salient themes. Program documents supplemented the interview data and were used to develop an inventory of the alliances' QI activities using the Leatherman and Sutherland taxonomy of quality-enhancing interventions. RESULTS: Alliances spent years planning their QI approaches and activities. Initial selection of QI activities was driven by the availability of local expertise and resources, rather than alignment with a community-wide vision for quality. Alliances were just as likely to rely on local partners to lead QI activities as they were to establish their own activities. The most commonly adopted QI activities were collaboratives aimed at producing organizational-level changes. CONCLUSIONS: Policy makers and program planners seeking to promote community-wide QI should consider developing clear expectations, offering technical assistance at the start of the program, providing information on the evidence base for QI activities, and highlighting additional funding opportunities that could support QI activities. Alliances may need a stronger push to move beyond coordinated, organizational-level activities to more community-focused, cross-organizational QI activities.

Evaluating a community-based program to improve healthcare quality: research design for the Aligning Forces for Quality initiative.

OBJECTIVE: The Aligning Forces for Quality (AF4Q) initiative is the Robert Wood Johnson Foundation's (RWJF's) signature effort to increase the overall quality of healthcare in targeted communities throughout the country. In addition to sponsoring this 16-site, complex program, the RWJF funds an independent scientific evaluation to support objective research on the initiative's effectiveness and contributions to basic knowledge in 5 core programmatic areas. The research design, data, and challenges faced in the evaluation of this 10-year initiative are discussed. STUDY DESIGN: A descriptive overview of the evaluation research design for a multi-site, community based, healthcare quality improvement initiative is provided. METHODS: The multiphase research design employed by the evaluation team is discussed. RESULTS: Evaluation provides formative feedback to the RWJF, participants, and other interested audiences in real time; develops approaches to assess innovative and under-studied interventions; furthers the analysis and understanding of effective community-based collaborative work in healthcare; and helps to differentiate the various facilitators, barriers, and contextual dimensions that affect the implementation and outcomes of community-based health interventions. CONCLUSIONS: The AF4Q initiative is arguably the largest community-level healthcare improvement demonstration in the United States to date; it is being implemented at a time of rapid change in national healthcare policy. The implementation of large-scale, multi-site initiatives is becoming an increasingly common approach for addressing problems in healthcare. The evaluation research design for the AF4Q initiative, and the lessons learned from its approach, may be valuable to others tasked with evaluating similar community-based initiatives.

A community-level effort to motivate physician participation in the National Committee for Quality Assurance Diabetes Physician Recognition Program.

This article presents the results of an exploratory case study that assessed a multistakeholder alliance's initiative to motivate quality improvement in primary care physician (PCP) practices. The initiative utilized a "pay-for participation" strategy that provided incentives to PCPs to participate in the National Committee for Quality Assurance's Diabetes Physician Recognition Program (DPRP). The intervention took place over a 2-year period in 8 practices with large safety-net populations located in the Rochester, New York area. The outcomes of interest were receipt of DPRP recognition and performance on DPRP measurements by the practices, as well as qualitative information regarding practice decisions about quality improvement. Of 79 physicians who participated, 37 (47%) received DPRP recognition. Receipt of recognition was likely the result of a combination of preexisting performance and improvements in processes made during the project. While sample size prevented hypothesis testing, size of practice was unrelated to receipt of DPRP recognition. All practices with an electronic medical record and a patient registry achieved recognition. Strong physician leadership and the presence of a quality improvement infrastructure were believed to be associated with DPRP recognition. The majority of practices cited the program's honorarium and other incentives as key motivators for participation. Our findings suggest that pay-for-participation may be a viable strategy to promote quality improvement in physician practices. However, absent continuing reinforcement, it is uncertain if such programs can lead to sustained quality improvement activities.

Financial and clinical impact of team-based treatment for medicaid enrollees with diabetes in a federally qualified health center.

OBJECTIVE: The purpose of this study was to determine whether multidisciplinary team-based care guided by the chronic care model can reduce medical payments and improve quality for Medicaid enrollees with diabetes. RESEARCH DESIGN AND METHODS: This study was a difference-in-differences analysis comparing Medicaid patients with diabetes who received team-based care versus those who did not. Team-based care was provided to patients treated at CareSouth, a multisite rural federally qualified community health center located in South Carolina. Control patients were matched to team care patients using propensity score techniques. Financial outcomes compared Medicaid (and Medicare for dually eligible patients) payments 1 year before and after intervention. Trends over time in levels of A1C, BMI, and systolic blood pressure (SBP) were analyzed for intervention patients during the postintervention period. RESULTS: Although average claims payments increased for both the CareSouth patients and control patients, there were no statistically significant differences in total payments between the two groups. In the intervention group, patients with A1C >9 at baseline experienced an average reduction of 0.75 mg/dl per year (95% CI 0.50-0.99), patients with BMI >30 at baseline had an average reduction of 2.3 points per year (95% CI 0.99-3.58), and patients with SBP >140 mmHg at baseline had an average reduction of 2.2 mmHg per year (95% CI 0.44-3.88). CONCLUSIONS: Team-based care following the chronic care model has the potential to improve quality without increasing payments. Short-term savings were not evident and should not be assumed when designing programs.

The role of disease management in pay-for-performance programs for improving the care of chronically ill patients.

To date, pay-for-performance programs targeting the care of persons with chronic conditions have primarily been directed at physicians and provide an alternative to health plan-sponsored chronic disease management (DM) programs. Both approaches require similar infrastructure, and each has its own advantages and disadvantages for program implementation. Pay-for-performance programs use incentives based on patient outcomes; however, an alternative system might incorporate measures of structure and process. Using a conceptual framework, the authors explore the variation in 50 diabetes DM programs using data from the 2002 National Business Coalition on Health's eValue8 Request for Information (RFI). The authors raise issues relevant to the assignment of accountability for patient outcomes to either health plans or physicians. They analyze the association between RFI scores measuring structures and processes, and HEDIS diabetes intermediate outcome measures. Finally, the strengths and weaknesses of using the RFI scores as an alternative metric for pay-for-performance programs are discussed.

Determining the value of disease management programs.

BACKGROUND: Increasing prevalence, rising costs, and persisting deficiencies in quality of care for chronic diseases pose economic and policy challenges to providers and purchasers. Disease management (DM) programs may address these challenges, but neither purchasers nor providers can assess their value. The potpourri of current quality indicators provides limited insight into the actual clinical benefit achieved. A conference sponsored by the Agency for Healthcare Research and Quality (AHRQ) and held in October 2002 explored new approaches to measuring and reporting the value of DM for diabetes mellitus. RESULTS: Quantifying the value of DM requires measuring clinical benefit and net impact on health care costs for the entire population with diabetes. If quality is measured with indicators that are clearly linked to outcomes, clinical benefit can be estimated. Natural history models combine the expected benefits of improvements in multiple indicators to yield a single, composite measure, the quality-adjusted life-year. Such metrics could fairly express, in terms of survival and complications prevention, relatively disparate DM programs' benefits. Measuring and comparing health care costs requires data validation and appropriate case-mix adjustment. Comparing value across programs may provide more accurate assessments of performance, enhance quality improvement efforts within systems, and contribute generalizable knowledge on the utility of DM approaches. CONCLUSIONS: Conference attendees recommended pilot projects to further explore use of natural history models for measuring and reporting the value of DM.

Measuring outcomes of type 2 diabetes disease management program in an HMO setting.

BACKGROUND: There is a need to evaluate empirical disease management programs used in managing chronic diseases such as diabetes mellitus in managed care settings. METHODS: We analyzed data from 252 patients with type 2 diabetes before and 1 year after enrollment in a disease management program. We examined clinical indicators such as HbA1C, HDL, LDL, total cholesterol, diastolic blood pressure, and BMI in addition to self-reported health status measured by SF-36 instrument. RESULTS: All clinical indicators showed statistically and clinically significant improvements. Only vitality and mental health showed statistically significant improvements in health status. Weak to moderate significant correlation between clinical indicators and health status was observed. CONCLUSIONS: Disease management can be effective at making significant clinical improvements for participants in a mixed-model HMO setting. No strong relationship between clinical indicators and health status was found. Future research is needed using a more specific health status measuring instrument and a randomized clinical trial design.