Social validation of goals, procedures, and effects in public health.

By relying solely on tests of statistical significance as the measure of success for interventions in public health, and in community-based participatory research, we can miss important social dimensions of the project. These dimensions include how our interventions might affect the lives of participants (social validity) and the lives of people more broadly (clinical or public health significance). Social validation procedures were originally developed to assess the acceptability of procedures and effects of behavioral interventions among clients and consumers. This article describes the methods used to obtain social validity data for goals, procedures, and effects of interventions in health settings. The challenges in using these procedures are also discussed, and suggestions are offered for future research and practice in this area.

Providing asthma care in elementary schools: understanding barriers to determine best practices.

School nurses play a critical role in the management of children's asthma, yet they face barriers in their efforts to deliver quality care. In this qualitative study involving focus groups with school nurses, we identified key barriers in order to inform best practices. School nurses identified 4 main barriers to effective asthma care in elementary schools: lack of education, lack of communication, lack of resources, and lack of respect. An analysis of the barriers suggests that best practices for asthma care in elementary school settings require collaborative strategies that involve schools, families, the community, and the healthcare profession.

Building allies coalitions: Why formation matters.

This article describes how formation of the Allies Against Asthma coalitions was influenced by community context, history, leadership, membership, structure, processes, and other factors. Based on member surveys, key informant interviews, and self-assessment tools, results indicate that three coalitions developed from existing coalitions, and four formed in response to funding. The coalitions took an average of 12 months to form and completed 98% of formation tasks. Funding, technical support, and networking among grantees promoted formation. Although cultures, geographies, lead agencies, and years of experience differed, coalitions developed similar structures and processes. Two of three new coalitions took the longest to form, highlighting the need for flexibility and technical assistance when funding new coalitions.

From formation to action: How allies against asthma coalitions are getting the job done.

Coalitions develop in and recycle through stages. At each stage (formation, implementation, maintenance, and institutionalization), certain factors enhance coalition function, accomplishment of tasks, and progression to the next stage. The Allies Against Asthma coalitions assessed stages of development through annual member surveys, key informant interviews of 16 leaders from each site, and other evaluation tools. Results indicate all coalitions completed formation and implementation, six achieved maintenance, and five are in the institutionalization stage. Differences among coalitions can be attributed to their maturity and experience working within a coalition framework. Participants agreed that community mobilization around asthma would not have happened without coalitions. They attributed success to being responsive to community needs and developing comprehensive strategies, and they believed that partners' goals were more innovative and achievable than any institution could have created alone.

Engaging the community in coalition efforts to address childhood asthma.

For health improvement efforts to effectively address community needs, community members must be engaged in planning and implementing public health initiatives. For Allies Against Asthma's coalitions, the community included not only the subpopulation of individuals who suffer disproportionately from asthma but also the individuals and institutions that surround them. Through a quantitative self-assessment survey, informal discussion among coalition leadership, and interviews with key informants, data relevant to community engagement identified a number of important ways the Allies coalitions approached community involvement. Respondents' comments made clear that the way the coalitions conduct their work is often as important as what they do. Across coalitions, factors that were identified as important for community involvement included (a) establishing a commitment to community involvement, (b) building trust, (c) making participation feasible and comfortable, (d) responding to community identified needs, (e) providing leadership development opportunities, and (f) building a shared commitment to desired outcomes.

Childhood asthma surveillance by community coalitions.

Multiple benefits can accrue when community coalitions conduct asthma surveillance activities. Surveillance data are used to identify children with asthma, assess disease burden and needs in the community, understand the illness and risk factors, identify children with asthma who are undertreated, plan community interventions, evaluate the effect of interventions, and monitor trends. These data, which are used to inform coalition and program decisions and to evaluate asthma interventions, can also be used to strengthen state and national asthma surveillance efforts and to inform clinical practice and public health policies. Local coalition data collection represents a complementary approach to national asthma surveillance, allowing action at the local level and showing how local findings vary from national observations. The Allies Against Asthma coalitions developed several practical means to conduct childhood asthma surveillance that informed coalition efforts and facilitated innovative linkages among government officials, health care providers, community agencies, families, and academicians and/or researchers.

Allies community health workers: Bridging the gap.

Allies Against Asthma coalitions each employ a community health worker (CHW) program as part of its community action plan. The structure and management of CHW programs vary in response to the resources and needs of the local community, as do the roles and characteristics of the CHWs hired. All programs utilize CHWs to provide community-based education and/or outreach to community members, primarily in their homes. Using an asthma action plan, most Allies CHW programs function as an extension of and link to the clinician, providing basic asthma education and care coordination in a supportive, family-friendly setting, context, and location. Community health workers rely heavily on relationship building and family empowerment to assist families in improving asthma control. Working within a coalition framework helps integrate the CHW program into other services and resources in the community. As participants in coalition activities, CHWs often bring an important and meaningful viewpoint to the coalition.

The coalition process at work: Building care coordination models to control chronic disease.

Asthma is a highly prevalent and frequently misunderstood chronic disease with significant morbidity. Integrating client services at the patient-centered level and using coalitions to build coordinated, linked systems to affect care may improve outcomes. All seven Allies Against Asthma coalitions identified inefficient, inconsistent, and/or fragmented care as issues for their communities. In response, the coalitions employed a collaborative process to identify and address problems related to system fragmentation and to improve coordination of care. Each coalition developed a variety of interventions related to its specific needs and assets, stakeholders, stage of coalition formation, and the dynamic structure of its community. Despite common barriers in forming alliances with busy providers and their staff, organizing administrative structures among interinstitutional cultures, enhancing patient and/or family involvement, interacting with multiple insurers, and contending with health system inertia, the coalitions demonstrated the ability to produce coordinated improvements to existing systems of care.

Using the State Plan Index to evaluate the quality of state plans to prevent obesity and other chronic diseases.

INTRODUCTION: Implicit in public health planning models is the assumption that good public health plans lead to good programs, and good programs lead to desired health outcomes. Despite considerable resources that are devoted to developing plans, public health agencies and organizations have lacked a tool for evaluating the finished product of their planning efforts -- the written plan itself -- as an important indicator of progress. To address the need for an instrument to assess the quality of state plans designed to prevent and control chronic diseases, we created and tested the State Plan Index and used it to evaluate the quality of nine state plans aimed at preventing and reducing obesity. METHODS: The State Plan Index was developed under the auspices of the Centers for Disease Control and Prevention (CDC) in collaboration with public health experts in federal, state, and academic settings. The State Plan Index included 55 items related to plan quality arranged into nine components. Each item was rated on a Likert scale from 0 to 5, with 5 being the highest rating. Each plan also received a separate overall plan quality score using the same scale. Each state plan was evaluated by four or five raters using the State Plan Index. For each plan, the 55 items were averaged to calculate an item average score, and a subscore was calculated for each State Plan Index component. Finally, five states also self-rated their own plans (self score). RESULTS: The mean item average score for all plans was 2.4 out of 5.0. The range of item average scores was 1.0 to 3.0. The component of the State Plan Index with the highest mean component score (3.3) was Presentation of Epidemiologic Data on Disease Burden. The components with the lowest component scores were Resources for Plan Implementation (0.7); Integration of Obesity Efforts with Other Chronic Disease Efforts (1.7); and Program Evaluation (2.0). Plan quality was rated higher when based on the single overall plan quality score assigned by raters. In addition, self scores were consistently and substantially higher than rater-assigned scores. CONCLUSION: Evaluation of plans early in the life of programs can be used to strengthen existing programs and to guide programs newly engaged in chronic disease prevention planning. The CDC has used the State Plan Index evaluation results to guide technical assistance, plan training sessions, and enhance communication with state staff about plan content, quality, and public health approach. Some state program directors self-evaluated their obesity draft plan and used the evaluation results to strengthen their planning process and to guide plan revisions. Other states have adapted the State Plan Index as a framework for new planning efforts to prevent obesity as well as other chronic diseases.

State Plan Index: a tool for assessing the quality of state public health plans.

INTRODUCTION: The State Plan Index is an evaluation instrument that uses a Likert scale to assess 60 indicators of the quality of state public health plans. The State Plan Index was needed to enable evaluation of plans that were developed using a variety of public health planning models. METHODS: Federal, state, and academic partners participated in developing and testing the instrument. The authors conducted a literature review, interviews with experts, and several rounds of formative evaluation to assess item inclusion, coverage, weighting, organization of items, and content validity. In two rounds of field testing, public health practitioners at the federal and state levels rated 10 state public health plans for obesity prevention. RESULTS: Field-test raters took an average of two hours to rate a plan and indicated that the State Plan Index was "easy to use," "comprehensive," and "fair." Mean Cronbach alpha for components of the State Plan Index was 0.88 (median 0.93). Component scores among the 10 plans rated ranged from 0.2 to 4.8, indicating that raters made distinctions in quality among the components and the plans they rated. Correlations between component scores and overall scores were statistically significant (P < .001), except for one component. CONCLUSION: Public health professionals at the federal and state levels found the State Plan Index to be a useful tool for evaluating public health plans that were developed by states using various planning approaches. After the field tests, state staff reported adapting the State Plan Index for use as a planning tool, an evaluation tool for local plans, and a self-assessment tool for drafts of state plans. In addition, the State Plan Index can be revised easily for use in other chronic disease areas.

Health planning that magnifies the community's voice: allies against asthma.

Allies Against Asthma, a working group of the Consortium for Infant and Child Health (CINCH), conducted a comprehensive asthma needs assessment in Hampton Roads, Virginia, in 2001. Results from extant data and parent surveys indicated that asthma prevalence was high (15% to 18%), 45% to 50% of children received primary care for asthma in the emergency department, 30% had been recently hospitalized, and most children were not adequately medicated. Focus groups revealed inadequate asthma education, low income, lack of resources and consistent care, disparities in insurance coverage, and noncompliance with national asthma guidelines. An integrated community asthma action plan was developed and funded. Members were satisfied with the planning process--88% felt the plan reflected the needs assessment, and 86% agreed the plan would effectively improve asthma management. Interventions commenced in January 2002. The inclusive process that led to these interventions will ensure that the project is successful and sustainable.

Evaluating community partnerships and coalitions with practitioners in mind.

Evaluation plays a key role in developing and sustaining community partnerships and coalitions. We recommend focusing on three levels of coalition evaluation that measure (a) processes that sustain and renew coalition infrastructure and function; (b) programs intended to meet target activities, or those that work directly toward the partnership's goals; and (c) changes in health status or the community. A tendency to focus on quick wins and short-term effects of programs may explain why some coalitions are not able to achieve systems and/or health outcomes change. Although measuring community-level or system changes (e.g., improving environmental quality or changing insurance coverage policies) is much more difficult than evaluating program outcomes, it is essential. This article presents challenges that coalition practitioners and evaluators face and concludes with practical resources for evaluation.

The coalition technical assistance and training framework: helping community coalitions help themselves.

Coalition staff, leaders, and members need training to promote coalition building and maintenance as well as ongoing technical assistance. The Coalition Technical Assistance and Training Framework uses a 6-step process to diagnose coalition strengths and challenges and provide a prescription for action. Re-evaluation after a specified time period helps determine whether a coalition adopted recommended changes in coalition participants, structures, and/or processes and progressed through stages of development. This empowerment approach helps coalition staff and members help themselves to be more effective and efficient. The framework was piloted with the Virginia Healthy Start Initiative from November 1997 to June 2001. Seven perinatal councils that focused on preventing low-weight births and infant mortality adopted 75% of the recommended actions within 1 year. Results from a pre and post-assessment tool after 3 years showed significant progress in the coalitions' ability to develop effective participants, processes, and structures

The Coalition Training Institute: training for the long haul.

Public-private partnerships are integral to our public health paradigm. The Coalition Training Institute (1995-1998) trained 283 participants from 29 U.S. cities, 49 states, and 7 U.S. territories to foster and sustain partnerships that improve immunization rates. Evaluation consisted of on-site and follow-up surveys, effectiveness inventories, and focus groups. The Institute met participants' expectations. Four months later, participants reported training was applicable (93%) and helpful in overcoming organizational barriers. Most built or improved coalitions (81%), helped organizations apply new ideas (86%), and obtained training/support (60%). Participants requested more on-site and distance-learning opportunities to network, train coalition leaders and members, and learn new skills.

How do we know if we are making a difference with our program or community initiative?

In this article, the authors discuss issues related to knowing if health workers are making a difference with their community initiatives or programs being delivered in a community setting. Issues relating to the choice of evaluation questions, relevance of data sets used to answer those questions, and the rigor by which the answer can be evaluated--and the ability to generalize to other communities or populations--are explored. Finally, relevant resources to help you pursue these issues are provided.