Improving Outcome Accountability of Block Grants: Lessons Learned From the Preventive Health and Health Services Block Grant Evaluation.

The flexibility federal block grants provide recipients poses challenges for evaluation. These challenges include aggregating data on wide-ranging activities grant recipients implement and the outcomes they achieve. In 2014, we began designing an evaluation to address the challenges of assessing outcomes and to improve outcome accountability for the Preventive Health and Health Services Block Grant. Through the use of evaluability assessment methodology, review of existing data and the literature, and key informant interviews, we developed a measurement framework to assess outcomes resulting from recipients' ability to use grant funds to meet their locally prioritized needs. We argue our evaluation approach demonstrates that block grants, and other similarly flexible programs, can be evaluated through appropriately designed measures. Our efforts challenge the idea that flexibility presents an insurmountable barrier to evaluation and outcome accountability for federal block grants.

Tracking Public Health Workforce Retention: Observations From CDC's Public Health Associate Program.

Objectives. To describe the career trajectories of 1 cohort of US Public Health Associate Program (PHAP) alumni over 3 years since completing PHAP.Methods. We distributed a Web-based survey at 3 time points between 2014 and 2017 (response rate = 76%). We calculated descriptive statistics in SPSS.Results. At all time points, most alumni were employed. Of those, the percentage employed in public health was 100% at program completion, 86% at year 1, and 68% at year 3.Conclusions. Most alumni were employed in public health jobs at each time point. At the 3-year mark, approximately a third of the alumni had left public health employment, which is in line with documented rates of turnover within the broader public health workforce.Public Health Implications. Service learning programs like PHAP are effective at recruiting early career professionals into public health. The extent to which PHAP is effective at retaining workers in public health after the program appears most promising immediately following the program or in the short term after the program concludes. The extent to which workers are retained in the longer term requires further study.

CDC's Prevention Status Reports: Monitoring the Status of Public Health Policies and Practices for Improved Performance and Accountability.

CONTEXT: Increasing the adoption and implementation of evidence-based policies and practices is a key strategy for improving public health. Although there is widespread agreement about the importance of implementing evidence-based public health policies and practices, there are gaps between what has been shown to be effective and what is implemented at the state level. OBJECTIVE: The Centers for Disease Control and Prevention (CDC) developed the Prevention Status Reports (PSRs), a performance measurement system, to highlight evidence-based public health policies and practices and catalyze state performance and quality improvement efforts across the nation. DESIGN: CDC selected a set of 10 topics representing some of the most important public health challenges in the nation. Stakeholders, including state health departments and other partners, helped conceptualize the PSRs and informed the development of the PSR framework, which provides an organizational structure for the system. CDC subject matter experts developed criteria for selecting policies and practices, indicators for each policy and practice, and a criteria-based rating system for each indicator. PARTICIPANTS AND SETTING: The PSRs were developed for all 50 states and the District of Columbia. MAIN OUTCOME: The PSRs were developed and serve as a performance measurement system for monitoring the adoption, reach, and implementation fidelity of evidence-based public health policies and practices nationwide. RESULTS: The PSRs include 33 policy and practice indicators across the 10 health topics. They use a simple 3-level rating system-green, yellow, and red-to report the extent to which each state (and the District of Columbia) has implemented the policy or practice in accordance with supporting evidence or expert recommendations. Results from aggregate analyses show positive change or improvement. CONCLUSION: The PSRs are a unique part of CDC's work to improve the performance and accountability of the public health system, serving as both a monitoring tool and a call to action to improve health outcomes. The PSRs can be used to track the reach of and fidelity to evidence-based policies and practices nationally over time, as well as inform state efforts to improve their use of evidence-based policies and practice.

Assessing Participants' Experiences in a Service-Learning Program: A Psychometric Evaluation.

There is currently a gap in the literature regarding the creation of psychometrically sound measurement tools assessing service-learning programs in health-related fields. Without comprehension of a survey's psychometric properties, evaluators cannot ensure that survey instruments are reliable or valid. This study describes the psychometric evaluation of the Public Health Associate Program (PHAP) Service-Learning Scale (PSLS). PSLS assesses participant experience in PHAP, a Centers for Disease Control and Prevention program. This paper explains survey development, scale validity and reliability, and the internal factor structure of the PSLS. The final scale consisted of 22 items with a high internal consistency (Cronbach's α=.90). Exploratory Factor Analysis (EFA) was used to determine the scale's factor structure; five factors comprising of all 22 items were retained. The factors, or subscales, were Learning Outcomes, Mentoring, Experiential Assignment, Self-Efficacy in Program Competency Domains, and Program Satisfaction. All were also found to have adequate internal consistency (Cronbach's α >.70). Service-learning is vital in developing the next generation of the workforce. These study findings suggest the PSLS fills a critical gap in the literature by providing a valid and reliable instrument to evaluate experiences and satisfaction in service-learning programs and other fellowships.

Early Evaluation Findings From a Federally Funded Training Program: The Public Health Associate Program.

OBJECTIVE: The Centers for Disease Control and Prevention (CDC) created the Public Health Associate Program (PHAP) to establish a continuous source of public health professionals who can deliver frontline services at the federal, state, tribal, local, and territorial levels. The article describes preliminary evaluation findings for PHAP. DESIGN: The evaluation's primary purposes are to assess the quality and effectiveness of PHAP, determine its value and impact, and provide information to continuously improve the program. Because the evaluation is both formative and summative and focuses on aggregate outputs and outcomes of PHAP, the methodology is complex and builds over time as different cohorts cycle into and out of the program. Results presented are outcomes of various Web-based surveys and reporting systems. PARTICIPANTS: Four PHAP cohorts, consisting of 579 individuals, participated in 1 or more of the evaluation activities described in this article. RESULTS: The majority of participants report satisfaction with their PHAP experiences, and 74% of recent graduates indicate they are continuing their careers or education in public health immediately after program completion. Seventy-eight percent of recent PHAP graduates who accept a job in public health are employed by the federal government. One year post-PHAP, 74% of alumni report that PHAP has been influential in their careers. CONCLUSION: CDC's investment in PHAP has increased the capacity and capabilities of the public health workforce. Results presented are early indicators of program quality, effectiveness, and impact. Today's public health workers are asked to do more with less, in the face of a dynamic array of complex public health challenges. PHAP offers public health agencies assistance in tackling these losses and challenges.

Service Learning in Public Health: Exploring the Benefit to Host Agencies in CDC's Public Health Associate Program.

The "learn by doing" approach to training is common in the public health field and is a core component of service-learning programs. Trainee satisfaction, learning, and application of learning have been studied. What is less understood is the perspective of the agencies that host trainees. This study aimed to identify whether and how the Centers for Disease Control and Prevention's Public Health Associate Program (PHAP) adds value to the agencies that host trainees during 2-year field assignments. An exploratory, qualitative study design consisting of 9 semistructured telephone interviews with PHAP host agency supervisors was used. Results suggested that PHAP increased host agencies' capacity by assigning capable trainees to host agencies. Trainees made quality contributions that led to agency- and/or community-wide improvements and positively affected the agencies' culture. Further evaluation of the host perspective is necessary; as coupled with the trainee's perspective, it will provide a more holistic understanding of program value.

Federal Public Health Workforce Development: An Evidence-Based Approach for Defining Competencies.

OBJECTIVE: This study reports the use of exploratory factor analysis to describe essential skills and knowledge for an important segment of the domestic public health workforce-Centers for Disease Control and Prevention (CDC) project officers-using an evidence-based approach to competency development and validation. DESIGN: A multicomponent survey was conducted. Exploratory factor analysis was used to examine the underlying domains and relationships between competency domains and key behaviors. The Cronbach α coefficient determined the reliability of the overall scale and identified factors. SETTING AND PARTICIPANTS: All domestic (US state, tribe, local, and territorial) grantees who received funding from the CDC during fiscal year 2011 to implement nonresearch prevention or intervention programs were invited to participate in a Web-based questionnaire. MAIN OUTCOME MEASURE(S): A total of 34 key behaviors representing knowledge, skills, and abilities, grouped in 7 domains-communication, grant administration and management, public health applied science and knowledge, program planning and development, program management, program monitoring and improvement, and organizational consultation-were examined. RESULTS: There were 795 responses (58% response rate). A total of 6 factors were identified with loadings of 0.40 or more for all 34 behavioral items. The Cronbach α coefficient was 0.95 overall and ranged between 0.73 and 0.91 for the factors. CONCLUSIONS: This study provides empirical evidence for the construct validity of 6 competencies and 34 key behaviors important for CDC project officers and serves as an important first step to evidence-driven workforce development efforts in public health.

Advances in public health accreditation readiness and quality improvement: evaluation findings from the National Public Health Improvement Initiative.

INTRODUCTION: Continuous quality improvement is a central tenet of the Public Health Accreditation Board's (PHAB) national voluntary public health accreditation program. Similarly, the Centers for Disease Control and Prevention launched the National Public Health Improvement Initiative (NPHII) in 2010 with the goal of advancing accreditation readiness, performance management, and quality improvement (QI). OBJECTIVE: Evaluate the extent to which NPHII awardees have achieved program goals. DESIGN: NPHII awardees responded to an annual assessment and program monitoring data requests. Analysis included simple descriptive statistics. SETTING: Seventy-four state, tribal, local, and territorial public health agencies receiving NPHII funds. PARTICIPANTS: NPHII performance improvement managers or principal investigators. MAIN OUTCOME MEASURE(S): Development of accreditation prerequisites, completion of an organizational self-assessment against the PHAB Standards and Measures, Version 1.0, establishment of a performance management system, and implementation of QI initiatives to increase efficiency and effectiveness. RESULTS: Of the 73 responding NPHII awardees, 42.5% had a current health assessment, 26% had a current health improvement plan, and 48% had a current strategic plan in place at the end of the second program year. Approximately 26% of awardees had completed an organizational PHAB self-assessment, 72% had established at least 1 of the 4 components of a performance management system, and 90% had conducted QI activities focused on increasing efficiencies and/or effectiveness. CONCLUSIONS: NPHII appears to be supporting awardees' initial achievement of program outcomes. As NPHII enters its third year, there will be additional opportunities to advance the work of NPHII, compile and disseminate results, and inform a vision of high-quality public health necessary to improve the health of the population.

The meta-leadership summit for preparedness initiative: an innovative model to advance public health preparedness and response.

This article reports on the design, evaluation framework, and results from the Meta-Leadership Summit for Preparedness Initiative. The Meta-Leadership Summit for Preparedness was a 5-year initiative based on the premise that national preparedness and emergency response is not solely the responsibility of government. From 2006 to 2011, 36 Meta-Leadership Summits were delivered in communities across the country. Summits were customized, 10-hour leadership development, networking, and community action planning events. They included participation from targeted federal, state, local, nonprofit/philanthropic, and private sector leaders who are directly involved in decision making during a major community or state-wide emergency. A total of 4,971 government, nonprofit, and business leaders attended Meta-Leadership Summits; distribution of attendees by sector was balanced. Ninety-three percent of respondents reported the summit was a valuable use of time, 91% reported the overall quality as "good" or "outstanding," and 91% would recommend the summit to their colleagues. In addition, approximately 6 months after attending a summit, 80% of respondents reported that they had used meta-leadership concepts or principles. Of these, 93% reported that using meta-leadership concepts or principles had made a positive difference for them and their organizations. The Meta-Leadership Summit for Preparedness Initiative was a value-added opportunity for communities, providing the venue for learning the concepts and practice of meta-leadership, multisector collaboration, and resource sharing with the intent of substantively improving preparedness, response, and recovery efforts.

Evaluation of a federally funded workforce development program: the Centers for Public Health Preparedness.

The Centers for Public Health Preparedness (CPHP) program was a five-year cooperative agreement funded by the Centers for Disease Control and Prevention (CDC). The program was initiated in 2004 to strengthen terrorism and emergency preparedness by linking academic expertise to state and local health agency needs. The purposes of the evaluation study were to identify the results achieved by the Centers and inform program planning for future programs. The evaluation was summative and retrospective in its design and focused on the aggregate outcomes of the CPHP program. The evaluation results indicated progress was achieved on program goals related to development of new training products, training members of the public health workforce, and expansion of partnerships between accredited schools of public health and state and local public health departments. Evaluation results, as well as methodological insights gleaned during the planning and conduct of the CPHP evaluation, were used to inform the design of the next iteration of the CPHP Program, the Preparedness and Emergency Response Learning Centers (PERLC).

Tackling the characterization of canine chromosomal breakpoints with an integrated in-situ/in-silico approach: the canine PAR and PAB.

The domestic dog continues to represent an influential model organism for comparative biomedical research owing to the numerous genetic and pathophysiological similarities shared between human and dog diseases. The combined availability of a high-quality genome assembly and a 1 Mb-resolution genome-assembly integrated bacterial artificial chromosome (BAC) panel now provides the essential resources to combine cytogenetic and computational analyses to determine the precise locations of chromosome breakpoint regions within aberrant karyotypes. In this study we demonstrate the synergy of using a such a combined in-situ/in-silico approach to define chromosome breakpoints using the naturally occurring breakpoint present on all canine X chromosomes--the pseudoautosomal breakpoint (PAB). In so doing we have further characterized the canine pseudoautosomal region (PAR) to extend approximately 6.6 Mb from the telomeric end of CFA Xp and established that the canine PAB is contained within a 2 kb region. Our characterization of the canine PAR allowed for the comparative study of gene content across previously defined mammalian PARs and indicates that the canine PAB is contained with the gene Shroom2. The future application of the approach demonstrated in this study will prove useful when seeking to identify the genomic sequences surrounding recurrent chromosome breakpoints present in canine cancers.

A cytogenetically characterized, genome-anchored 10-Mb BAC set and CGH array for the domestic dog.

The generation of a 7.5x dog genome assembly provides exciting new opportunities to interpret tumor-associated chromosome aberrations at the biological level. We present a genomic microarray for array comparative genomic hybridization (aCGH) analysis in the dog, comprising 275 bacterial artificial chromosome (BAC) clones spaced at intervals of approximately 10 Mb. Each clone has been positioned accurately within the genome assembly and assigned to a unique chromosome location by fluorescence in situ hybridization (FISH) analysis, both individually and as chromosome-specific BAC pools. The microarray also contains clones representing the dog orthologues of 31 genes implicated in human cancers. FISH analysis of the 10-Mb BAC clone set indicated excellent coverage of each dog chromosome by the genome assembly. The order of clones was consistent with the assembly, but the cytogenetic intervals between clones were variable. We demonstrate the application of the BAC array for aCGH analysis to identify both whole and partial chromosome imbalances using a canine histiocytic sarcoma case. Using BAC clones selected from the array as probes, multicolor FISH analysis was used to further characterize these imbalances, revealing numerous structural chromosome rearrangements. We outline the value of a combined aCGH/FISH approach, together with a well-annotated dog genome assembly, in canine and comparative cancer studies.