Measuring infrastructure: A key step in program evaluation and planning.

State tobacco prevention and control programs (TCPs) require a fully functioning infrastructure to respond effectively to the Surgeon General's call for accelerating the national reduction in tobacco use. The literature describes common elements of infrastructure; however, a lack of valid and reliable measures has made it difficult for program planners to monitor relevant infrastructure indicators and address observed deficiencies, or for evaluators to determine the association among infrastructure, program efforts, and program outcomes. The Component Model of Infrastructure (CMI) is a comprehensive, evidence-based framework that facilitates TCP program planning efforts to develop and maintain their infrastructure. Measures of CMI components were needed to evaluate the model's utility and predictive capability for assessing infrastructure. This paper describes the development of CMI measures and results of a pilot test with nine state TCP managers. Pilot test findings indicate that the tool has good face validity and is clear and easy to follow. The CMI tool yields data that can enhance public health efforts in a funding-constrained environment and provides insight into program sustainability. Ultimately, the CMI measurement tool could facilitate better evaluation and program planning across public health programs.

Linkages between clinical practices and community organizations for prevention: a literature review and environmental scan.

OBJECTIVES: We conducted a literature review and environmental scan to develop a framework for interventions that utilize linkages between clinical practices and community organizations for the delivery of preventive services, and to identify and characterize these efforts. METHODS: We searched 4 major health services and social science electronic databases and conducted an Internet search to identify examples of linkage interventions in the areas of tobacco cessation, obesity, nutrition, and physical activity. RESULTS: We identified 49 interventions, of which 18 examples described their evaluation methods or reported any intervention outcomes. Few conducted evaluations that were rigorous enough to capture changes in intermediate or long-term health outcomes. Outcomes in these evaluations were primarily patient-focused and did not include organizational or linkage characteristics. CONCLUSIONS: An attractive option to increase the delivery of preventive services is to link primary care practices to community organizations; evidence is not yet conclusive, however, that such linkage interventions are effective. Findings provide recommendations to researchers and organizations that fund research, and call for a framework and metrics to study linkage interventions.

Linkages between clinical practices and community organizations for prevention: a literature review and environmental scan.

CONTEXT: A literature review and environmental scan were conducted to develop a framework for interventions that utilize linkages between clinical practices and community organizations for the delivery of preventive services, and to identify and characterize these efforts. EVIDENCE ACQUISITION: A search was made of four major health services and social science electronic databases and an Internet search was conducted to identify examples of linkage interventions in the areas of tobacco cessation, obesity, nutrition, and physical activity. EVIDENCE SYNTHESIS: In all, 49 interventions were identified, of which 18 examples described their evaluation methods or reported any intervention outcomes. Few conducted evaluations that were rigorous enough to capture changes in intermediate or long-term health outcomes. Outcomes in these evaluations were primarily patient-focused and did not include organizational or linkage characteristics. CONCLUSIONS: An attractive option to increase the delivery of preventive services is to link primary care practices to community organizations; evidence is not yet conclusive, however, that such linkage interventions are effective. Findings provide recommendations to researchers and organizations that fund research, and call for a framework and metrics to study linkage interventions.

Cancer Plan Index: a measure for assessing the quality of cancer plans.

OBJECTIVE: To (1) conduct an in-depth assessment of the content of comprehensive cancer control plans and (2) obtain data that can be used to provide guidance to grantees supported by the Centers for Disease Control and Prevention's National Comprehensive Cancer Control Program (NCCCP) as they refine their plans, and to other health professionals as similar planning is done. DESIGN: Through an iterative development process, a workgroup of subject matter experts from NCCCP and Research Triangle Institute International (RTI International) identified 11 core or essential components that should be considered in cancer plans on the basis of their professional experience and expertise. They also developed a tool, the Cancer Plan Index (CPI), to assess the extent to which cancer plans addressed the 11 core components. SETTING: Sixty-five comprehensive cancer control programs in states, tribes, territories, and jurisdictions funded by the NCCCP. DATA SOURCE: Raters reviewed and abstracted all available cancer plans (n = 66), which included plans from 62 funded programs and 4 states of the Federated States of Micronesia funded by Centers for Disease Control and Prevention as a subcontractor of one funded program. Of the 66 plans, 3 plans were used to pilot test the CPI and the remaining 63 plans were subsequently reviewed and abstracted. MAIN OUTCOME MEASURE(S): The primary outcome measures are national-level component scores for 11 defined domains (global involvement of stakeholders, developing the plan, presentation of data on disease burden, goals, objectives, strategies, reduction of cancer disparities, implementation, funds for implementation of plan, evaluation, usability of plan), which represent an average of the component scores across all available cancer plans. RESULTS: To aid in the interpretation and usability of findings, the components were segmented into 3 tiers, representing a range high (average score = 2.01-4.00), moderate (average score = 1.01-2.00), and low (average score = 0-1.00) levels of description of the component. Programs overall provided relatively comprehensive descriptions of goals, objectives, and strategies; moderate description of the plan development process, presentation of data on disease burden, and plans on the reduction of cancer disparities; and little to no description of stakeholder involvement plans for implementation, funds for implementation, and evaluation of the plan. CONCLUSIONS: Areas of the CPI with low average component scores should stimulate technical assistance to the funded programs, either to increase program activities or to increase discussion of key activities in the plan.

Piloting performance measurement for Comprehensive Cancer Control programs.

OBJECTIVE: To implement a pilot test of performance measures for National Comprehensive Cancer Control (CCC) programs funded by the Centers for Disease Control and Prevention (CDC). DESIGN: A cross-sectional assessment conducted in 2008. SETTING: A total of 65 CCC-funded entities (51 states, 7 tribes, and 7 territories or jurisdictions) representing 69 CCC programs. PARTICIPANTS: Comprehensive Cancer Control program staff. MAIN OUTCOME MEASURES: In a process that involved stakeholders from funded programs, academia, and nonprofit organizations, the CDC developed a framework for evaluation and a performance measures worksheet containing 11 performance measures for CCC programs that assessed grantee attainment of key components of CCC as required in the funding announcement. The framework was based on a CCC logic model. The performance measures worksheet contained detailed description of the measures, definitions, and suggested data sources for the 11 measures. RESULTS: Of the 69 programs, 61 completed the worksheets. The median time reported to complete the worksheet was 10 hours (interquartile range = 6-20). Almost all programs reported having representation of relevant populations in their coalition and having conducted a recent assessment of the burden of cancer. Less frequently, programs reported having a written evaluation plan or having enacted policy changes. Additional performance measures described non-CDC funding, the percentage of partners implementing CCC activities, and the percentage of implemented interventions that were evidence-based. CONCLUSIONS: This pilot test of the performance measures worksheet established the feasibility of conducting a standardized survey of CCC programs to identify issues of importance to developing and implementing the CCC program at national and program levels. The performance measures provided unique data on CCC grantees to the CDC funders and feedback on performance measures for improving questions on future surveys. Refinement of the performance measures will provide a tool for monitoring processes of action and accountability of grantees and will encourage a culture of quality improvement through systematic evaluation.