Toward a Culture of Consequences: Performance-Based Accountability Systems for Public Services.

Performance-based accountability systems (PBASs), which link incentives to measured performance as a means of improving services to the public, have gained popularity. While PBASs can vary widely across sectors, they share three main components: goals, incentives, and measures. Research suggests that PBASs influence provider behaviors, but little is known about PBAS effectiveness at achieving performance goals or about government and agency experiences. This study examines nine PBASs that are drawn from five sectors: child care, education, health care, public health emergency preparedness, and transportation. In the right circumstances, a PBAS can be an effective strategy for improving service delivery. Optimum circumstances include having a widely shared goal, unambiguous observable measures, meaningful incentives for those with control over the relevant inputs and processes, few competing interests, and adequate resources to design, implement, and operate the PBAS. However, these conditions are rarely fully realized, so it is difficult to design and implement PBASs that are uniformly effective. PBASs represent a promising policy option for improving the quality of service-delivery activities in many contexts. The evidence supports continued experimentation with and adoption of this approach in appropriate circumstances. Even so, PBAS design and its prospects for success depend on the context in which it will operate. Also, ongoing system evaluation and monitoring are integral components of a PBAS; they inform refinements that improve system functioning over time. Empirical evidence of the effects of performance-based public management is scarce. This article also describes a framework used to evaluate a PBAS. Such a system identifies individuals or organizations that must change their behavior for the performance of an activity to improve, chooses an implicit or explicit incentive structure to motivate these organizations or individuals to change, and then chooses performance measures tailored to inform the incentive structure appropriately. The study focused on systems in the child care, education, health care, public health emergency preparedness, and transportation sectors, mainly in the United States. Analysts could use this framework to seek empirical information in other sectors and other parts of the world. Additional empirical information could help refine existing PBASs and, more broadly, improve decisions on where to initiate new PBASs, how to implement them, and then how to design, manage, and refine them over time.

The effect of legal and hospital policies on physician response to prenatal substance exposure.

OBJECTIVES: To determine the influence of a state's legal environment and a hospital's Prenatal Substance Exposure (PSE) protocol on physicians' propensity to respond when prenatal substance exposure is suspected. METHODS: Using a sample of 1367 physicians from every state and the District of Columbia, we formulate a set of linear models to determine the impact of the legal environment and hospital protocol on physicians' response to PSE, the agreement between physicians' perceptions and actual state legal environments, and physicians' motivation to act when PSE is suspected. RESULTS: Both protocol and legal environment showed to be significantly correlated with physicians' propensity to take action when PSE is suspected (p < 0.05). Our analysis shows that physicians prefer a public health (patient-centered) approach to more punitive measures. CONCLUSIONS: Our results suggest a policy strategy focused first on enacting laws that would encourage a patient-centered approach, by developing and using hospital protocols to implement state policy, and then on educating physicians about the actual legal environment.

Evaluating the Lions-Quest "Skills for Adolescence" drug education program. Second-year behavior outcomes.

Thirty-four schools (n=7426 consented sixth graders, 71% of the eligible population) were randomized to conditions to test the hypothesis that Skills for Adolescence (SFA), a widely used comprehensive life skills training curriculum with a dedicated drug education unit, is more effective than standard care in deterring and delaying substance use through middle school. Two-year posttest (1-year post-intervention) data were collected from 5691 eighth graders (77% of those who completed the sixth-grade survey and 87% of those who completed the seventh-grade survey). Lifetime and recent (last 30 days) use of five substances or combinations of substances was compared using mixed-model regression to control for school clustering. There were two significant treatment main effects at the end of the eighth grade: lifetime (P=.05) and recent (P<.03) marijuana use were lower in SFA than control schools with pretest usage and salient demographic and psychosocial variables controlled. There was also one significant Treatment x Pretest Usage interaction around binge drinking. Baseline binge drinkers in SFA schools were less likely to report recent binge drinking than students in control schools (P<.01); there were no treatment differences among baseline nonbinge drinkers. Analyses of potential mediators of SFA treatment effects on eighth-grade binge drinking and marijuana use suggested that SFA increased self-efficacy around drug refusal skills, but did not affect behavioral intentions, perceptions of harm, or perceived peer norms. These 2-year (1-year post-intervention) outcomes offer some additional support for SFA effectiveness and the general thrust of school-based, life skills-based prevention programs. The promising sixth- through eighth-grade findings for SFA, a commercially available program, provide a further step in bridging a major gap in the "research to practice" literature: theory-based interventions that have documented behavioral effects have not enjoyed large-scale implementation, while intuition-based programs that have no documented effects still enjoy wide exposure.

Cryopreserved embryos in the United States and their availability for research.

OBJECTIVE: To determine the number of embryos stored at assisted reproductive technology (ART) clinics in the United States and their current disposition. DESIGN: A targeted survey instrument sent by the SART-RAND team to all medical practices providing in vitro fertilization services in the United States. RESULTS: The SART-RAND team surveyed all 430 ART practices in the United States. Of these practices, 340 returned surveys for analysis. The data from these surveys were merged with data taken from the 1999 SART dataset, which contains information about practice size and success rates. Responding clinics reported a total of 396,526 embryos in storage as of April 11, 2002. The vast majority of the embryos (88.2%) were targeted for patient use. Small numbers of embryos were available for research, donation, destruction, quality assurance, or other uses. CONCLUSIONS: Nearly 400,000 embryos are stored in the United States, the majority of which (88.2%) are targeted for patient use. Few are available for research (2.8%), limiting possible conversion into embryonic stem cell lines.

A search for guidance: examining prenatal substance exposure protocols.

OBJECTIVES: To describe key elements of a set of hospital prenatal substance exposure protocols, and to relate variations in protocol content to the state legislative environment and hospital characteristics. METHODS: Nurse managers and hospital administrators with responsibility for perinatal care were asked to provide their hospital's prenatal substance exposure protocol. Using a structured coding form, two independent coders read and abstracted information from the 87 protocols received. Hospital and patient characteristics and the state's legal environment were cross-tabulated. RESULTS: Only half of coded protocols included an implementation date; 37% lacked any goal or statement of purpose. Most covered the key components of prenatal substance exposure management, such as precipitants and guidelines for toxicology screening, but failed to present their contents clearly. Only a few discussed whether specific maternal consent is required for a maternal or a newborn toxicology screen. Protocols from states that had made some legislative response to prenatal substance exposure were more likely to provide reporting guidelines and a discussion of consent for a toxicology screen for mothers and newborns. Protocols were more likely to be found in larger hospitals and were more detailed in hospitals serving more affluent and less minority patient populations. CONCLUSIONS: More attention needs to be devoted to the development of prenatal substance exposure protocols, as their lack of clarity precludes most from meeting protocol development goals, such as encouraging standardized care. Associations between hospital characteristics, state legislative environment and protocol features suggest that legislative mandates could shape their development and features.

Evaluating the Lions-Quest "Skills for Adolescence" drug education program: first-year behavior outcomes.

Thirty-four schools (n=7426 consented sixth graders, 71% of the eligible population) were randomized to conditions to test the hypothesis that "Skills for Adolescence" (SFA) is more effective than standard care in deterring and delaying substance use through middle school. One-year posttest data were collected from 6239 seventh graders (84% of those eligible). Initiation of "ever" and "recent" use of five substances for baseline nonusers and changes in recent use for baseline users by experimental condition were compared using mixed model regression to control for school clustering. For pretest nonusers, recent cigarette smoking was lower for SFA than controls (P<.05), as was lifetime marijuana use (P<.06). There were also three Treatment x Ethnicity interactions around drinking behaviors. Hispanics in SFA were less likely to ever and recently drink, and to recently binge drink than Hispanic controls; there were no treatment differences among non-Hispanics. For baseline users, there were three significant SFA delays in transition to experimental or recent use of more "advanced" substances: drinking to smoking, drinking to marijuana use, and binge drinking to marijuana.