Approaches to risk-adjusting outcome measures applied to criminal justice involvement after community service.

The ethic of fairness in program evaluation requires that measures of behavioral health agency performance be sensitive to differences in those agencies' caseload composition. The authors describe two traditional approaches to the statistical risk adjustment of outcome measures (stratification weighting and pre-post measurement) that are designed to account for differences in caseload composition and introduce a method that incorporates the strengths of both approaches. Procedures for deriving each of these measures are described in detail and demonstrated in the evaluation of a statewide system of community-based behavioral health care programs. This evaluation examines the degree to which service recipients get into trouble with the law after treatment. Three measures are recommended for inclusion in outcome-oriented "report cards," and the interpretation of each measure is discussed. Finally, the authors suggest formats for graphic and tabular presentation of the risk-adjusted evaluation for sharing findings with diverse stakeholder groups.

Probabilistic population estimation of the size and overlap of data sets based on date of birth.

Probabilistic population estimation is a statistical procedure for deriving unduplicated counts of the number of people represented in data sets that do not include unique person identifiers and the number of people shared by data sets that do not share personal identifiers. Because the procedure relies on anonymous data sets, the personal privacy of individuals and the confidentiality of medical records is protected. This paper describes the mathematics of probabilistic population estimation, and applies the procedure to an important contemporary public policy issue.

Consumer satisfaction and incarceration after treatment.

Consumer satisfaction is increasingly used to measure community mental health program performance. Understanding the relationship between consumer satisfaction and treatment outcomes is only beginning. This article adds to this understanding by reporting on an assessment of the relationship between consumer evaluation of community mental health services and incarceration after treatment in a statewide system of care. Results indicate that satisfaction with services is related to incarceration after treatment, with satisfied consumers having lower incarceration rates. These results support the use of self-reported consumer satisfaction as a measure of mental health program performance.

Utilization of local jails and general hospitals by state psychiatric center patients.

The idea that the deinstitutionalization of state psychiatric centers has resulted in increased utilization of general hospitals and correctional facilities by people with severe and persistent mental illness is widely held. This hypothesis of trans-institutionalization was tested by examining hospitalization and incarceration rates of people who had been or would be institutionalized in state psychiatric centers in 16 upstate New York counties. The results do not support the hypothesis of trans-institutionalization. Assumptions underlying the hypothesis are examined, potential explanations for the observed patterns are discussed, and areas for further research are suggested.

Age and mortality among white male problem drinkers.

AIMS: This study is designed to determine the relative risk of mortality for white male problem drinkers compared to white males in the general population, and to identify any variation in relative risk of problem drinking in three age groups (18-29, 30-49 and 50-79 years). DESIGN: The research design is prospective, using historical administrative datasets from treatment programs in conjunction with vital records datasets. PARTICIPANTS: Participants include all white men aged 18-79 treated for alcohol-related problems in community mental health substance abuse programs in Vermont during 1991. The treatment group includes 1853 service recipients; the comparison group includes 196,443 adult white male residents of Vermont. MEASUREMENT: Measurement of mortality rates for problem drinkers was based on probabilistic determination of overlap between treatment and vital record datasets. FINDINGS: Mortality for problem drinkers is greater than the general population in all three age groups. The estimated relative risk of mortality in the oldest age group was lower than the other groups, but substantially higher than found in recently published research. The estimated relative risk of mortality in the youngest age group, which has rarely been addressed in previous research, was higher than the relative risk in the middle age group. CONCLUSION: The application of a public health research model in which problem drinkers are compared to the general population has potential to inform public policy. In this case, the public health approach identified an elevated risk of mortality associated with problem drinking among older adults that had not been evident in the previous research.

A risk-adjusted measure of hospitalization rates for evaluating community mental health program performance.

The authors describe the use of risk-adjusted hospitalization rates to measure community mental health treatment outcomes. The risk adjustment involves comparing rates of hospitalization subsequent to treatment with rates of hospitalization prior to treatment. The research uses a probabilistic methodology that reliably estimates caseload overlap by comparing the distribution of the dates of birth observed in data sets to the distribution of dates of birth in the general population. Findings indicate that risk-adjusted hospitalization rates are substantially different than unadjusted rates. Half of the community programs in one state consistently achieved positive outcomes in four consecutive years; other programs had mixed results or no change.

Practice patterns and hospitalization rates.

Basic indicators of community support program treatment outcome as well as the degree to which the programs conform to practice guidelines for major depression are evaluated. Hospitalization rates subsequent to treatment are measured, and the correlation between hospitalization rates and practice patterns is determined. Data sets that describe outpatient and inpatient services over a 4-year period, but do not include common person identifiers were analyzed using probabilistic population estimates. Results indicate that there is substantial variation among the community programs in practice patterns and hospitalization rates, and the two are negatively correlated.

Personal privacy versus public accountability: a technological solution to an ethical dilemma.

The tension between personal privacy and public accountability produces one of the major ethical dilemmas facing behavioral health program evaluators and service system researchers. This article discusses the source of this tension and introduces a research methodology that allows program evaluators to fully and equally respect both ethical principles. This methodology uses contemporary computer and statistical technology in conjunction with aggregated, de-identified information derived from existing databases to provide valid and reliable measures of the performance of treatment programs while it protects the personal privacy of individuals.

Using incarceration rates to measure mental health program performance.

This article introduces a more refined conceptualization of the criminal justice involvement of clients of mental health programs than is evident in the existing literature, and demonstrates a research methodology that provides program administrators with standardized measures of program performance in this area. The conceptualization of the criminal justice involvement of people served by community mental health programs distinguishes between three distinct areas of concern: (1) program accessibility to people with a history of criminal justice involvement, (2) criminal justice outcomes, and (3) quality of program performance. The methodology makes use of existing data resources to provide a valid and reliable measure of program performance in these three areas.

The use of state and general hospitals for inpatient psychiatric care.

OBJECTIVES: This paper explores the relationship of state hospital and general hospital psychiatric caseloads in a statewide system of care. METHODS: Probabilistic population estimation was applied to general hospital and state hospital data sets. RESULTS: General hospitals provide inpatient psychiatric services to more people than do state hospitals, and a significant number are served in both sectors. There were notable differences in use patterns related to patient gender and age. CONCLUSIONS: These results demonstrate that probabilistic methodologies can significantly enhance the value of existing databases for epidemiological research.

A global measure of access to mental health services for a managed care environment.

The growing influence of managed care in mental health services has raised important questions about access to services. This article introduces and demonstrates a global measure of access that is based on the relationship between service utilization and the need for services. This measure has become practical because of recent advances in measurement technology that provide more valid and reliable estimates of the prevalence of mental illness in general populations and of the number of people who receive mental health services across service sectors. The methodology is used to produce a report card type profile of access to inpatient mental health services (in state, general, private, and veterans hospitals) by residents of one state. This global measure can provide a powerful and efficient tool for monitoring and comparing the impact of managed care plans on access to mental health services.

An examination of variation in long-term community tenure after psychiatric hospitalization in eight states.

The influence of seven personal and clinical characteristics on the community tenure of discharged state hospital patients in eight states is analyzed and discussed. Previous admission to a state hospital and a diagnosis of schizophrenia were associated with shorter community tenure. Older patients had longer community tenure than younger. These findings were consistent across the states and are consistent with previous work that focused on much narrower populations. The relationship of these risk factors to community tenure are discussed from an individualistic perspective, a treatment oriented perspective, and a community integrationist perspective, and implications for further research are explored.

The mental health care reform debate: a content analysis of position papers.

Traditional and ethnographic content analyses of 23 position papers on mental health care reform issued during 1993 yield a comprehensive overview of the specific subjects and broad policy issues that were of concern at that time. Five concrete aspects of mental health service delivery systems and eight policy issues were identified as most important in position papers by service provider organizations and advocacy groups. These analyses identified overall similarities in the content of the position papers, although interesting differences among organizations emerged. The analyses also identified a measure of the quality of policy discussions that is derived from comparing the frequency of concrete references to aspects of mental health care with references to broader policy issues.

A longitudinal study of the impact of changing public policy on community mental health client residential patterns and staff attitudes.

Three studies of perceived residential needs and community residential patterns of adults with severe and persistent mental illness over a period of 12 years are used to assess the effect of changing public policy in this area. During a period in which public policy shifted from advocacy of congregate living in treatment settings to independent living in generic community housing, there were significant changes in community residential patterns and in the attitudes of case managers. Clinical and demographic data collected in the most recent survey provides a much more complete profile of the community living situation of adults with severe and persistent mental illness than was previously available.

Development, implementation, and findings of a continuing client satisfaction survey.

Development of a client-satisfaction survey is described and details are given for sampling clients and iterating survey mailings on a monthly basis. Consent rates, response rates, confidentiality concerns, and costs are discussed on the basis of one year of data collection experience. Analyses revealed a general satisfaction factor and problem severity factor for both active and terminated clients. A factor of improvement was found for active clients. The utility of continuous monitoring of client satisfaction is discussed.