Toward a national consumer survey: evaluation of the CABHS and MHSIP instruments.

This article describes a study evaluating the Consumer Assessment of Behavioral Health Survey (CABHS) and the Mental Health Statistics Improvement Program (MHSIP) surveys. The purpose of the study was to provide data that could be used to develop recommendations for an improved instrument. Subjects were 3,443 adults in six behavioral health plans. The surveys did not differ significantly in response rate or consumer burden. Both surveys reliably assessed access to treatment and aspects of appropriateness and quality. The CABHS survey reliably assessed features of the insurance plan; the MHSIP survey reliably assessed treatment outcome. Analyses of comparable items suggested which survey items had greater validity. Results are discussed in terms of consistency with earlier research using these and other consumer surveys. Implications and recommendations for survey development, quality improvement, and national policy initiatives to evaluate health plan performance are presented.

Use of consumer ratings for quality improvement in behavioral health insurance plans.

BACKGROUND: The Consumer Assessment of Behavioral Healthcare Services (CABHS) survey collects consumers' reports about their health care plans and treatment. The use of the CABHS to identify opportunities for improvement, with specific attention to how organizations have used the survey information for quality improvement, is described. METHODS: In 1998 and 1999, data were collected from five groups of adult patients in commercial health plans and five groups of adult patients in public assistance health plans with services received through four organizations (one of three managed behavioral health care organizations or a health system). Patients who received behavioral health care services during the previous year were mailed the CABHS survey. Non-respondents were contacted by telephone to complete the survey. RESULTS: Response rates ranged from 49% to 65% for commercial patient groups and from 36% to 51% for public assistance patients. Promptly getting treatment from clinicians and aspects of care most influenced by health plan policies and operations, such as access to treatment and plan administrative services, received the least positive responses, whereas questions about communication received the most positive responses. In addition, questions about access- and plan-related aspects of quality showed the most interplan variability. Three of the organizations in this study focused quality improvement efforts on access to treatment. DISCUSSION: Surveys such as the CABHS can identify aspects of the plan and treatment that are improvement priorities. Use of these data is likely to extend beyond the behavioral health plan to consumers, purchasers, regulators, and policymakers, particularly because the National Committee for Quality Assurance is encouraging behavioral health plans to use a similar survey for accreditation purposes.

Implementing evidence-based practices in routine mental health service settings.

The authors describe the rationale for implementing evidence-based practices in routine mental health service settings. Evidence-based practices are interventions for which there is scientific evidence consistently showing that they improve client outcomes. Despite extensive evidence and agreement on effective mental health practices for persons with severe mental illness, research shows that routine mental health programs do not provide evidence-based practices to the great majority of their clients with these illnesses. The authors define the differences between evidence-based practices and related concepts, such as guidelines and algorithms. They discuss common concerns about the use of evidence-based practices, such as whether ethical values have a role in shaping such practices and how to deal with clinical situations for which no scientific evidence exists.

Quality measures for mental health care: results from a national inventory.

The National Inventory of Mental Health Quality Measures was funded by the Agency for Healthcare Research and Quality to (1) inventory process measures for assessing the quality of mental health care; (2) identify clinical, administrative, and quality domains where measures have been developed; and (3) identify areas where further research and development is needed. Among the 86 measures identified, most evaluated treatment of major mental disorders, for example, schizophrenia (24 percent) and major depression (21 percent). A small proportion focused on children (8 percent) or the elderly (9 percent). Domains of quality included treatment appropriateness (65 percent), continuity (26 percent), access (26 percent), coordination (13 percent), detection (12 percent), and prevention (6 percent). Few measures were evaluated for reliability (12 percent) or validity (3 percent). Measures imposing a lower burden were more likely to be in use (chi 2 = 4.41, p = .036). Further measures are needed to assess care for several priority clinical and demographic groups. Research should focus on measure validity, reliability, and implementation costs. In order to foster quality improvement activities and use of common measures and specifications for mental health care, the inventory of quality measures will be made available at www.challiance.org/cqaimh.

Evidence-based treatment of schizophrenia.

People with schizophrenia can be helped greatly with pharmacologic and psychosocial interventions that are known to be effective. Several interventions are now supported by research: use of medications following specific guidelines, training in illness self-management, case management based on principles of assertive community treatment, family psychoeducation, supported employment, and integrated substance abuse treatment. However, few patients actually receive these evidence-based interventions because they are not provided in routine mental health settings. Therefore, implementing effective treatments in mental health treatment programs is a critical challenge for the field. We review the six areas of evidence-based treatment of schizophrenia, as well as knowledge regarding implementation of mental health programs in routine practice settings.

A meta-analytic approach to growth curve analysis.

A meta-analytic approach to growth curve analysis is described and illustrated by applying it to the evaluation of the Arizona Pilot Project, an experimental project for financing the treatment of the severely mentally ill. In this approach to longitudinal data analysis, each individual subject for which repeated measures are obtained is initially treated as a separate case study for analysis. This approach has at least two distinct advantages. First, it does not assume a balanced design (equal numbers of repeated observations) across all subjects; to accommodate a variable number of observations for each subject, individual growth curve parameters are differentially weighted by the number of repeated measures on which they are based. Second, it does not assume homogeneity of treatment effects (equal slopes) across all subjects. Individual differences in growth curve parameters representing potentially unequal developmental rates through time are explicitly modeled. A meta-analytic approach to growth curve analysis may be the optimal analytical strategy for longitudinal studies where either (1) a balanced design is not feasible or (2) an assumption of homogeneity of treatment effects across all individuals is theoretically indefensible. In our evaluation of the Arizona Pilot Project, individual growth curve parameters were obtained for each of the 13 rationally derived subscales of the New York Functional Assessment Survey, over time, by linear regression analysis. The slopes, intercepts, and residuals obtained for each individual were then subjected to meta-analytic causal modeling. Using factor analytic models and then general linear models for the latent constructs, the growth curve parameters of all individuals were systematically related to each other via common factors and predicted based on hypothesized exogenous causal factors. The same two highly correlated common factors were found for all three growth curve parameters analyzed, a general psychological factor and a general functional factor. The factor patterns were found to be nearly identical across the separate analyses of individual intercepts, slopes, and residuals. Direct effects on the unique factors of each subscale of the New York Functional Assessment Survey were tested for each growth curve parameter by including the common factors as hierarchically prior predictors in the structural model for each of the indicator variables, thus statistically controlling for any indirect effect produced on the indicator through the common factors. The exogenous predictors modeled were theoretically specified orthogonal contrasts for Method of Payment (comparing Arizona Pilot Project treatment or "capitation" to traditional or "fee-for-service" care as a control), Treatment Administration Site (comparing various locations within treatment or control groups), Pretreatment Assessment (comparing general functional level at intake as assigned by an Outside Assessment Team), and various interactions among these main effects. The intercepts, representing the initial status of individual subjects on both the two common factors and the 13 unique factors of the subscales of the New York Functional Assessment Survey, were found to vary significantly across many of the various different treatment conditions, treatment administration sites, and pretreatment functional levels. This indicated a severe threat to the validity of the originally intended design of the Arizona Pilot Project as a randomized experiment. When the systematic variations were statistically controlled by including intercepts as hierarchically prior predictors in the structural models for slopes, recasting the experiment as a nonequivalent groups design, the effects of the intercepts on the slopes were found to be both statistically significant and substantial in magnitude. (ABSTRACT TRUNCATED)

Assessing behavioral health outcomes in outpatient programs: reliability and validity of the BASIS-32.

The Behavior and Symptom Identification Scale (BASIS-32) was developed to assess mental health outcomes among patients with severe illness treated on inpatient programs. However, its applicability and utility to those treated in outpatient programs has not been determined. The objective of this study was to assess reliability, validity, and sensitivity to change of the BASIS-32 among mental health consumers treated in outpatient programs. A total of 407 outpatients completed the BASIS-32 and the Short Form Health Status Profile (SF-36) at the beginning of a treatment episode and again 30 to 90 days later. Outpatients reported less difficulty at intake than did inpatients, and the BASIS-32 detected statistically significant changes 30 to 90 days after beginning outpatient treatment. Factor structure and construct validity were partially confirmed on this sample of outpatient consumers. Analyses of data from a wide range of facilities and samples would add to validation efforts and to further refinement of the BASIS-32.

Implementing outcome systems: lessons from a test of the BASIS-32 and the SF-36.

With increasing pressure from third-party payers to assess client outcomes, clinical programs want to know how to implement outcome systems. This article focuses on practical and logistic questions involved in implementing an outcome assessment system in ambulatory behavioral healthcare settings. Study questions addressed outcome systems in general and the use of the Behavior and Symptom Identification Scale (BASIS-32) and the Short Form Health Status Profile (SF-36) in particular. General questions focused on obtaining provider buy-in, client consent and confidentiality, data collection methods, sampling, time points, maximizing client participation, clinical utility of outcome data, and resources needed for outcome assessment. Measure-specific questions focused on client acceptability of the instruments and applicability of measures to diverse populations. The article suggests several strategies for enhancing outcome assessment efforts and concludes that there remains a need for further understanding of ways to maximize the utility and value of outcome measurement.

[Needs and utilization of mental health services in schizophrenia]. / Besoins et utilisation de services de santé mentale dans la schizophrénie.

The planning of mental health treatment for people with severe mental disorders constitutes a major public health problem and an important social challenge. This study aims to identify the needs and to describe the utilisation of 20 services or modalities of mental health treatment among a schizophrenic population living in the Estrie region of Quebec. The results confirm that the needs are great and show that the most important deficiencies are in the psychosocial and readaptation services. The implications of these results with regard to the organisation of mental health services for schizophrenics are briefly discussed.