Measuring mental health outcomes with pre-post designs.

The pre-post design has been the workhorse of outcome evaluations for many years. Using data from a study of 984 treated children (ages 5 to 17 years), this article argues that there are two structural problems with the pre-post evaluation of outcome: (1) excessively large intervals of uncertainty for individual outcomes and (2) paradoxical inconsistencies in the evaluation of groups. These problems can be solved by designs with three or more repeated measures analyzed with longitudinal multilevel analytic models.

Looking for the disorder in conduct disorder.

Besides their well-known externalizing behavior, children with conduct disorder (CD) often have additional impairments outside the criteria for the CD diagnosis. In a 5-year study of 984 treated children (ages 5-17 years), those with CD had an average of 2.2 primary diagnoses. Children with CD showed the worst problem and impairment scores in comparison with 11 common diagnoses. Compared with other treated children, children with CD achieved worse scores on 14 of 15 syndromes, including internalizing problems such as withdrawal and major depression. The average child with CD had larger relapse scores in the 1.5- to 3-year period after admission to treatment. This pattern, pervasive at intake and chronic in course, resembles a global disability more than a circumscribed problem managed with a narrow range of treatments specific to it.

The co-occurrence of psychiatric and substance use diagnoses in adolescents in different service systems: frequency, recognition, cost, and outcomes.

The frequency, severity, recognition, cost, and outcomes of adolescent substance use comorbidity were analyzed in the Fort Bragg Demonstration Project. Comorbidity was defined as the co-occurrence of substance use disorder (SUD) with other psychiatric diagnosis. The sample consisted of 428 adolescent clients whose providers' diagnoses were compared with research diagnoses. The project identified 59 clients (13.8%) with SUD, all with additional psychiatric diagnoses. Providers recognized only 21 of these 59 comorbid cases. The frequency and severity of comorbidity did not differ between service system samples, although recognition did. Comorbid clients had more behavior problems and more functioning impairment, and their average treatment cost ($29,057) was more than twice as high as that of noncomorbid clients ($13,067). Mental health outcomes were not influenced by type of service system, comorbid diagnosis, or treatment. Screening for and prevention of SUD are discussed as a potential cost-savings opportunity in mental health services.

The Fort Bragg continuum of care for children and adolescents: mental health outcomes over 5 years.

Controversial early results of the Fort Bragg mental-health-effectiveness study indicated that the continuum of care did not produce better outcomes (i.e., children's rate of improvement was the same in both the demonstration and comparison sites). The present study considered outcomes at 5-year follow-up to examine long-term effects from the continuum of care. A random regression longitudinal model analyzed 10 key outcome variables measured 7 times. Long-term outcomes in continuum-treated children were no better than those of comparison children; results are consistent with those of earlier studies.

Dose effect in child psychotherapy: outcomes associated with negligible treatment.

OBJECTIVE: To compare the outcomes of children who received negligible amounts of outpatient treatment to children receiving more treatment. METHOD: A random regression longitudinal model was used to analyze outcomes of children (aged 5-17 years) from the Fort Bragg Evaluation Project. RESULTS: In examining several outcomes, the results show no statistically significant dose effect. CONCLUSIONS: Children receiving substantial amounts of treatment showed no better mental health outcomes than those receiving negligible amounts of treatment. The results do not support the existence of a dose effect consistent enough to guide clinicians, administrators, or policymakers.

Dose-effect relationship in children's psychotherapy services.

Concerns about session limits imposed by behavioral health care organizations (BHOs) have prompted an examination of the dose-effect relationship in psychotherapy. This study of the dose-effect relationship in child psychotherapy used multiple outcome measures and standardized data-collection points and addressed the confound between dose, effect, and initial severity. Data obtained from 567 outpatient children who participated in the Fort Bragg Evaluation Project were used. Probit and longitudinal growth curve analyses found no evidence of a general dose-effect relationship. Implications of these findings are discussed, especially the need for BHOs to justify use limits and suggestions for clinicians to justify use requests.

Rejoinder to Mordock's critique of the Fort Bragg Evaluation Project: the sample is generalizable and the outcomes are clear.

The Fort Bragg Evaluation Project (FBEP) showed that children in a well implemented and expensive continuum of care had no better clinical outcomes than those experiencing more traditional and fragmented services. In an article published in this journal that was critical of the evaluation, Mordock argued that the FBEP results be viewed with skepticism because of what he perceived to be methodological, design, measurement, and analytic failures of this study. We think it is important to respond to Mordock's critique since it contributes to the great reluctance to seriously consider the study's findings and their implications.

Children's mental health in a continuum of care: clinical outcomes at 18 months for the Fort Bragg demonstration.

This article uses data collected at 18 months from the evaluation of the continuum of mental health services of the Fort Bragg Child and Adolescent Mental Health Demonstration Project (the Demonstration) to address the hypotheses that longer term follow-up (beyond 12 months) will show that the Demonstration is more effective and more successful for children with serious emotional disturbance (SED). The effects of the Demonstration are examined in comparison to those of traditional care by analyzing 12 key mental health outcomes with a random regression model, and the potential impact of attrition on results is explored. Results show neither hypothesis is supported, and the attrition analysis showed that the influence of missing data on the outcome analyses is negligible. Implications of these results for mental health policy are discussed.

Ability of the Cognitive Behavioral Driver's Inventory to distinguish malingerers from brain-damaged subjects.

The Cognitive Behavioral Driver's Inventory (CBDI) was analyzed for its ability to discriminate brain-damaged patients from intact subjects who feigned brain-damage. In a sample of 251 neurologically impaired patients and 48 malingering volunteers, the computer-administered distinguished most malingerers from genuine patients. A jackknifed count revealed that the CBDI had 90% sensitivity for detecting malingerers, and 98% specificity for detecting non-malingering brain damaged patients. Success was due to the inability of malingerers to avoid quantitative errors: excessive response latencies, unusual error rates, inflated variability in response latencies, and excessive within-subject, between-item variability. The computer-administered battery may be an effective clinical tool for identifying patients who malinger brain-damage in neuropsychological testing.

Who gets hospitalized in a continuum of care?

OBJECTIVE: To compare children and adolescents hospitalized under a continuum of care with those hospitalized under traditional insurance coverage. METHOD: With comprehensive data, logistic regressions were used to predict hospitalization and to identify its determinants. RESULTS: As expected, the probability of being hospitalized was much higher under traditional care. In addition, the predictors of hospitalization differed by site. Accuracy of predictions was high. CONCLUSIONS: Different kinds of children were hospitalized under a continuum of care than under a traditional insurance system. Hospitalizations under both systems were highly predictable.

Clinical outcomes of a children's mental health managed care demonstration.

The Fort Bragg Evaluation Project hypothesized that the mental health of children treated in the Demonstration's continuum of care would improve more than that of children receiving traditional mental health services at a comparison site. Program theory further predicted site differences in outcome for certain subgroups of children, such as those with severe mental health problems. These hypotheses were tested at 6-month and 1-year follow-ups in several ways, but results showed only slightly more site differences than expected by chance. For the evaluation sample of N = 984 treated children aged 5-17, site differences favored the Comparison about as often as the Demonstration. Children at both sites improved, but there was no overall superiority in mental health outcomes at the Demonstration.

Social skills training with hospitalized adolescents: an evaluative experiment.

The effectiveness of social skills training on adolescent males admitted to a psychiatric hospital was evaluated through the use of empirically derived anger scenarios, blind raters, matched experimental and control treatment groups, and a repeated measures analysis. Results indicated that the social skills training effectively improved the dimension of verbal response and eye contact. Facial expression did not show measurable improvement. The use of specific skills components and scoring criteria in the social skills training was emphasized. Overall, support was found for the efficacy of social skills training for hospitalized adolescents.

Predicting recidivism among first admissions at Tennessee's state psychiatric hospitals.

Using data from computerized patient records, the authors attempted to identify characteristics of first admissions to state psychiatric facilities who would later become recidivists. In an examination of 22,062 first admissions to all state hospitals in Tennessee, they found six variables with significant ability to predict recidivism: age, delusional beliefs, assaultive acts, out-of-state residence, indigence, and living with parents. A risk profile that predicted future recidivism with statistical significance at all five Tennessee state hospitals was subsequently developed, but the predictive accuracy was too low for the profile to be clinically useful. The authors believe the benefits of early identification of patients at risk justify further research.