Response to Disclosure as an Indicator of Competence to Stand Trial.

Ethics guidelines recommend that forensic mental health professionals begin in-person assessments by explaining the nature and purpose of the examination. To learn whether evaluees have understood and can give consent, forensic practitioners may ask evaluees to paraphrase the explanation. This article explores how a forensic evaluee's disclosure response (DR) reveals substantive information relevant to the purposes of a forensic examination. We examined archival data from 255 reports on competence to stand trial (CST) that a Midwest public sector hospital had previously submitted to courts. We classified each evaluee's DR at one of three levels: DR = yes (accurate paraphrasing), DR = no (inability to paraphrase or provide a relevant response), or DR = other (an intermediate level implying a less-than-accurate response). None of the 28 DR = no evaluees was CST, and only 7 (17%) of the 48 DR = other evaluees were CST. Thus, a CST evaluee who cannot paraphrase an examiner's explanation is likely to be incompetent to stand trial, and an examiner would need to adduce a strong argument to support any opinion to the contrary.

Trial 1 versus Trial 2 of the Test of Memory Malingering: Evaluating accuracy without a "gold standard".

This study examines the accuracy of the Test of Memory Malingering (TOMM), a frequently administered measure for evaluating effort during neurocognitive testing. In the last few years, several authors have suggested that the initial recognition trial of the TOMM (Trial 1) might be a more useful index for detecting feigned or exaggerated impairment than Trial 2, which is the source for inference recommended by the original instruction manual (Tombaugh, 1996). We used latent class modeling (LCM) implemented in a Bayesian framework to evaluate archival Trial 1 and Trial 2 data collected from 1,198 adults who had undergone outpatient forensic evaluations. All subjects were tested with 2 other performance validity tests (the Word Memory Test and the Computerized Assessment of Response Bias), and for 70% of the subjects, data from the California Verbal Learning Test-Second Edition Forced Choice trial were also available. Our results suggest that not even a perfect score on Trial 1 or Trial 2 justifies saying that an evaluee is definitely responding genuinely, although such scores imply a lower-than-base-rate probability of feigning. If one uses a Trial 2 cut-off higher than the manual's recommendation, Trial 2 does better than Trial 1 at identifying individuals who are almost certainly feigning while maintaining a negligible false positive rate. Using scores from both trials, one can identify a group of definitely feigning and very likely feigning subjects who comprise about 2 thirds of all feigners; only 1% of the members of this group would not be feigning. (PsycINFO Database Record

Using Dual Beta Distributions to Create "Proper" ROC Curves Based on Rating Category Data.

BACKGROUND: Receiver operating characteristic (ROC) analysis helps investigators quantify and describe how well a diagnostic system discriminates between 2 mutually exclusive conditions. The conventional binormal (CvB) curve-fitting model usually produces ROCs that are improper in that they do not have the ever-decreasing slope required by signal detection theory. When data sets evaluated under the CvB model have hooks, the resulting ROCs can contain misleading information about the diagnostic performance of the method at low and high false positive rates. OBJECTIVE: To present and evaluate a dual beta (DB) ROC model that assumes diagnostic data arise from 2 ß distributions. The DB model's parameter constraints assure that the resulting ROC curve has a positive, monotonically decreasing slope. DESIGN/METHOD: Computer simulation study comparing results from CvB, DB, and weighted power function (WPF) models. RESULTS: The DB model produces results that are as good as or better than those from the WPF model, and less biased and closer to the true values than curves obtained using the CvB model. CONCLUSIONS: The DB ROC model expresses the relationship between the false positive rate and true positive rate in closed form and allows for quick ROC area calculations using spreadsheet functions. Because it posits simple relationships among the decision axis, operating points, and model parameters, the DB model offers investigators a flexible, easy-to-grasp ROC form that is simpler to implement than other proper ROC models.

A Bayesian approach to mixed group validation of performance validity tests.

Mental health professionals often use structured assessment tools to help detect individuals who are feigning or exaggerating symptoms. Yet estimating the accuracy of these tools is problematic because no "gold standard" establishes whether someone is malingering or not. Several investigators have recommended using mixed group validation (MGV) to estimate the accuracy of malingering measures, but simulation studies show that typical implementations of MGV may yield vague, biased, or logically impossible results. In this article we describe a Bayesian approach to MGV that addresses and avoids these limitations. After explaining the concepts that underlie our approach, we use previously published data on the Test of Memory Malingering (TOMM; Tombaugh, 1996) to illustrate how our method works. Our findings concerning the TOMM's accuracy, which include insights about covariates such as study population and litigation status, are consistent with results that appear in previous publications. Unlike most investigations of the TOMM's accuracy, our findings neither rely on possibly flawed assumptions about subjects' intentions nor assume that experimental simulators can duplicate the behavior of real-world examinees. Our conceptual approach may prove helpful in evaluating the accuracy of many assessment tools used in clinical contexts and psycholegal determinations.

From group data to useful probabilities: the relevance of actuarial risk assessment in individual instances.

Probability plays a ubiquitous role in decision-making through a process in which we use data from groups of past outcomes to make inferences about new situations. Yet in recent years, many forensic mental health professionals have become persuaded that overly wide confidence intervals render actuarial risk assessment instruments virtually useless in individual assessments. If this were true, the mathematical properties of probabilistic judgments would preclude forensic clinicians from applying group-based findings about risk to individuals. As a consequence, actuarially based risk estimates might be barred from use in legal proceedings. Using a fictional scenario, I seek to show how group data have an obvious application to individual decisions. I also explain how misunderstanding the aims of risk assessment has led to mistakes about how, when, and why group data apply to individual instances. Although actuarially based statements about individuals' risk have many pitfalls, confidence intervals pose no barrier to using actuarial tools derived from group data to improve decision-making about individual instances.

Correlations of inflammatory gene pathways, corticolimbic functional activities, and aggression in pediatric bipolar disorder: a preliminary study.

The mechanisms underlying aggression in adolescents with bipolar disorder have been poorly understood. The present study has investigated the associations among TNF gene expressions, functional brain activations under the frustrative non-reward task, and aggression in adolescents with bipolar disorder. Baseline gene expressions and aggressive tendencies were measured with the RNA-sequencing and Brief Rating of Aggression by Children and Adolescents (BRACHA), respectively. Our results show that activity levels of left subgenual anterior cingulate gyrus (ACG), right amygdala, left Brodmann area 10 (orbitofrontal cortex), and right thalamus were inversely correlated with BRACHA scores and were activated with frustrative non-reward during the affective Posner Task. In addition, 11 TNF related gene expressions were significantly correlated with activation of amygdala or ACG during the affective Posner Task. Three TNF gene expressions were inversely correlated with BRACHA score while one TNF gene (TNFAIP3) expression was positively correlated with BRACHA score. Therefore, TNF-related inflammatory cytokine genes may play a role in neural activity associated with frustrative non-reward and aggressive behaviors in pediatric bipolar disorder.

Constructing "proper" ROCs from ordinal response data using weighted power functions.

BACKGROUND: Receiver operating characteristic (ROC) analysis is the standard method for describing the accuracy of diagnostic systems where the decision task involves distinguishing between 2 mutually exclusive possibilities. The popular binormal curve-fitting model usually produces ROCs that are improper in that they do not have the ever-decreasing slope required by signal detection theory. Not infrequently, binormal ROCs have visible hooks that falsely imply worse-than-chance diagnostic differentiation where the curve lies below the no-information diagonal. In this article, we present and evaluate a 2-parameter, weighted power function (WPF) model that always results in a proper ROC curve with a positive, monotonically decreasing slope. METHODS: We used a computer simulation study to compare results from binormal and WPF models. RESULTS: The WPF model produces ROC curves that are less biased and closer to the true values than are curves obtained using the binormal model. The better performance of the WPF model follows from its design constraint as a necessarily proper ROC. CONCLUSIONS: The WPF model fits a broader variety of data sets than previously published power function models while maintaining straightforward relationships among the original decision variable, specific operating points, ROC curve contours, and model parameters. Compared with other proper ROC models, the WPF model is distinctive in its simplicity, and it avoids the flaws of the conventional binormal ROC model.

The association between salivary hormone levels and children's inpatient aggression: a pilot study.

Aggression is a common management problem for child psychiatry hospital units. We describe an exploratory study with the primary objective of establishing the feasibility of linking salivary concentrations of three hormones (testosterone, dehydroepiandrosterone [DHEA], and cortisol) with aggression. Between May 2011 and November 2011, we recruited 17 psychiatrically hospitalized boys (age 7-9 years). We administered the Brief Rating of Aggression by Children and Adolescents (BRACHA) and Predatory-Affective Aggression Scale (PAAS) upon admission. Saliva samples were collected from the participants during a 24-h period shortly after admission: immediately upon awakening, 30 min later, and again between 3:45 and 7:45 P.M. Nursing staff recorded Overt Aggression Scale ratings twice a day during hospitalization to quantify aggressive behavior. The salivary cortisol concentrations obtained from aggressive boys 30 min after awakening trended higher than levels from the non-aggressive boys (p = 0.06), were correlated with the number of aggressive incidents (p = 0.04), and trended toward correlation with BRACHA scores (p = 0.06). The aggressive boys also showed greater morning-to-evening declines in cortisol levels (p = 0.05). Awakening levels of DHEA and testosterone were correlated with the severity of the nearest aggressive incident (p < 0.05 for both). The BRACHA scores of the aggressive boys were significantly higher than scores of the non-aggressive boys (p < 0.001). Our data demonstrate the feasibility of collecting saliva from children on an inpatient psychiatric unit, affirm the utility of the BRACHA in predicting aggressive behavior, and suggest links between salivary hormones and aggression by children who undergo psychiatric hospitalization.

Evaluating risk assessments using receiver operating characteristic analysis: rationale, advantages, insights, and limitations.

The last two decades have witnessed major changes in the way that mental health professionals assess, describe, and think about persons' risk for future violence. Psychiatrists and psychologists have gone from believing that they could not predict violence to feeling certain they can assess violence risk with well-above-chance accuracy. Receiver operating characteristic (ROC) analysis has played a central role in changing this view. This article reviews the key concepts underlying ROC methods, the meaning of the area under the ROC curve (AUC), the relationship between AUC and effect size d, and what these two indices tell us about evaluations of violence risk. The area under the ROC curve and d provide succinct but incomplete descriptions of discrimination capacity. These indices do not provide details about sensitivity-specificity trade-offs; they do not tell us how to balance false-positive and false-negative errors; and they do not determine whether a diagnostic system is accurate enough to make practically useful distinctions between violent and non-violent subject groups. Justifying choices or clinical practices requires a contextual investigation of outcomes, a process that takes us beyond simply knowing global indices of accuracy.

Brief Rating of Aggression by Children and Adolescents (BRACHA): a reliability study.

The Brief Rating of Aggression by Children and Adolescents (BRACHA) is a 14-item instrument scored by emergency room staff members to assess aggression risk during an upcoming psychiatric hospitalization. In this study, we investigated the inter-rater reliability of the BRACHA 0.9, the latest version of the instrument. After receiving training based on the BRACHA 0.9 manual, 10 intake workers viewed 24 ten-minute videos in which child and adolescent actors portrayed pediatric emergency room patients with low, moderate, or high levels of risk for aggression during an upcoming hospitalization. We then evaluated inter-rater reliability for individual BRACHA items, using three measures of agreement, and reliability for total BRACHA 0.9 scores, using conventional (frequentist) methods and Bayesian techniques for calculating the intraclass correlation coefficient ICC (2,1). Inter-rater reliability for individual items ranged from good to almost perfect, with Kendall's W exceeding 0.75 for eight of 14 BRACHA items. The ICC (2,1) for the total BRACHA 0.9 score was 0.9099, with both conventional and Bayesian methods (95% credible interval 0.8530-0.9533), suggesting an excellent level of overall agreement. The BRACHA appears to be an accurate, highly reliable instrument for assessing the risk of aggression by children and adolescents who are about to undergo psychiatric hospitalization.

Estimating the accuracy of neurocognitive effort measures in the absence of a "gold standard".

Psychologists frequently use symptom validity tests (SVTs) to help determine whether evaluees' test performance or reported symptoms accurately represent their true functioning and capability. Most studies evaluating the accuracy of SVTs have used either known-group comparisons or simulation designs, but these approaches have well-known limitations (potential misclassifications or lack of ecological validity). This study uses latent class modeling (LCM) implemented in a Bayesian framework to estimate SVT classification accuracy based on data obtained from real-life forensic evaluations. We obtained archival data from 1,301 outpatient evaluees who underwent testing with the Computerized Assessment of Response Bias (CARB), the Test of Memory Malingering (TOMM), and the Word Memory Test (WMT) in a forensic evaluation context. Under various data models, Markov chain Monte Carlo methods implemented via WinBUGS converged to target distributions that permitted Bayesian estimates of SVT accuracy. Under the most plausible model (conditional dependence in test results), classification accuracies (expressed as area under the "trapezoidal" receiver operating characteristic curve ± standard deviation) were as follows: CARB = 0.765 ± 0.030, WMT = 0.929 ± 0.020, and TOMM = 0.771 ± 0.034. At decision thresholds that hold false positive rates at 0.02, the SVTs would detect invalid responses (true positives) at rates of approximately 35%, 65%, and 49%, respectively, for the 3 tests. Though LCM methods have limitations, this study suggests that they offer an approach to SVT evaluation that avoids methodological pitfalls of known-group research designs while retaining ecological validity that is absent in simulation studies.

Brief Rating of Aggression by Children and Adolescents (BRACHA): development of a tool for assessing risk of inpatients' aggressive behavior.

This study evaluated the Brief Rating of Aggression by Children and Adolescents-Preliminary Version (BRACHA 0.8), an actuarial method of assessing the risk of aggressive behavior by hospitalized children and adolescents. Licensed psychiatric social workers used a 16-item questionnaire to assess all patients seen in the emergency department (ED) of a major urban children's hospital. Over a six-month period, 418 patients (age range, 3.5-19.0 years) underwent psychiatric hospitalization after ED evaluation. The hospital nursing staff recorded the inpatients' behavior, with the Overt Aggression Scale (OAS). Inpatients were deemed aggressive if, during the first six days of their hospital stay, they scored one or higher on any OAS subscale. We evaluated questionnaire properties, items, and demographic covariates (e.g., age, sex, and living situation) by using factor analyses, logistic regression models, and receiver operating characteristic (ROC) methods. A total of 292 aggressive acts were committed by 120 (29% of 418) patients. Fourteen of the 16 items predicted (p < .007) inpatient aggression and showed good internal consistency (Cronbach's α = 0.837). Age was inversely related to probability of aggression and was incorporated into the final assessment instrument. Predictive power was comparable with other published risk assessment instruments (ROC areas of .75 for any aggression and .82 for aggression toward others). BRACHA 0.8 shows promise in rapidly assessing risk of inpatient aggression, but further studies are needed to establish the reliability and validity of the instrument.

Quantifying the accuracy of forensic examiners in the absence of a "gold standard".

This study asked whether latent class modeling methods and multiple ratings of the same cases might permit quantification of the accuracy of forensic assessments. Five evaluators examined 156 redacted court reports concerning criminal defendants who had undergone hospitalization for evaluation or restoration of their adjudicative competence. Evaluators rated each defendant's Dusky-defined competence to stand trial on a five-point scale as well as each defendant's understanding of, appreciation of, and reasoning about criminal proceedings. Having multiple ratings per defendant made it possible to estimate accuracy parameters using maximum likelihood and Bayesian approaches, despite the absence of any "gold standard" for the defendants' true competence status. Evaluators appeared to be very accurate, though this finding should be viewed with caution.

Let's think about human factors, not human failings.

Doctors typically think about medical errors as potential causes of malpractice litigation, as failures by individuals, and as evidence of personal incompetence that may deserve sanctions. Other professions take a different view: designing of safer systems, rather than criticism and punishment, is the way to prevent unintentional mishaps. In his article, Jeffrey Janofsky shows how psychiatrists can think about making care systems safer for patients. He also provides a splendid example of how forensic psychiatrists should conceptualize legal and medical problems encountered in clinical practice.

Conceptualizing and characterizing accuracy in assessments of competence to stand trial.

This article describes a mathematical framework for conceptualizing the accuracy of forensic experts' opinions on competence to stand trial (CST) and explains how an expert's expressed opinion about CST can be decomposed into four elements: (1) contextual requirements of the defendant (determined partly by the defendant's past actions) that lie outside the defendant's future control; (2) personal attributes of the defendant that are relevant to competence; (3) the expert's intrinsic ability to distinguish competent from incompetent defendants; and (4) the expert's wish to favor or avoid certain types of outcomes (e.g., a preference to avoid seeing an incompetent defendant stand trial for a serious charge). Because experts are imperfect and have varying levels of confidence in their opinions, one can describe the accuracy of CST assessments by using receiver operating characteristic (ROC) analysis. The article describes some types of insights one might derive from ROC analyses of CST assessments if experts, at least for research purposes, expressed opinions as graded levels of confidence. Although no satisfactory gold standard exists for establishing the truth about a defendant's competence, statistical methods developed over the past two decades may allow investigators to make inferences about the diagnostic accuracy of experts' CST assessments.

Analyzing the performance of risk assessment instruments: a response to Vrieze and Grove (2007).

In a recent article, Vrieze and Grove (Law Hum Behav, doi: 10.1007/s10979-007-9092-x , 2007) argue that, because of low recidivism base rates and limited predictive accuracy, an actuarial risk assessment instrument (ARAI) may produce decisions about sex offenders that are worse than simply predicting that no one will commit another sex offense. This article examines: (1) the construction and potential overfitting of ARAIs; (2) the meaning, value, and limitations of ROC areas; and (3) the relationship between the operating point that maximizes an ARAI's correct classifications and the legal criterion-"likely to reoffend"-used for sex offender designations. Contrary to what Vrieze and Grove suggest, ARAIs of modest accuracy yield probabilistic information that is more relevant to legal decision-making than just "betting the base rate."

Predicting restorability of incompetent criminal defendants.

U.S. courts frequently require forensic examiners to offer opinions concerning the likelihood that criminal defendants found incompetent to stand trial can have their competence "restored" through treatment. Yet no jurisdiction has established legal guidelines for testimony concerning restorability, and several authors have suggested that mental health professionals cannot accurately predict whether treatment to restore competence will succeed. This study asked whether reliable information that is consistently available at the time of examination might support empirically grounded opinions about the likelihood of restoration. Using records from all 351 inpatient pretrial defendants who underwent competence restoration at a state psychiatric hospital from 1995 through 1999, I evaluated whether several types of information that are reliable and that could consistently be made available to forensic examiners--including evaluees' demographic characteristics, diagnoses, symptom patterns, criminal charges, number of prior public sector hospitalizations, and cumulative prior length of stay (LOS)--would predict outcome of restoration efforts. I modeled the probability of successful restoration using logistic regression equations, and evaluated the equations' predictive accuracy using k-fold cross-validation and receiver operating characteristic (ROC) analysis. Lower probability of restoration was associated with having a misdemeanor charge, longer cumulative LOS, older age, and diagnoses of mental retardation, schizophrenia, and schizoaffective disorder. Although the overall rate of successful restoration for felony defendants was 75 percent, logistic equations allowed selection of subgroups with high predicted probabilities of restoration (>90%) and low probabilities of restoration (<35%). In cross-validation simulations, predictive equations had ROC areas of 0.727 for all defendants, and 0.735 for felony defendants. These findings provide scientific support for testimony that two types of incompetent evaluees have well-below-average probabilities of being restored: chronically psychotic defendants with histories of lengthy inpatient hospitalizations and defendants whose incompetence stems from irremediable cognitive disorders (such as mental retardation). Nonetheless, courts may still deem low probabilities of success to be "substantial" enough to warrant attempts at restoration.

AAPL Practice Guideline for the forensic psychiatric evaluation of competence to stand trial.

Competence to stand trial is a legal construct used to identify those criminal defendants who have the requisite mental capacity to understand the nature and objective of the proceedings against them and to participate rationally in preparing their defense. This Practice Guideline has described how psychiatrists should evaluate individuals concerning their competence to stand trial. The Guideline describes acceptable forensic psychiatric practice for such evaluations. Where possible, it specifies standards of practice and principles of ethics and also emphasizes the importance of analyzing an individual defendant's case in the context of statutes and case law applicable in the jurisdiction where the evaluation takes place. The recommendations in the Guideline both reflect and are limited by evolving case law, statutory requirements, legal publications, and the current state of psychiatric knowledge. The authors have taken note of nationally applicable case law, federal constitutional standards, statutory language, and federal and state interpretations of the rights or statutes, recognizing that jurisdictions may differ in their specific interpretation or application of statutes or general constitutional standards. The review of cases concerning specific psychiatric diagnoses illustrates general U.S. trends, and psychiatrists must remain cognizant of their jurisdictions' interpretations of statutes or constitutional requirements. By surveying a variety of practices and approaches to data gathering and case analysis, the authors believe that this Guideline will stimulate additional collegial discussion about what is necessary and sufficient for adequate evaluations of adjudicative competence. The notion that psychiatrists should apply expertise to competence assessments stems from the principal that, before allowing a defendant to face criminal prosecution and possible punishment, courts need reasonable assurance--based, if necessary, on a careful, individualized evaluation--that the defendant has adequate mental capacity to make a defense. At a minimum, a psychiatrist's opinion about adjudicative competence should reflect an understanding of the jurisdictional standard and of how the defendant's mental condition affects competence as defined with the jurisdiction. The psychiatrist's report should clearly describe the opinion and the reasoning that leads to it. Psychiatrists who provide mental health expertise concerning adjudicative competence give trial courts information needed to assure that defendants can appropriately protect themselves and that criminal proceedings will be accurate, dignified,and just.