Outlier-response pattern checks to improve measurement with the Montgomery-Asberg depression rating scale (MADRS).

Symptom manifestations in affective disorders can be subtle. Small imprecisions in measurement can lead to incorrect estimation of change. Previously, expert-derived scoring inconsistency flags were developed for MADRS. Currently, we derive empirically based outlier-pattern flags, to further detect imprecisions in ratings. NEWMEDS data repository of almost 25,000 MADRS administrations from 11 registration trials of antidepressants was used to identify outlier response patterns reflecting potentially careless responses. Coverage of these flags was compared to previously published expert derived flags. Both sets of flags were also further tested in Monte Carlo simulated data as a proxy to applying flags under conditions of known inconsistency. The outlier flags derived provide cutting points to identify: (1) under and overuse of values (e.g., Scoring "1″ on 6 or more items), (2) disproportionate use of even or odd response choices (e.g., 8 or more odd values), (3) longest consecutive use of value (e.g., more than 5 items in a row scored with same value), (4) high variability within administration (standard deviation greater than 1.8), (5) outlier responses on multiple items (i.e., multivariate outliers), and (6) outlier scoring (e.g., scoring 4,5 or 6 on item 1). Outlier response flags were raised in 26% of the MADRS administration and in 97% of the Monte Carlo data. Of administrations with no expert flag, 21.7% had an outlier flag and of administrations with at least one expert flag, 27.7% also had an outlier flag. Outlier-pattern flags appear to be a useful adjunct to expert derived flags in the quest to improve measurement in clinical trials.

Outlier-response pattern checks to improve measurement with the Positive and Negative Syndrome Scale (PANSS).

We derived outlier-response pattern checks to flag possible careless PANSS (Positive and Negative Syndrome Scale) administrations based on analysis of 122,000 administrations from 29 registration trials of antipsychotics from NEWMEDS data repository. Flags identify outlier administrations based on frequency of endorsing a given response value, use of even or odd values, consecutive use of same value, variability of values, responses per specific item, and values on multiple items. Outlier flags were compared to published expert derived scoring inconsistency flags and tested in Monte Carlo simulated data, with known inconsistency, and appear to be useful at identifying administrations that require review.

Consistency checks to improve measurement with the Personal and Social Performance Scale (PSP).

International Society for CNS Clinical Trials and Methodology convened an expert Working Group that assembled consistency/inconsistency flags for the Personal and Social Performance Scale (PSP). One hundred and forty seven flags were identified, 16 flag errors in deriving the PSP decile (i.e., total) score from the four individual domain scores, 74 flag inconsistencies between domain scores relative to Positive and Negative Symptom Scale (PANSS) item ratings and 57 flag inconsistencies between PSP decile score and PANSS items ratings. The flags were applied to assessments from randomized clinical trial data of antipsychotics in schizophrenia from almost 18,000 ratings. Twenty-two flags were raised in at least 5 of 1000 ratings. Nearly 20% of the PSP ratings had at least one inconsistency flag raised. Application of flags to clinical ratings may improve the reliability of ratings and validity of trials.