Methods for Neuroscience Drug Development: Guidance on Standardization of the Process for Defining Clinical Outcome Strategies in Clinical Trials.

Neurosciences clinical trials continue to have notoriously high failure rates. Appropriate outcomes selection in early clinical trials is key to maximizing the likelihood of identifying new treatments in psychiatry and neurology. The field lacks good standards for designing outcome strategies, therefore The Outcomes Research Group was formed to develop and promote good practices in outcome selection. This article describes the first published guidance on the standardization of the process for clinical outcomes in neuroscience. A minimal step process is defined starting as early as possible, covering key activities for evidence generation in support of content validity, patient-centricity, validity requirements and considerations for regulatory acceptance. Feedback from expert members is provided, regarding the risks of shortening the process and examples supporting the recommended process are summarized. This methodology is now available to researchers in industry, academia or clinics aiming to implement consensus-based standard practices for clinical outcome selection, contributing to maximizing the efficiency of clinical research.

Recommendations for selection and adaptation of rating scales for clinical studies of rapid-acting antidepressants.

The novel mechanisms of action (MOA) derived from some recently introduced molecular targets have led to regulatory approvals for rapid acting antidepressants (RAADs) that can generate responses within hours or days, rather than weeks or months. These novel targets include the N-methyl-D-glutamate receptor antagonist ketamine, along with its enantiomers and various derivatives, and the allosteric modulators of gamma-aminobutyric acid (GABA) receptors. There has also been a strong resurgence in interest in psychedelic compounds that impact a range of receptor sites including D1, 5-HT7, KOR, 5-HT5A, Sigma-1, NMDA, and BDNF. The RAADs developed from these novel targets have enabled successful treatment for difficult to treat depressed individuals and has generated a new wave of innovation in research and treatment. Despite the advances in the neurobiology and clinical treatment of mood disorders, we are still using rating instruments that were created decades ago for drugs from a different era (e.g., The Hamilton and Montgomery-Åsberg depression rating scales, HDRS, and MADRS) continue to be used. These rating instruments were designed to assess mood symptoms over a 7-day time frame. Consequently, the use of these rating instruments often requires modifications to address items that cannot be assessed in short time frames, such as the sleep and appetite items. This review describes the adaptative approaches that have been made with the existing scales to meet this need and examines additional domains such as daily activities, side effects, suicidal ideation and behavior, and role functioning. Recommendations for future studies are described, including the challenges related to implementation of these adapted measures and approaches to mitigation.

Consistency checks to improve measurement with the Hamilton Rating Scale for Anxiety (HAM-A).

BACKGROUND: Mitigating rating inconsistency can improve measurement fidelity and detection of treatment response. METHODS: The International Society for CNS Clinical Trials and Methodology convened an expert Working Group that developed consistency checks for ratings of the Hamilton Anxiety Rating Scale (HAM-A) and Clinical Global Impression of Severity of anxiety (CGIS) that are widely used in studies of mood and anxiety disorders. Flags were applied to 40,349 HAM-A administrations from 15 clinical trials and to Monte Carlo-simulated data as a proxy for applying flags under conditions of inconsistency. RESULTS: Thirty-three flags were derived these included logical consistency checks and statistical outlier-response pattern checks. Twenty-percent of the HAM-A administrations had at least one logical scoring inconsistency flag, 4 % had two or more. Twenty-six percent of the administrations had at least one statistical outlier flag and 11 % had two or more. Overall, 35 % of administrations had at least one flag of any type, 19 % had one and 16 % had 2 or more. Most of administrations in the Monte Carlo- simulated data raised multiple flags. LIMITATIONS: Flagged ratings may represent less-common presentations of administrations done correctly. Conclusions-Application of flags to clinical ratings may aid in detecting imprecise measurement. Flags can be used for monitoring of raters during an ongoing trial and as part of post-trial evaluation. Appling flags may improve reliability and validity of trial data.

Ketamine for Bipolar Depression: Biochemical, Psychotherapeutic, and Psychedelic Approaches.

Bipolar disorder (type 1) is a serious and chronic psychiatric illness that can be difficult to treat. Many bipolar patients have refractory depressive episodes. Racemic ketamine, a glutamate modulator with prominent dissociate and psychedelic properties, has been demonstrated to have rapid acting antidepressant and anti-obsessional effects which may be useful for treating the symptoms of bipolar depression. Most of the existing research literature on unipolar and bipolar depression has looked at racemic ketamine in the sub-psychedelic dose range given by infusion as a stand-alone treatment (without concurrent psychotherapy). This article expands on the existing research by articulating three different paradigms for ketamine treatment: biochemical, psychotherapeutic, and psychedelic. The authors use composite clinical vignettes to illustrate different ways of working with ketamine to treat bipolar depression, and discuss a variety of clinical considerations for using ketamine with this population, including route, dose, frequency, chemical mitigators, and adverse events. Note that the conceptual paradigms could be applied to any ketamine treatment, with broad applicability beyond bipolar treatment.

Consistency checks to improve measurement with the Hamilton Rating Scale for Depression (HAM-D).

BACKGROUND: Symptom manifestations in mood disorders can be subtle. Cumulatively, small imprecisions in measurement can limit our ability to measure treatment response accurately. Logical and statistical consistency checks between item responses (i.e., cross-sectionally) and across administrations (i.e., longitudinally) can contribute to improving measurement fidelity. METHODS: The International Society for CNS Clinical Trials and Methodology convened an expert Working Group that assembled flags indicating consistency/inconsistency ratings for the Hamilton Rating Scale for Depression (HAM-D17), a widely-used rating scale in studies of depression. Proposed flags were applied to assessments derived from the NEWMEDS data repository of 95,468 HAM-D administrations from 32 registration trials of antidepressant medications and to Monte Carlo-simulated data as a proxy for applying flags under conditions of known inconsistency. RESULTS: Two types of flags were derived: logical consistency checks and statistical outlier-response pattern checks. Almost thirty percent of the HAMD administrations had at least one logical scoring inconsistency flag. Seven percent had flags judged to suggest that a thorough review of rating is warranted. Almost 22% of the administrations had at least one statistical outlier flag and 7.9% had more than one. Most of the administrations in the Monte Carlo- simulated data raised multiple flags. LIMITATIONS: Flagged ratings may represent less-common presentations of administrations done correctly. CONCLUSIONS: Application of flags to clinical ratings may aid in detecting imprecise measurement. Reviewing and addressing these flags may improve reliability and validity of clinical trial data.

Consistency checks to improve measurement with the Montgomery-Asberg Depression Rating Scale (MADRS).

International Society for CNS Clinical Trials and Methodology convened an expert Working Group that assembled consistency/inconsistency flags for the Montgomery-Asberg Depression Rating Scale (MADRS). Twenty-two flags were identified. Seven flags are believed to be strong flags that suggest that a thorough review of rating is warranted. The flags were applied to assessments derived from the NEWMEDS data repository. Almost 65% of ratings had at least one inconsistency flag raised and 22% had two or more. Application of flags to clinical ratings may improve reliability of ratings and validity of trials.

Risk-Based Data Monitoring: Quality Control in Central Nervous System (CNS) Clinical Trials.

Monitoring the quality of clinical trial efficacy outcome data has received increased attention in the past decade, with regulatory guidance encouraging it to be conducted proactively, and remotely. However, the methods utilized to develop and implement risk-based data monitoring (RBDM) programs vary, and there is a dearth of published material to guide these processes in the context of central nervous system (CNS) trials. We reviewed regulatory guidance published within the past 6 years, generic white papers, and studies applying RBDM to data from CNS clinical trials. Methodologic considerations and system requirements necessary to establish an effective, real-time risk-based monitoring platform in CNS trials are presented. Key RBDM terms are defined in the context of CNS trial data, such as "critical data," "risk indicators," "noninformative data," and "mitigation of risk." Additionally, potential benefits of, and challenges associated with implementation of data quality monitoring are highlighted. Application of methodological and system requirement considerations to real-time monitoring of clinical ratings in CNS trials has the potential to minimize risk and enhance the quality of clinical trial data.

Consistency checks to improve measurement with the Positive and Negative Syndrome Scale (PANSS).

International Society for CNS Clinical Trials and Methodology convened an expert working-group that assembled consistency/inconsistency flags for the Positive and Negative Syndrome Scale (PANSS). Twenty-four flags were identified and divided based on extent to which they represent error (Possibly, Probably, Very probably or definitely). The flags were applied to assessments derived from the NEWMEDS data repository and the CATIE clinical trial data. Almost 40% of ratings had at least one inconsistency flag raised and 10% had two. Application of flags to clinical rating can improve reliability and validity of trials.

Positive and Negative Syndrome Scale (PANSS) Training: Challenges, Solutions, and Future Directions.

Rater training and the maintenance of the consistency of ratings are critical to ensuring reliability of study measures and sensitivity to changes in the course of a clinical trial. The Positive and Negative Syndrome Scale (PANSS) has been widely used in clinical trials of schizophrenia and other disorders and is considered the "gold standard" for assessment of antipsychotic treatment efficacy. The various features associated with training and calibration of this scale are complex, reflecting the intricacy and heterogeneity of the disorders that the PANSS is used to evaluate. In this article, the authors review the methods for ensuring reliability of the PANSS as well as a proposed trajectory for its use in the future. An overview of the current principles, implementation, technologies, and strategies for the best use of the PANSS; tips for how to achieve consistency among raters; and optimal training practices of this instrument are presented.

Reliability of the Alzheimer's disease assessment scale (ADAS-Cog) in longitudinal studies.

BACKGROUND: Considering the scarcity of longitudinal assessments of reliability, there is need for a more precise understanding of cognitive decline in Alzheimer's Disease (AD). The primary goal was to assess longitudinal changes in inter-rater reliability, test retest reliability and internal consistency of scores of the ADAS-Cog. METHODS: 2,618 AD subjects were enrolled in seven randomized, double-blind, placebo-controlled, multicenter-trials from 1986 to 2009. Reliability, internal-consistency and cross-sectional analysis of ADAS-Cog and MMSE across seven visits were examined. RESULTS: Intra-class correlation (ICC) for ADAS-Cog was moderate to high supporting their reliability. Absolute Agreement ICCs 0.392 (Visit-7) to 0.806 (Visit-2) showed a progressive decrease in correlations across time. Item analysis revealed a decrease in item correlations, with the lowest correlations for Visit 7 for Commands (ICC=0.148), Comprehension (ICC=0.092), Spoken Language (ICC=0.044). DISCUSSION: Suitable assessment of AD treatments is maintained through accurate measurement of clinically significant outcomes. Targeted rater education ADAS-Cog items over-time can improve ability to administer and score the scale.

A new Integrated Negative Symptom structure of the Positive and Negative Syndrome Scale (PANSS) in schizophrenia using item response analysis.

BACKGROUND: Debate persists with regard to how best to categorize the syndromal dimension of negative symptoms in schizophrenia. The aim was to first review published Principle Components Analysis (PCA) of the PANSS, and extract items most frequently included in the negative domain, and secondly, to examine the quality of items using Item Response Theory (IRT) to select items that best represent a measurable dimension (or dimensions) of negative symptoms. METHODS: First, 22 factor analyses and PCA met were included. Second, using a large dataset (n=7187) of participants in clinical trials with chronic schizophrenia, we extracted items loading on one or more PCA. Third, items not loading with a value of ≥ 0.5, or loading on more than one component with values of ≥ 0.5 were discarded. Fourth, resulting items were included in a non-parametric IRT and retained based on Option Characteristic Curves (OCCs) and Item Characteristic Curves (ICCs). RESULTS: 15 items loaded on a negative domain in at least one study, with Emotional Withdrawal loading on all studies. Non-parametric IRT retained nine items as an Integrated Negative Factor: Emotional Withdrawal, Blunted Affect, Passive/Apathetic Social Withdrawal, Poor Rapport, Lack of Spontaneity/Conversation Flow, Active Social Avoidance, Disturbance of Volition, Stereotyped Thinking and Difficulty in Abstract Thinking. CONCLUSIONS: This is the first study to use a psychometric IRT process to arrive at a set of negative symptom items. Future steps will include further examination of these nine items in terms of their stability, sensitivity to change, and correlations with functional and cognitive outcomes.

A rasch model to test the cross-cultural validity in the positive and negative syndrome scale (PANSS) across six geo-cultural groups.

BACKGROUND: The objective of this study was to examine the cross-cultural differences of the PANSS across six geo-cultural regions. The specific aims are (1) to examine measurement properties of the PANSS; and (2) to examine how each of the 30 items function across geo-cultural regions. METHODS: Data was obtained for 1,169 raters from 6 different regions: Eastern Asia (n = 202), India (n = 185), Northern Europe (n = 126), Russia & Ukraine (n = 197), Southern Europe (n = 162), United States (n = 297). A principle components analysis assessed unidimensionality of the subscales. Rasch rating scale analysis examined cross-cultural differences among each item of the PANSS. RESULTS: Lower item values reflects items in which raters often showed less variation in the scores; higher item values reflects items with more variation in the scores. Positive Subscale: Most regions found item P5 (Excitement) to be the most difficult item to score. Items varied in severity from -0.93 [item P6. Suspiciousness/persecution (USA) to 0.69 item P4. Excitement (Eastern Asia)]. Item P3 (Hallucinatory Behavior) was the easiest item to score for all geographical regions. Negative Subscale: The most difficult item to score for all regions is N7 (Stereotyped Thinking) with India showing the most difficulty Δ = 0.69, and Northern Europe and the United States showing the least difficulty Δ = 0.21, each. The second most difficult item for raters to score was N1 (Blunted Affect) for most countries including Southern Europe (Δ = 0.30), Eastern Asia (Δ = 0.28), Russia & Ukraine (Δ = 0.22) and India (Δ = 0.10). General Psychopathology: The most difficult item for raters to score for all regions is G4 (Tension) with difficulty levels ranging from Δ = 1.38 (India) to Δ = 0.72. CONCLUSIONS: There were significant differences in response to a number of items on the PANSS, possibly caused by a lack of equivalence between the original and translated versions, cultural differences among interpretation of items or scoring parameters. Knowing which items are problematic for various cultures can help guide PANSS training and make training specialized for specific geographical regions.