Methods for Neuroscience Drug Development: Guidance on Standardization of the Process for Defining Clinical Outcome Strategies in Clinical Trials.

Neurosciences clinical trials continue to have notoriously high failure rates. Appropriate outcomes selection in early clinical trials is key to maximizing the likelihood of identifying new treatments in psychiatry and neurology. The field lacks good standards for designing outcome strategies, therefore The Outcomes Research Group was formed to develop and promote good practices in outcome selection. This article describes the first published guidance on the standardization of the process for clinical outcomes in neuroscience. A minimal step process is defined starting as early as possible, covering key activities for evidence generation in support of content validity, patient-centricity, validity requirements and considerations for regulatory acceptance. Feedback from expert members is provided, regarding the risks of shortening the process and examples supporting the recommended process are summarized. This methodology is now available to researchers in industry, academia or clinics aiming to implement consensus-based standard practices for clinical outcome selection, contributing to maximizing the efficiency of clinical research.

Sensitivity of Individual and Composite Test Scores from the Cogstate Brief Battery to Mild Cognitive Impairment and Dementia Due to Alzheimer's Disease.

BACKGROUND: The Cogstate Brief Battery (CBB) is a computerized cognitive test battery used commonly to identify cognitive deficits related to Alzheimer's disease (AD). However, AD and normative samples used to understand the sensitivity of the CBB to AD in the clinic have been limited, as have the outcome measures studied. OBJECTIVE: This study investigated the sensitivity of CBB outcomes, including potential composite scores, to cognitive impairment in mild cognitive impairment (MCI) and dementia due to AD, in carefully selected samples. METHODS: Samples consisted of 4,871 cognitively unimpaired adults and 184 adults who met clinical criteria for MCI (Clinical Dementia Rating (CDR) = 0.5) or dementia (CDR > 0.5) due to AD and CBB naive. Speed and accuracy measures from each test were examined, and theoretically- and statistically-derived composites were created. Sensitivity and specificity of classification of cognitive impairment were compared between outcomes. RESULTS: Individual CBB measures of learning and working memory showed high discriminability for AD-related cognitive impairment for CDR 0.5 (AUCs ∼ 0.79-0.88), and CDR > 0.5 (AUCs ∼ 0.89-0.96) groups. Discrimination ability for theoretically derived CBB composite measures was high, particularly for the Learning and Working Memory (LWM) composite (CDR 0.5 AUC = 0.90, CDR > 0.5 AUC = 0.97). As expected, statistically optimized linear composite measures showed strong discrimination abilities albeit similar to the LWM composite. CONCLUSIONS: In older adults, the CBB is effective for discriminating cognitive impairment due to MCI or AD-dementia from unimpaired cognition with the LWM composite providing the strongest sensitivity.

Recommendations on the Selection, Development, and Modification of Performance Outcome Assessments: A Good Practices Report of an ISPOR Task Force.

In evaluating the clinical benefit of new therapeutic interventions, it is critical that the treatment outcomes assessed reflect aspects of health that are clinically important and meaningful to patients. Performance outcome (PerfO) assessments are measurements based on standardized tasks actively undertaken by a patient that reflect physical, cognitive, sensory, and other functional skills that bring meaning to people's lives. PerfO assessments can have substantial value as drug development tools when the concepts of interest being measured best suit task performance and in cases where patients may be limited in their capacity for self-report. In their development, selection, and modification, including the evaluation and documentation of validity, reliability, usability, and interpretability, the good practice recommendations established for other clinical outcome assessment types should continue to be followed, with concept elicitation as a critical foundation. In addition, the importance of standardization, and the need to ensure feasibility and safety, as well as their utility in patient groups, such as pediatric populations, or those with cognitive and psychiatric challenges, may enhance the need for structured pilot evaluations, additional cognitive interviewing, and evaluation of quantitative data, such as that which would support concept confirmation or provide ecological evidence and other forms of construct evidence within a unitary approach to validity. The opportunity for PerfO assessments to inform key areas of clinical benefit is substantial and establishing good practices in their selection or development, validation, and implementation, as well as how they reflect meaningful aspects of health is critical to ensuring high standards and in furthering patient-focused drug development.

Recommendations to address key recruitment challenges of Alzheimer's disease clinical trials.

Clinical trials for Alzheimer's disease (AD) are slower to enroll study participants, take longer to complete, and are more expensive than trials in most other therapeutic areas. The recruitment and retention of a large number of qualified, diverse volunteers to participate in clinical research studies remain among the key barriers to the successful completion of AD clinical trials. An advisory panel of experts from academia, patient-advocacy organizations, philanthropy, non-profit, government, and industry convened in 2020 to assess the critical challenges facing recruitment in Alzheimer's clinical trials and develop a set of recommendations to overcome them. This paper briefly reviews existing challenges in AD clinical research and discusses the feasibility and implications of the panel's recommendations for actionable and inclusive solutions to accelerate the development of novel therapies for AD.

Clinically Relevant Changes for Cognitive Outcomes in Preclinical and Prodromal Cognitive Stages: Implications for Clinical Alzheimer Trials.

BACKGROUND AND OBJECTIVES: Identifying a clinically meaningful change in cognitive test score is essential when using cognition as an outcome in clinical trials. This is especially relevant because clinical trials increasingly feature novel composites of cognitive tests. Our primary objective was to establish minimal clinically important differences (MCIDs) for commonly used cognitive tests, using anchor-based and distribution-based methods, and our secondary objective was to investigate a composite cognitive measure that best predicts a minimal change in the Clinical Dementia Rating-Sum of Boxes (CDR-SB). METHODS: From the Swedish BioFINDER cohort study, we consecutively included cognitively unimpaired (CU) individuals with and without subjective or mild cognitive impairment (MCI). We calculated MCIDs associated with a change of ≥0.5 or ≥1.0 on CDR-SB for Mini-Mental State Examination (MMSE), ADAS-Cog delayed recall 10-word list, Stroop, Letter S Fluency, Animal Fluency, Symbol Digit Modalities Test (SDMT) and Trailmaking Test (TMT) A and B, and triangulated MCIDs for clinical use for CU, MCI, and amyloid-positive CU participants. For investigating cognitive measures that best predict a change in CDR-SB of ≥0.5 or ≥1.0 point, we conducted receiver operating characteristic analyses. RESULTS: Our study included 451 cognitively unimpaired individuals, 90 with subjective cognitive decline and 361 without symptoms of cognitive decline (pooled mean follow-up time 32.4 months, SD 26.8, range 12-96 months), and 292 people with MCI (pooled mean follow-up time 19.2 months, SD 19.0, range 12-72 months). We identified potential triangulated MCIDs (cognitively unimpaired; MCI) on a range of cognitive test outcomes: MMSE -1.5, -1.7; ADAS delayed recall 1.4, 1.1; Stroop 5.5, 9.3; Animal Fluency: -2.8, -2.9; Letter S Fluency -2.9, -1.8; SDMT: -3.5, -3.8; TMT A 11.7, 13.0; and TMT B 24.4, 20.1. For amyloid-positive CU, we found the best predicting composite cognitive measure included gender and changes in ADAS delayed recall, MMSE, SDMT, and TMT B. This produced an AUC of 0.87 (95% CI 0.79-0.94, sensitivity 75%, specificity 88%). DISCUSSION: Our MCIDs may be applied in clinical practice or clinical trials for identifying whether a clinically relevant change has occurred. The composite measure can be useful as a clinically relevant cognitive test outcome in preclinical AD trials.

A Paradox in Digital Memory Assessment: Increased Sensitivity With Reduced Difficulty.

The One Card Learning Test (OCL80) from the Cogstate Brief Battery-a digital cognitive test used both in-person and remotely in clinical trials and in healthcare contexts to inform health decisions-has shown high sensitivity to changes in memory in early Alzheimer's disease (AD). However, recent studies suggest that OCL sensitivity to memory impairment in symptomatic AD is not as strong as that for other standardized assessments of memory. This study aimed to improve the sensitivity of the OCL80 to AD-related memory impairment by reducing the test difficultly (i.e., OCL48). Experiment 1 showed performance in healthy adults improved on the OCL48 while the pattern separation operations that constrain performance on the OCL80 were retained. Experiment 2 showed repeated administration of the OCL48 at short retest intervals did not induce ceiling or practice effects. Experiment 3 showed that the sensitivity of the OCL48 to AD-related memory impairment (Glass's Δ = 3.11) was much greater than the sensitivity of the OCL80 (Glass's Δ = 1.94). Experiment 4 used data from a large group of cognitively normal older adults to calibrate performance scores between the OCL80 and OCL48 using equipercentile equating. Together these results showed the OCL48 to be a valid and reliable test of learning with greater sensitivity to memory impairment in AD than the OCL80.

Pilot Evaluation of the Unsupervised, At-Home Cogstate Brief Battery in ADNI-2.

BACKGROUND: There is a need for feasible, scalable assessments to detect cognitive impairment and decline. The Cogstate Brief Battery (CBB) is validated for Alzheimer's disease (AD) and in unsupervised and bring your own device contexts. The CBB has shown usability for self-completion in the home but has not been employed in this way in a multisite clinical trial in AD. OBJECTIVE: The objective of the pilot was to evaluate feasibility of at-home, self-completion of the CBB in the Alzheimer's Disease Neuroimaging Initiative (ADNI) over 24 months. METHODS: The CBB was included as a pilot for cognitively normal (CN) and mild cognitive impairment (MCI) participants in ADNI-2, invited to take the assessment in-clinic, then at at-home over a period of 24 months follow-up. Data were analyzed to explore acceptability/usability, concordance of in-clinic and at-home assessment, and validity. RESULTS: Data were collected for 104 participants (46 CN, 51 MCI, and 7 AD) who consented to provide CBB data. Subsequent analyses were performed for the CN and MCI groups only. Test completion rates were 100%for both the first in-clinic supervised and first at-home unsupervised assessments, with few repeat performances required. However, available follow-up data declined sharply over time. Good concordance was seen between in-clinic and at-home assessments, with non-significant and small effect size differences (Cohen's d between -0.04 and 0.28) and generally moderate correlations (r = 0.42 to 0.73). Known groups validity was also supported (11/16 comparisons with Cohen's d≥0.3). CONCLUSION: These data demonstrate the feasibility of use for the CBB for unsupervised at-home, testing, including MCI groups. Optimal approaches to the application of assessments to support compliance over time remain to be determined.

Building clinically relevant outcomes across the Alzheimer's disease spectrum.

Demonstrating that treatments are clinically meaningful across the Alzheimer's disease (AD) continuum is critical for meeting our goals of accelerating a cure by 2025. While this topic has been a focus of several Alzheimer's Association Research Roundtable (AARR) meetings, there remains no consensus as to what constitutes a "clinically meaningful outcome" in the eyes of patients, clinicians, care partners, policymakers, payers, and regulatory bodies. Furthermore, the field has not come to agreement as to what constitutes a clinically meaningful treatment effect at each stage of disease severity. The AARR meeting on November 19-20, 2019, reviewed current approaches to defining clinical meaningfulness from various perspectives including those of patients and care partners, clinicians, regulators, health economists, and public policymakers. Participants discussed approaches that may confer clinical relevance at each stage of the disease continuum and fostered discussion about what should guide us in the future.

Cognitive Go/No-Go decision-making criteria in Alzheimer's disease drug development.

Go/No-Go decision making in early phase clinical trials is challenging for drug developers working in Alzheimer's disease. Recent negative trial results have been attributed to a lack of efficacy and important safety concerns. Furthermore, demonstrated target engagement has rarely translated into demonstrable clinical efficacy. Cognitive data might provide valuable insights at various points during drug development, and a thoughtful and robust set of decision-making criteria, specified a priori, can and should be applied under many circumstances. This review provides insights into how to utilize cognitive data for Go/No-Go decisions, with an emphasis on how these cognitive criteria differ depending on the context (e.g., stage of development, mechanism of action and trial design).

Outcomes Assessment in Clinical Trials of Alzheimer's Disease and its Precursors: Readying for Short-term and Long-term Clinical Trial Needs.

An evolving paradigm shift in the diagnostic conceptualization of Alzheimer's disease is reflected in its recently updated diagnostic criteria from the National Institute on Aging-Alzheimer's Association and the International Working Group. Additionally, it is reflected in the increased focus in this field on conducting prevention trials in addition to improving cognition and function in people with dementia. These developments are making key contributions towards defining new regulatory thinking around Alzheimer's disease treatment earlier in the disease continuum. As a result, the field as a whole is now concentrated on exploring the next-generation of cognitive and functional outcome measures that will support clinical trials focused on treating the slow slide into cognitive and functional impairment. With this backdrop, the International Society for CNS Clinical Trials and Methodology convened semi-annual working group meetings which began in spring of 2012 to address methodological issues in this area. This report presents the most critical issues around primary outcome assessments in Alzheimer's disease clinical trials, and summarizes the presentations, discussions, and recommendations of those meetings, within the context of the evolving landscape of Alzheimer's disease clinical trials.

The effect of desvenlafaxine on cognitive functioning in employed outpatients with major depressive disorder: a substudy of a randomized, double-blind, placebo-controlled trial.

The objective of this substudy was to examine the effect of desvenlafaxine 50 mg/day compared with placebo on cognitive function in employed outpatients with major depressive disorder. A total of 11/55 (20%) study sites in a 12-week, randomized, double-blind, placebo-controlled trial administered cognitive assessments in memory, attention, and executive functioning domains using the cognitive drug research system. Changes from baseline were subjected to analysis of covariance with baseline levels as covariates, using last observation carried forward data. A significant improvement with desvenlafaxine 50 mg/day (n=52) compared with placebo (n=29) was observed on the quality of working memory composite measure (0.081 units (0.005, 0.156); P=0.0365) at last observation carried forward. Improvement from baseline on the speed of working memory composite was significant for desvenlafaxine (-226.6 msec (-316.7, -136.4); P<0.0001) and for placebo (-133.3 msec (-257.2, -9.4); P=0.0354); however, the treatment effect was not significant. No significant differences between groups were observed on composite measures for attention. Treatment of depression with desvenlafaxine 50 mg/day may improve aspects of cognitive functioning, including working memory.Clinical Trial Registry No.: Clinicaltrials.gov identifier: NCT00824291.

Psychometric evaluation of the Work Readiness Questionnaire in schizophrenia.

OBJECTIVE/INTRODUCTION: Unemployment can negatively impact quality of life among patients with schizophrenia. Employment status depends on ability, opportunity, education, and cultural influences. A clinician-rated scale of work readiness, independent of current work status, can be a valuable assessment tool. A series of studies were conducted to create and validate a Work Readiness Questionnaire (WoRQ) for clinicians to assess patient ability to engage in socially useful activity, independent of work availability. METHODS: Content validity, test-retest and inter-rater reliability, and construct validity were evaluated in three separate studies. RESULTS: Content validity was supported. Cronbach's α was 0.91, in the excellent range. Clinicians endorsed WoRQ concepts, including treatment adherence, physical appearance, social competence, and symptom control. The final readiness decision showed good test-retest reliability and moderate inter-rater reliability. Work readiness was associated with higher function and lower levels of negative symptoms. Low positive and high negative predictive values confirmed the concept validity. DISCUSSION: The WoRQ has suitable psychometric properties for use in a clinical trial for patients with a broad range of symptom severity. The scale may be applicable to assess therapeutic interventions. It is not intended to assess eligibility for supported work interventions. CONCLUSIONS: The WoRQ is suitable for use in schizophrenia clinical trials to assess patient work functional potential.

Establishing Equivalence of Electronic Clinician-Reported Outcome Measures.

BACKGROUND: Electronic administration of clinician-reported outcomes (eClinROs) has advantages over paper-based methods, but the mode of administration change has the potential to affect the validity of the scale. The literature on migration of patient-reported outcomes (PROs) suggests that there are different levels of modification, which necessitate different approaches to demonstrating mode equivalence. However, little has been written on the migration of ClinROs to electronic administration. METHODS: We propose a method of comparing paper and electronic versions of scales that includes a comparison based on content and a comparison based on format. The determination of whether the eClinRO has undergone minor, moderate, or substantial modification will drive the necessary studies required for validation. RESULTS: The unique characteristics of ClinROs suggest 2 additional types of modifications, including functionality adaptation and adaptation of instructions. CONCLUSIONS: In many respects, the migration of a ClinRO to electronic administration is similar to that of a PRO. This article has explored the ways in which there might be special considerations for ClinROs that have not been elaborated for PROs.

A prospective cohort study assessing clinical referral management & workforce allocation within a UK regional medical genetics service.

Ensuring patient access to genomic information in the face of increasing demand requires clinicians to develop innovative ways of working. This paper presents the first empirical prospective observational cohort study of UK multi-disciplinary genetic service delivery. It describes and explores collaborative working practices including the utilisation and role of clinical geneticists and non-medical genetic counsellors. Six hundred and fifty new patients referred to a regional genetics service were tracked through 850 clinical contacts until discharge. Referral decisions regarding allocation of lead health professional assigned to the case were monitored, including the use of initial clinical contact guidelines. Significant differences were found in the cases led by genetic counsellors and those led by clinical geneticists. Around a sixth, 16.8% (109/650) of referrals were dealt with by a letter back to the referrer or re-directed to another service provider and 14.8% (80/541) of the remaining patients chose not to schedule an appointment. Of the remaining 461 patients, genetic counsellors were allocated as lead health professional for 46.2% (213/461). A further 61 patients did not attend. Of those who did, 86.3% (345/400) were discharged after one or two appointments. Genetic counsellors contributed to 95% (784/825) of total patient contacts. They provided 93.7% (395/432) of initial contacts and 26.8% (106/395) of patients were discharged at that point. The information from this study informed a planned service re-design. More research is needed to assess the effectiveness and efficiency of different models of collaborative multi-disciplinary working within genetics services.

Defining therapeutic benefit for people with schizophrenia: focus on negative symptoms.

Schizophrenia is a complex, heterogeneous, multidimensional disorder within which negative symptoms are a significant and disabling feature. Whilst there is no established treatment for these symptoms, some pharmacological and psychosocial interventions have shown promise and this is an active area of research. Despite the effort to identify effective interventions, as yet there is no broadly accepted definition of therapeutic success. This article reviews concepts of clinical relevance and reports on a consensus conference whose goal was to apply these concepts to the treatment of negative symptoms. A number of key issues were identified and discussed including: assessment of specific negative symptom domains; defining response and remission for negative symptoms; assessment of functional outcomes; measurement of outcomes within clinical trials; and the assessment of duration/persistence of a response. The group reached a definition of therapeutic success using an achieved threshold of function that persisted over time. Recommendations were agreed upon with respect to: assessment of negative symptom domains of apathy-avolition and deficit of expression symptoms; thresholds for response and remission of negative symptoms based on level of symptomatology; assessing multiple domains of function including social occupation, activities of daily living, and socialization; the need for clinical trial data to include rate of change over time and converging sources of evidence; use of clinician, patient and caregiver perspectives to assess success; and the need for establishing criteria for the persistence of therapeutic benefit. A consensus statement and associated research criteria are offered as an initial step towards developing broad agreement regarding outcomes of negative symptoms treatment.

Reliability, validity and ability to detect change of the PANSS negative symptom factor score in outpatients with schizophrenia on select antipsychotics and with prominent negative or disorganized thought symptoms.

The PANSS is a valid instrument assessing schizophrenia symptom severity. Analyses have identified a five-factor solution. The negative symptom factor (NSFS) is robust, having been replicated in multiple analyses. The score has superior content validity versus the negative subscale. Aspects of validity in patients with predominant negative symptoms have yet to be established. The present data are from a Phase IIb study of add-on bitopertin therapy in schizophrenia outpatients with prominent negative or disorganized thought symptoms treated with antipsychotics. Analyses were conducted to evaluate reliability, validity and sensitivity to change. Test-retest screening to baseline was high (ICC=0.93). This was maintained in-study, for patients with no change in CGI negative symptom severity (CGI-S-N). Internal consistency at baseline was adequate (α=0.71) and increased at later assessments. Pearson correlation at baseline showed a good association between NSFS and CGI-S-N (0.63), but not overall CGI-S (0.31). Association with PSP at baseline was moderate (-0.39) and for change at Week eight good (-0.65). NSFS responders (≥20% improvement) at Week eight showed a significant improvement in function. The analyses demonstrated reliability, validity and ability to detect change of the NSFS, in schizophrenia patients with prominent negative or disorganized thought symptoms.

Performance on a pattern separation task by Alzheimer's patients shows possible links between disrupted dentate gyrus activity and apolipoprotein E ∈4 status and cerebrospinal fluid amyloid-ß42 levels.

INTRODUCTION: Emerging evidence suggests that decreased adult hippocampal neurogenesis represents an early critical event in the course of Alzheimer's disease (AD). In mice, adult neurogenesis is reduced by knock-in alleles for human apolipoprotein E (ApoE) ∈4. Decreased dentate gyrus (DG) neural progenitor cells proliferation has been observed in the triple-transgenic mouse model of AD (3xTg-AD); this reduction being directly associated with the presence of amyloid-ß (Aß) plaques and an increase in the number of Aß-containing neurons in the hippocampus. Cognitive tasks involving difficult pattern separations have been shown to reflect DG activity and thus potentially neurogenesis in both animals and man. This study involved the administration of a pattern separation paradigm to Alzheimer's patients to investigate relationships between task performance and both ApoE status and cerebrospinal fluid (CSF) Aß42 levels. METHODS: The CDR System pattern separation task involves the presentation of pictures that must later be discriminated from closely similar pictures. This paper presents pattern separation data from 66 mild to moderate AD patients, of which 50 were genotyped and 65 in whom CSF Aß42 was measured. RESULTS: ApoE ∈4 homozygotes were not compromised on the easy pattern separations compared with the other patients, but they were statistically significantly poorer at the difficult separations. In all patients CSF Aß42 correlated significantly with the ability to make the difficult discriminations, but not easier discriminations. Pattern separation speed correlated negatively with CSF Aß42, and thus the association was not due to increased impulsivity. CONCLUSIONS: These are, to our knowledge, the first human pattern separation data to suggest a possible genetic link to poor hippocampal neurogenesis in AD, as well as a relationship to Aß42. Therapies which target neurogenesis may thus be useful in preventing the early stages of AD, notably in ApoE ∈4 homocygotes.