Impact of BPRS interview length on ratings reliability in a schizophrenia trial.

Signal detection in clinical trials relies on ratings reliability. We conducted a reliability analysis of site-independent rater scores derived from audio-digital recordings of site-based rater interviews of the structured Brief Psychiatric Rating Scale (BPRS) in a schizophrenia study. "Dual" ratings assessments were conducted as part of a quality assurance program in a 12-week, double-blind, parallel-group study of PF-02545920 compared to placebo in patients with sub-optimally controlled symptoms of schizophrenia (ClinicalTrials.gov identifier NCT01939548). Blinded, site-independent raters scored the recorded site-based BPRS interviews that were administered in relatively stable patients during two visits prior to the randomization visit. We analyzed the impact of BPRS interview length on "dual" scoring variance and discordance between trained and certified site-based raters and the paired scores of the independent raters. Mean total BPRS scores for 392 interviews conducted at the screen and stabilization visits were 50.4±7.2 (SD) for site-based raters and 49.2±7.2 for site-independent raters (t=2.34; p=0.025). "Dual" rated total BPRS scores were highly correlated (r=0.812). Mean BPRS interview length was 21:05±7:47min ranging from 7 to 59min. 89 interviews (23%) were conducted in less than 15min. These shorter interviews had significantly greater "dual" scoring variability (p=0.0016) and absolute discordance (p=0.0037) between site-based and site-independent raters than longer interviews. In-study ratings reliability cannot be guaranteed by pre-study rater certification. Our findings reveal marked variability of BPRS interview length and that shorter interviews are often incomplete yielding greater "dual" scoring discordance that may affect ratings precision.

A comparative analysis between site-based and centralized ratings and patient self-ratings in a clinical trial of Major Depressive Disorder.

We compared scores from three different ratings methods in a clinical trial of patients with Major Depressive Disorder (MDD). The Quick Inventory of Depressive Symptoms (QIDS-SR16) was compared to site-based clinician and centralized (site-independent) ratings of the Inventory of Depressive Symptoms (IDSc30). An extracted QIDSc16 was used for a matched comparison with the QIDS-SR16. Patient self-ratings were more depressed at baseline than either site-based ratings (p = 0.131) or centralized ratings (p = 0.005), but significantly less depressed at the end of double-blind treatment than either site-based (p = 0.006) or centralized ratings (p = 0.014), and after 12 weeks (site-based ratings: p = 0.048; centralized ratings: p = 0.004). The matched comparisons with patient self-ratings revealed ICC of r = 0.55 (site-based raters) and r = 0.49 (centralized raters) at baseline. After baseline, the correlations between the two different clinician ratings and patient self-ratings improved to r-values between 0.78 and 0.89. At the end of double-blind treatment, site-based raters separated the combination treatment from placebo on the IDSc30 (p = 0.030) whereas neither centralized ratings nor patient self-ratings achieved statistical significance. Alternatively, patient self-ratings separated the combination treatment from buspirone (p = 0.030) whereas neither clinician rating method achieved significance. A "dual" scoring concordance range reduced the placebo response rate and increased the drug effect between the combination treatment and placebo. These findings reveal scoring variability between each of the three ratings methods and challenge the reliability of any single method to accurately assess symptom severity scores, particularly at baseline. The use of "dual" scoring criteria may help to confirm symptom severity scores and improve ratings precision, particularly prior to enrolling subjects into CNS trials.