Evidence and reporting standards in N-of-1 medical studies: a systematic review.

N-of-1 trials, a special case of Single Case Experimental Designs (SCEDs), are prominent in clinical medical research and specifically psychiatry due to the growing significance of precision/personalized medicine. It is imperative that these clinical trials be conducted, and their data analyzed, using the highest standards to guard against threats to validity. This systematic review examined publications of medical N-of-1 trials to examine whether they meet (a) the evidence standards and (b) the criteria for demonstrating evidence of a relation between an independent and an outcome variable per the What Works Clearinghouse (WWC) standards for SCEDs. We also examined the appropriateness of the data analytic techniques in the special context of N-of-1 designs. We searched for empirical journal articles that used N-of-1 design and published between 2013 and 2022 in PubMed and Web of Science. Protocols or methodological papers and studies that did not manipulate a medical condition were excluded. We reviewed 115 articles; 4 (3.48%) articles met all WWC evidence standards. Most (99.1%) failed to report an appropriate design-comparable effect size; neither did they report a confidence/credible interval, and 47.9% reported neither the raw data rendering meta-analysis impossible. Most (83.8%) ignored autocorrelation and did not meet distributional assumptions (65.8%). These methodological problems could lead to significantly inaccurate effect sizes. It is necessary to implement stricter guidelines for the clinical conduct and analyses of medical N-of-1 trials. Reporting neither raw data nor design-comparable effect sizes renders meta-analysis impossible and is antithetical to the spirit of open science.

Bayesian analysis of single case experimental design count data in trauma research: A tutorial.

OBJECTIVE: Single case experimental designs (SCEDs) are suited to psychological trauma research that involves individualized treatments and where randomization might be impossible or inappropriate. However, SCED data present challenges such as autocorrelations, short time-series, nonnormal distributions (in count data), and the effect sizes being comparable across phase changes and participants. The present study illustrates the Bayesian rate ratio (BRR) that can be used when the phase lengths vary across phases to quantify treatment effect and can be aggregated across participants. METHOD: Bayesian estimation allows flexible modeling of autocorrelations and distributions, works well for small samples, and provides easily interpretable estimates of uncertainty in the form of credible intervals. BRR estimates from a published dataset from trauma research are compared with commonly used nonoverlap of all pairs (NAP) effect sizes. RESULTS: Bayesian estimates show superior performance over NAP estimates because they reflect the patterns in the data and account for distance between observations, unlike NAP. All the R programs are shared with the readers and are annotated for their ease. CONCLUSIONS: Through tutorials such as these, current and future researchers can be educated about Bayesian methodology, its appropriateness to address the idiosyncrasies posed by SCED count data, and aid further development of models for other SCEDs. Implications of the study are also discussed. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Power analysis for single-case designs: Computations for (AB)k designs.

Currently, the design standards for single-case experimental designs (SCEDs) are based on validity considerations as prescribed by the What Works Clearinghouse. However, there is a need for design considerations such as power based on statistical analyses. We compute and derive power using computations for (AB)k designs with multiple cases which are common in SCEDs. Our computations show that effect size has the maximum impact on power followed by the number of subjects and then the number of phase reversals. An effect size of 0.75 or higher, at least one set of phase reversals (i.e., where k > 1), and at least three subjects showed high power. The latter two conditions agree with current standards about either having at least an ABAB design or a multiple baseline design with three subjects to meet design standards. An effect size of 0.75 or higher is not uncommon in SCEDs either. Autocorrelations, the number of time-points per phase, and intraclass correlations had a smaller but non-negligible impact on power. In sum, power analyses in the present study show that conditions to meet power requirements are not unreasonable in SCEDs. The software code to compute power is available on GitHub for the use of the reader.

Evaluation of the Posttraumatic Stress Disorder Checklist for DSM-5 across United States and India samples: An item response theory analysis using the graded response model.

OBJECTIVES: The Posttraumatic Stress Disorder (PTSD) Checklist for DSM-5 (PCL-5) is frequently used to assess PTSD symptoms. Extending its psychometric investigations across distinct samples (United States and India), the aims of the present study included investigating the item characteristics, person fit, and differential item functioning (DIF) across the two samples. METHOD: We (a) conducted item analysis using the graded response model to examine item characteristics (thresholds and discrimination parameters) and (b) examined person fit to determine participants' response patterns. The U.S. sample included 176 trauma-exposed individuals seeking mental health treatment, and the Indian sample included 148 trauma-exposed ex-military personnel. RESULTS: Results indicated low discrimination for Item 8 and low but acceptable discrimination for Item 17 for the U.S. and Indian samples, respectively. Across both samples, the most unused response categories were the middle-point or extreme (higher severity) categories (all response categories were better utilized in the Indian sample), and half the participants exhibited person misfit. Supplemental DIF analysis indicated that five items exhibited DIF indicating potential item bias; however, effect sizes for DIF were negligible. CONCLUSIONS: Although the PCL-5 demonstrated strong psychometric properties and showed promise for use across the samples differing on cultural and demographic characteristics, some of the items and the number of categories used to measure them could be revisited to create a more efficient instrument. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Longitudinal relationships among posttraumatic stress disorder symptom clusters in response to positive memory processing.

BACKGROUND AND OBJECTIVES: Avoidance, inherent to posttraumatic stress disorder (PTSD) symptomatology, is theoretically and empirically linked to the maintenance of PTSD symptom severity. While research indicates traumatized individuals avoid positive and trauma memories, several PTSD treatments focus exclusively on traumatic memories. We examined the mediating role of PTSD's avoidance in the relationship between processing positive memories and PTSD cluster severity (intrusion, mood/cognitions, arousal). METHODS: Sixty-five trauma-exposed college students (Mage = 22.52; 86.10% female) were randomly assigned to 3 conditions: narrating/processing, writing/processing, or control (same task across baseline [T0] and follow-up [T1]). RESULTS: Half-longitudinal mediation models indicated participation in the narrating vs. writing and control conditions predicted decreases in T1 intrusion severity via reduction in T1 avoidance severity. Similarly, participation in the narrating vs. writing and control conditions predicted decreases in T1 mood/cognitions' severity via reduction in T1 avoidance severity. Participation in the narrating vs. writing condition predicted decreases in T1 arousal severity via reduction in T1 avoidance severity. LIMITATIONS: Data was obtained from an analogue small-size sample of university students. In addition, sessions were only 6-8 days apart, with the processing component of each session lasting ∼30 min. CONCLUSIONS: Processing positive memories may relate to lower PTSD severity via a reduction in PTSD's avoidance, paralleling effects of processing trauma memories. Our findings support future investigations of the mechanisms underlying impacts of positive memory processing in the context of PTSD treatments.

Psychometric Evaluation of the Posttrauma Risky Behaviors Questionnaire: Item Response Theory Analyses.

The Posttrauma Risky Behaviors Questionnaire (PRBQ) assesses extent of engagement in posttrauma reckless and self-destructive behaviors (RSDBs). Given PRBQ's recent development with limited psychometric investigations, we used item response theory to examine (a) item analysis, (b) person fit, and (c) differential item functioning (DIF) across gender-based groups and two different samples. One sample included 464 participants reporting potentially traumatic experiences (Mechanical Turk [MTurk], recruited online), and the other sample included 171 trauma-exposed women reporting current intimate partner violence and substance use (recruited in-person). All PRBQ items contributed to the RSDB scale, and all PRBQ items and the PRBQ scale provided maximum information for high levels of the RSDB latent trait. Seven and 11 items were conceptualized as low information items in the MTurk and intimate partner violence samples, respectively. Eight MTurk participants' responses did not fit the overall pattern of responses as expected. Seven items were flagged for DIF between the two samples, and eight items were flagged for DIF between men and women in the MTurk sample. However, all effect sizes were <8%. Conclusively, results suggest good psychometric properties for the PRBQ and support its use to compare RSDBs across different samples and gender-based groups.

A psychometric evaluation of the Marijuana Problems Index among college students: Confirmatory factor analysis and measurement invariance by gender.

Although the Marijuana Problems Index (MPI) is widely used in studies with college student samples to reflect a unidimensional measure of cannabis-related problems, no studies have assessed the psychometric properties of the MPI in a college student population. The present study sought to resolve this gap in a sample of 879 college students reporting past-year cannabis use. Confirmatory factor analyses were used to test the factor structure of the unidimensional 23- and 18-item MPI and measurement invariance across gender. Bivariate correlations between the final factors, cannabis use history/frequency, and other substance use outcomes were used to examine concurrent and discriminant validities (i.e., vs. noncannabis outcomes). The 18-item (but not the 23-item) MPI demonstrated good model fit, measurement invariance across gender, adequate internal reliability, as well as concurrent and discriminant validities. Results support the use of the 18-item MPI over the 23-item MPI for conceptualizing problematic cannabis use, including the testing of gender-specific differences, among college students. Findings also reinforce the importance of evaluating the psychometric properties of widely used measures across samples. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Accurate models vs. accurate estimates: A simulation study of Bayesian single-case experimental designs.

Although statistical practices to evaluate intervention effects in single-case experimental design (SCEDs) have gained prominence in recent times, models are yet to incorporate and investigate all their analytic complexities. Most of these statistical models incorporate slopes and autocorrelations, both of which contribute to trend in the data. The question that arises is whether in SCED data that show trend, there is indeterminacy between estimating slope and autocorrelation, because both contribute to trend, and the data have a limited number of observations. Using Monte Carlo simulation, we compared the performance of four Bayesian change-point models: (a) intercepts only (IO), (b) slopes but no autocorrelations (SI), (c) autocorrelations but no slopes (NS), and (d) both autocorrelations and slopes (SA). Weakly informative priors were used to remain agnostic about the parameters. Coverage rates showed that for the SA model, either the slope effect size or the autocorrelation credible interval almost always erroneously contained 0, and the type II errors were prohibitively large. Considering the 0-coverage and coverage rates of slope effect size, intercept effect size, mean relative bias, and second-phase intercept relative bias, the SI model outperformed all other models. Therefore, it is recommended that researchers favor the SI model over the other three models. Research studies that develop slope effect sizes for SCEDs should consider the performance of the statistic by taking into account coverage and 0-coverage rates. These helped uncover patterns that were not realized in other simulation studies. We underline the need for investigating the use of informative priors in SCEDs.

Bayesian Time-Series Models in Single Case Experimental Designs: A Tutorial for Trauma Researchers.

Single-case experimental designs (SCEDs) involve obtaining repeated measures from one or a few participants before, during, and, sometimes, after treatment implementation. Because they are cost-, time-, and resource-efficient and can provide robust causal evidence for more large-scale research, SCEDs are gaining popularity in trauma treatment research. However, sophisticated techniques to analyze SCED data remain underutilized. Herein, we discuss the utility of SCED data for trauma research, provide recommendations for addressing challenges specific to SCED approaches, and introduce a tutorial for two Bayesian models-the Bayesian interrupted time-series (BITS) model and the Bayesian unknown change-point (BUCP) model-that can be used to analyze the typically small sample, autocorrelated, SCED data. Software codes are provided for the ease of guiding readers in estimating these models. Analyses of a dataset from a published article as well as a trauma-specific simulated dataset are used to illustrate the models and demonstrate the interpretation of the results. We further discuss the implications of using such small-sample data-analytic techniques for SCEDs specific to trauma research.

Investigating immediacy in multiple-phase-change single-case experimental designs using a Bayesian unknown change-points model.

Immediacy is one of the necessary criteria to show strong evidence of treatment effect in single-case experimental designs (SCEDs). However, with the exception of Natesan and Hedges (2017), no inferential statistical tool has been used to demonstrate or quantify it until now. We investigate and quantify immediacy by treating the change points between the baseline and treatment phases as unknown. We extend Natesan and Hedges' work to multiple-phase-change (e.g. ABAB) designs using a variational Bayesian (VB) unknown change-point model. VB was used instead of Markov chain Monte Carlo methods (MCMC), because MCMC cannot be used effectively to determine multiple change points. Combined and individual probabilities of correctly estimating the change points were used as indicators of the algorithm's accuracy. Unlike MCMC in the Natesan and Hedges (2017) study, the VB method was able to recover the change points with high accuracy even for short time series and in only a fraction of the time for all time-series lengths. We illustrate the algorithm with 13 real data sets. Additionally, we discuss the advantages of the unknown change-point approach, and the Bayesian and variational Bayesian estimation for SCEDs.

Comparison of latent typologies of posttraumatic stress disorder and depression symptoms across military personnel from India and the US.

Research has identified heterogeneous subgroups of individuals based on posttraumatic stress disorder (PTSD) and depression symptoms. Using data collected from military personnel in India (N = 146) and U.S. (N = 194), we examined (1) the best-fitting latent class solution; (2) multi-group invariance of the class solution; and (3) construct validity of optimal class solution. Results indicated that the optimal 4-class solution differed in severity and severity/type in the India and U.S. samples respectively. With similarity in the optimal number of classes across cultural samples, the meaning/nature of classes differed. In the India sample, anxiety severity predicted the Low Severity Class vs. all other classes, and the Moderately High Severity/High Severity Classes vs. the Moderately Low Severity Class; number of traumas predicted the High Severity Class vs. other classes; and resilience predicted the Moderately Low Severity Class vs. the Moderately High Severity Class. In the U.S. sample, alcohol use predicted the High Severity Class vs. all other classes, and the High Depression-Low PTSD Class vs. the Low Severity Class; rumination significantly predicted the High Severity and High Depression-Low PTSD Classes vs. each of the High PTSD-Low Depression and Low Severity Classes. Thus, meaning and nature of PTSD-depression subgroups may vary culturally; hence, culturally-sensitive interventions need to account for this heterogeneity.

Comparing the Bayesian Unknown Change-Point Model and Simulation Modeling Analysis to Analyze Single Case Experimental Designs.

Recently, there has been an increased interest in developing statistical methodologies for analyzing single case experimental design (SCED) data to supplement visual analysis. Some of these are simulation-driven such as Bayesian methods because Bayesian methods can compensate for small sample sizes, which is a main challenge of SCEDs. Two simulation-driven approaches: Bayesian unknown change-point model (BUCP) and simulation modeling analysis (SMA) were compared in the present study for three real datasets that exhibit "clear" immediacy, "unclear" immediacy, and delayed effects. Although SMA estimates can be used to answer some aspects of functional relationship between the independent and the outcome variables, they cannot address immediacy or provide an effect size estimate that considers autocorrelation as required by the What Works Clearinghouse (WWC) Standards. BUCP overcomes these drawbacks of SMA. In final analysis, it is recommended that both visual and statistical analyses be conducted for a thorough analysis of SCEDs.