Estimands for clinical endpoints in tuberculosis treatment randomized controlled trials: a retrospective application in a completed trial.

BACKGROUND: Randomized trials for the treatment of tuberculosis (TB) rely on a composite primary outcome to capture unfavorable treatment responses. However, variability between trials in the outcome definition and estimation methods complicates across-trial comparisons and hinders the advancement of treatment guidelines. The International Council for Harmonization (ICH) provides international regulatory standards for clinical trials. The estimand framework outlined in the recent ICH E9(R1) addendum offers a timely opportunity for randomized trials of TB treatment to adopt broadly standardized outcome definitions and analytic approaches. We previously proposed and defined four estimands for use in this context. Our objective was to evaluate how the use of these estimands and choice of estimation method impacts results and interpretation of a large phase III TB trial. METHODS: We reanalyzed participant-level data from the REMoxTB trial. We applied four estimands and various methods of estimation to assess non-inferiority of both novel 4-month treatment regimens against standard of care. RESULTS: With each of the four estimands, we reached the same conclusion as the original trial analysis that the novel regimens were not non-inferior to standard of care. Each estimand and method of estimation gave similar estimates of the treatment effect with fluctuations in variance and differences driven by the methods applied for handling intercurrent events. CONCLUSIONS: Our application of estimands defined by the ICH E9 (R1) addendum offers a formalized framework for addressing the primary TB treatment trial objective and can promote uniformity in future trials by limiting heterogeneity in trial outcome definitions. We demonstrated the utility of our proposal using data from the REMoxTB randomized trial. We outlined methods for estimating each estimand and found consistent conclusions across estimands. We recommend future late-phase TB treatment trials to implement some or all of our estimands to promote rigorous outcome definitions and reduce variability between trials. TRIAL REGISTRATION: ClinicalTrials.gov NCT00864383. Registered on March 2009.

An evaluation of confidence intervals for a cumulative proportion to enable decisions at interim reviews of single-arm trials.

BACKGROUND: Clinical trials often include interim analyses of the proportion of participants experiencing an event by a fixed time-point. A pre-specified proportion excluded from a corresponding confidence interval (CI) may lead an independent monitoring committee to recommend stopping the trial. Frequently this cumulative proportion is estimated by the Kaplan-Meier estimator with a Wald approximate CI, which may have coverage issues with small samples. METHODS: We reviewed four alternative CI methods for cumulative proportions (Beta Product Confidence Procedure (BPCP), BPCP Mid P, Rothman-Wilson, Thomas-Grunkemeier) and two CI methods for simple proportions (Clopper-Pearson, Wilson). We conducted a simulation study comparing CI methods across true event proportions for 12 scenarios differentiated by sample sizes and censoring patterns. We re-analyzed interim data from A5340, a HIV cure trial considering the proportion of participants experiencing virologic failure. RESULTS: Our simulation study highlights the lower and upper tail error probabilities for each CI method. Across scenarios, we found differences in the performance of lower versus upper bounds. No single method is always preferred. The upper bound of a Wald approximate CI performed reasonably with some error inflation, whereas the lower bound of the BPCP Mid P method performed well. For a trial design similar to A5340, we recommend BPCP Mid P. CONCLUSIONS: The design of future single-arm interim analyses of event proportions should consider the most appropriate CI method based on the relevant bound, anticipated sample size and event proportion. Our paper summarizes available methods, demonstrates performance in a simulation study, and includes code for implementation.

Estimands for clinical endpoints in Tuberculosis treatment randomized controlled trials: a retrospective application in a completed trial.

Background: Randomized trials for the treatment of tuberculosis (TB) rely on a composite primary outcome to capture unfavorable treatment responses. However, variability between trials in the outcome definition and estimation methods complicates across-trial comparisons and hinders the advancement of treatment guidelines. The International Council for Harmonization (ICH) provides international regulatory standards for clinical trials. The estimand framework outlined in the recent ICH E9(R1) addendum offers a timely opportunity for randomized trials of TB treatment to adopt broadly standardized outcome definitions and analytic approaches. We previously proposed and defined four estimands for use in this context. Our objective was to evaluate how the use of these estimands and choice of estimation method impacts results and interpretation of a large phase III TB trial. Methods: We reanalyzed participant level data from the REMoxTB trial. We applied four estimands and various methods of estimation to assess non-inferiority of both novel 4-month treatment regimens against standard of care. Results: With each of the four estimands we reached the same conclusion as the original trial analysis; that the novel regimens were not non-inferior to standard of care. Each estimand and method of estimation gave similar estimates of the treatment effect with fluctuations in variance and differences driven by the methods applied for handling intercurrent events. Conclusions: Our application of estimands defined by the ICH E9(R1) addendum offers a formalized framework for addressing the primary TB treatment trial objective and can promote uniformity in future trials by limiting heterogeneity in trial outcome definitions. We demonstrated the utility of our proposal using data from the REMoxTB randomized trial. We outlined methods for estimating each estimand and found consistent conclusions across estimands. We recommend future late-phase TB treatment trials to implement some or all of our estimands to promote rigorous outcome definitions and reduce variability between trials.Trial registration: NCT00864383.

Methods for handling missing data in serially sampled sputum specimens for mycobacterial culture conversion calculation.

BACKGROUND: The occurrence and timing of mycobacterial culture conversion is used as a proxy for tuberculosis treatment response. When researchers serially sample sputum during tuberculosis studies, contamination or missed visits leads to missing data points. Traditionally, this is managed by ignoring missing data or simple carry-forward techniques. Statistically advanced multiple imputation methods potentially decrease bias and retain sample size and statistical power. METHODS: We analyzed data from 261 participants who provided weekly sputa for the first 12 weeks of tuberculosis treatment. We compared methods for handling missing data points in a longitudinal study with a time-to-event outcome. Our primary outcome was time to culture conversion, defined as two consecutive weeks with no Mycobacterium tuberculosis growth. Methods used to address missing data included: 1) available case analysis, 2) last observation carried forward, and 3) multiple imputation by fully conditional specification. For each method, we calculated the proportion culture converted and used survival analysis to estimate Kaplan-Meier curves, hazard ratios, and restricted mean survival times. We compared methods based on point estimates, confidence intervals, and conclusions to specific research questions. RESULTS: The three missing data methods lead to differences in the number of participants achieving conversion; 78 (32.8%) participants converted with available case analysis, 154 (64.7%) converted with last observation carried forward, and 184 (77.1%) converted with multiple imputation. Multiple imputation resulted in smaller point estimates than simple approaches with narrower confidence intervals. The adjusted hazard ratio for smear negative participants was 3.4 (95% CI 2.3, 5.1) using multiple imputation compared to 5.2 (95% CI 3.1, 8.7) using last observation carried forward and 5.0 (95% CI 2.4, 10.6) using available case analysis. CONCLUSION: We showed that accounting for missing sputum data through multiple imputation, a statistically valid approach under certain conditions, can lead to different conclusions than naïve methods. Careful consideration for how to handle missing data must be taken and be pre-specified prior to analysis. We used data from a TB study to demonstrate these concepts, however, the methods we described are broadly applicable to longitudinal missing data. We provide valuable statistical guidance and code for researchers to appropriately handle missing data in longitudinal studies.

Asymptomatic SARS-CoV-2 Infection Is Common Among ART-Treated People With HIV.

BACKGROUND: Limited data are available regarding asymptomatic COVID-19 among people with HIV (PWH). Data on a representative subset of PWH enrolled in Randomized Trial to Prevent Vascular Events in HIV, a global clinical trial, are presented here. METHODS: Randomized Trial to Prevent Vascular Events in HIV is an atherosclerotic cardiovascular disease prevention trial among 7770 PWH on antiretroviral therapy. Beginning April 2020, targeted data on coronavirus disease 2019 (COVID-19) diagnosis and symptoms were collected during routine trial visits. Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection was defined as either COVID-19 clinical diagnosis or presence of SARS-CoV-2 Immunoglobulin G (IgG) or Immunoglobulin A (IgA) receptor binding domain protein (antispike) antibodies in the absence of prior COVID-19 vaccine. RESULTS: The group (N = 2464) had a median age 53 years, 35% female sex, 47% Black or African American race, median CD4 count 649 c/mm 3 , and 97% with HIV VL <400 cp/m. SARS-CoV-2 infection occurred in 318 persons (13%): 58 with clinical diagnosis and 260 with detectable antibodies. Of these PWH, 304 completed symptom questionnaires: 121 (40%) reported symptoms, but 183 (60%) were asymptomatic. PWH with asymptomatic SARS-CoV-2 infection were more likely to be from low-income or middle-income regions, of Black or African American race, older in age, and with higher atherosclerotic cardiovascular disease risk score. Symptomatic COVID was more common with obesity, metabolic syndrome, and low HDL levels. CD4 counts and HIV viral suppression rates were similar among PWH with symptomatic vs. asymptomatic COVID. CONCLUSIONS: Asymptomatic SARS-CoV-2 infection is common among antiretroviral therapy-treated PWH globally. We determined that 60% of infections in PWH were asymptomatic. HIV clinicians must remain vigilant about COVID-19 testing among PWH to identify asymptomatic cases.

Prevalence and Correlates of Electrocardiographic Abnormalities in Adults With HIV: Insights From the Randomized Trial to Prevent Vascular Events in HIV (REPRIEVE).

BACKGROUND: People with HIV (PWH) are at increased risk of cardiovasvular disease (CVD) and sudden cardiac death. Previous work has suggested an association between HIV infection and electrocardiographic (ECG) abnormalities. There are limited data on the burden of ECG abnormalities among PWH in a multiracial, multiethnic globally representative population. SETTING: One hundred twenty sites in the Randomized Trial to Prevent Vascular Events in HIV (REPRIEVE). METHODS: ECG findings were grouped into clinically relevant categories using sex-specific thresholds when indicated. We used the Fisher exact tests to assess associations of demographic characteristics and ECG abnormalities. We used logistic regression model to assess associations between demographic and HIV management measures, with adjustment. RESULTS: We analyzed data for 7720 PWH (99% of participants) (median age 50 years, 69% male participants). There were 3346 (43%) Black or African American, 2680 (35%) White, and 1139 (15%) Asian participants. Most of the participants (97%) had viral load that was <400 copies/mL or 400 copies/mL had approximately twice the odds of prolonged QTc compared with those that were undetectable (adjusted OR: 2.05, 95% CI: 1.22 to 3.45). CONCLUSIONS: Prolonged QTc is common among male, Asian, and REPRIEVE participants with higher viral loads. These relationships warrant future investigation of linkages to ensuing CVD events among PWH.

Counterfactual mediation analysis in the multistate model framework for surrogate and clinical time-to-event outcomes in randomized controlled trials.

In cancer randomized controlled trials, surrogate endpoints are frequently time-to-event endpoints, subject to the competing risk from the time-to-event clinical outcome. In this context, we introduce a counterfactual-based mediation analysis for a causal assessment of surrogacy. We use a multistate model for risk prediction to account for both direct transitions towards the clinical outcome and indirect transitions through the surrogate outcome. Within the counterfactual framework, we define natural direct and indirect effects with a causal interpretation. Based on these measures, we define the proportion of the treatment effect on the clinical outcome mediated by the surrogate outcome. We estimate the proportion for both the cumulative risk and restricted mean time lost. We illustrate our approach by using 18-year follow-up data from the SPCG-4 randomized trial of radical prostatectomy for prostate cancer. We assess time to metastasis as a surrogate outcome for prostate cancer-specific mortality.

Treatment Effect Measures for Culture Conversion Endpoints in Phase IIb Tuberculosis Treatment Trials.

Phase IIb trials of tuberculosis therapy rely on early biomarkers of treatment effect. Despite limited predictive ability for clinical outcomes, culture conversion, the event in which an individual previously culture positive for Mycobacterium tuberculosis yields a negative culture after initiating treatment, is a commonly used endpoint. Lack of consensus on how to define the outcome and corresponding measure of treatment effect complicates interpretation and limits between-trial comparisons. We review common analytic approaches to measuring treatment effect and introduce difference in restricted mean survival times as an alternative to identify faster times to culture conversion and express magnitude of effect on the time scale. Findings from the PanACEA MAMS-TB trial are reanalyzed as an illustrative example. In a systematic review we demonstrate variability in analytic approaches, sampling strategies, and outcome definitions in phase IIb tuberculosis trials. Harmonization would allow for larger meta-analyses and may help expedite advancement of new tuberculosis therapeutics.

Multivariate meta-analysis model for the difference in restricted mean survival times.

In randomized controlled trials (RCTs) with time-to-event outcomes, the difference in restricted mean survival times (RMSTD) offers an absolute measure of the treatment effect on the time scale. Computation of the RMSTD relies on the choice of a time horizon, $\tau$. In a meta-analysis, varying follow-up durations may lead to the exclusion of RCTs with follow-up shorter than $\tau$. We introduce an individual patient data multivariate meta-analysis model for RMSTD estimated at multiple time horizons. We derived the within-trial covariance for the RMSTD enabling the synthesis of all data by borrowing strength from multiple time points. In a simulation study covering 60 scenarios, we compared the statistical performance of the proposed method to that of two univariate meta-analysis models, based on available data at each time point and based on predictions from flexible parametric models. Our multivariate model yields smaller mean squared error over univariate methods at all time points. We illustrate the method with a meta-analysis of five RCTs comparing transcatheter aortic valve replacement (TAVR) with surgical replacement in patients with aortic stenosis. Over 12, 24, and 36 months of follow-up, those treated by TAVR live 0.28 [95% confidence interval (CI) 0.01 to 0.56], 0.46 (95% CI $-$0.08 to 1.01), and 0.79 (95% CI $-$0.43 to 2.02) months longer on average compared to those treated by surgery, respectively.

Design analysis indicates Potential overestimation of treatment effects in randomized controlled trials supporting Food and Drug Administration cancer drug approvals.

OBJECTIVE: Statistical significance drives interpretation of randomized controlled trials (RCTs). We examined the type S error risk-claiming a new drug is falsely beneficial-and exaggeration ratio-how estimated effects differ from true effects-to re-emphasize direction and magnitude of treatment effects. STUDY DESIGN AND SETTING: We systematically reviewed RCTs supporting Food and Drug Administration (FDA) approval of cancer drugs between 2007 and 2016. We extracted data for overall survival (OS), progression-free survival (PFS), and response outcomes from FDA reviews. We estimated type S error risks and exaggeration ratios by considering replicated RCTs of equal size and a range of true effects. RESULTS: We analyzed 42 RCTs for 39 approved drugs. Across 38 RCTs reporting OS, the median type S error risk was 0.00% (Q1-Q3, 0.00-0.01%) and 3.56% (0.40-6.74%), for true hazard ratios of 0.7 and 0.9, respectively, indicating confidence in effect direction. The corresponding exaggeration ratios were 1.09 (1.01-1.11) and 1.30 (1.13-1.42), indicating median overestimations of 9% and 30%. Similar results held for PFS and response outcomes. CONCLUSIONS: The type S error risk and exaggeration ratio provide additional insights into the replicability of RCTs. Our analyses also quantify the winner's curse, in which pivotal RCTs tend toward overoptimism.

Design of non-inferiority randomized trials using the difference in restricted mean survival times.

Background/aims Non-inferiority trials with time-to-event outcomes are becoming increasingly common. Designing non-inferiority trials is challenging, in particular, they require very large sample sizes. We hypothesized that the difference in restricted mean survival time, an alternative to the hazard ratio, could lead to smaller required sample sizes. Methods We show how to convert a margin for the hazard ratio into a margin for the difference in restricted mean survival time and how to calculate the required sample size under a Weibull survival distribution. We systematically selected non-inferiority trials published between 2013 and 2016 in seven major journals. Based on the protocol and article of each trial, we determined the clinically relevant time horizon of interest. We reconstructed individual patient data for the primary outcome and fit a Weibull distribution to the comparator arm. We converted the margin for the hazard ratio into the margin for the difference in restricted mean survival time. We tested for non-inferiority using the difference in restricted mean survival time and hazard ratio. We determined the required sample size based on both measures, using the type I error risk and power from the original trial design. Results We included 35 trials. We found evidence of non-proportional hazards in five (14%) trials. The hazard ratio and the difference in restricted mean survival time were consistent regarding non-inferiority testing, except in one trial where the difference in restricted mean survival time led to evidence of non-inferiority while the hazard ratio did not. The median hazard ratio margin was 1.43 (Q1-Q3, 1.29-1.75). The median of the corresponding margins for the difference in restricted mean survival time was -21 days (Q1-Q3, -36 to -8) for a median time horizon of 2.0 years (Q1-Q3, 1-3 years). The required sample size according to the difference in restricted mean survival time was smaller in 71% of trials, with a median relative decrease of 8.5% (Q1-Q3, 0.4%-38.0%). Across all 35 trials, about 25,000 participants would have been spared from enrollment using the difference in restricted mean survival time compared to hazard ratio for trial design. Conclusion The margins for the hazard ratio may seem large but translate to relatively small differences in restricted mean survival time. The difference in restricted mean survival time offers meaningful interpretation and can result in considerable reductions in sample size. Restricted mean survival time-based measures should be considered more widely in the design and analysis of non-inferiority trials with time-to-event outcomes.