Response to discussant comments on "NMA, the first 20 years".

We respond to discussant comments on our paper "Twenty years of network meta-analysis: Continuing controversies and recent developments" (https://doi.org/10.1002/jrsm.1700) and raise some additional points for consideration, including: the way in which methodological guidance is generated; integration of the estimand framework with evidence synthesis; and implications of the European Joint Clinical Assessment. We ask: what properties are required of population adjustment methods to enable transparent and consistent decision-making? We also ask why individual patient data is not routinely made available to re-imbursement authorities and clinical guideline developers.

Twenty years of network meta-analysis: Continuing controversies and recent developments.

Network meta-analysis (NMA) is an extension of pairwise meta-analysis (PMA) which combines evidence from trials on multiple treatments in connected networks. NMA delivers internally consistent estimates of relative treatment efficacy, needed for rational decision making. Over its first 20 years NMA's use has grown exponentially, with applications in both health technology assessment (HTA), primarily re-imbursement decisions and clinical guideline development, and clinical research publications. This has been a period of transition in meta-analysis, first from its roots in educational and social psychology, where large heterogeneous datasets could be explored to find effect modifiers, to smaller pairwise meta-analyses in clinical medicine on average with less than six studies. This has been followed by narrowly-focused estimation of the effects of specific treatments at specific doses in specific populations in sparse networks, where direct comparisons are unavailable or informed by only one or two studies. NMA is a powerful and well-established technique but, in spite of the exponential increase in applications, doubts about the reliability and validity of NMA persist. Here we outline the continuing controversies, and review some recent developments. We suggest that heterogeneity should be minimized, as it poses a threat to the reliability of NMA which has not been fully appreciated, perhaps because it has not been seen as a problem in PMA. More research is needed on the extent of heterogeneity and inconsistency in datasets used for decision making, on formal methods for making recommendations based on NMA, and on the further development of multi-level network meta-regression.

Unanchored Population-Adjusted Indirect Comparison Methods for Time-to-Event Outcomes Using Inverse Odds Weighting, Regression Adjustment, and Doubly Robust Methods With Either Individual Patient or Aggregate Data.

OBJECTIVES: Several methods for unanchored population-adjusted indirect comparisons (PAICs) are available. Exploring alternative adjustment methods, depending on the available individual patient data (IPD) and the aggregate data (AD) in the external study, may help minimize bias in unanchored indirect comparisons. However, methods for time-to-event outcomes are not well understood. This study provides an overview and comparison of methods using a case study to increase familiarity. A recent method is applied to marginalize conditional hazard ratios, which allows for the comparisons of methods, and a doubly robust method is proposed. METHODS: The following PAIC methods were compared through a case study in third-line small cell lung cancer, comparing nivolumab with standard of care based on a single-arm phase II trial (CheckMate 032) and real-world study (Flatiron) in terms of overall survival: IPD-IPD analyses using inverse odds weighting, regression adjustment, and a doubly robust method; IPD-AD analyses using matching-adjusted indirect comparison, simulated treatment comparison, and a doubly robust method. RESULTS: Nivolumab extended survival versus standard of care with hazard ratios ranging from 0.63 (95% CI 0.44-0.90) in naive comparisons (identical estimates for IPD-IPD and IPD-AD analyses) to 0.69 (95% CI 0.44-0.98) in the IPD-IPD analyses using regression adjustment. Regression-based and doubly robust estimates yielded slightly wider confidence intervals versus the propensity score-based analyses. CONCLUSIONS: The proposed doubly robust approach for time-to-event outcomes may help to minimize bias due to model misspecification. However, all methods for unanchored PAIC rely on the strong assumption that all prognostic covariates have been included.

Transportability of two heart failure trials to a disease registry using individual patient data.

OBJECTIVES: Randomized controlled trials are the gold-standard for determining therapeutic efficacy, but are often unrepresentative of real-world settings. Statistical transportation methods (hereafter transportation) can partially account for these differences, improving trial applicability without breaking randomization. We transported treatment effects from two heart failure (HF) trials to a HF registry. STUDY DESIGN AND SETTING: Individual-patient-level data from two trials (Carvedilol or Metoprolol European Trial (COMET), comparing carvedilol and metoprolol, and digitalis investigation group trial (DIG), comparing digoxin and placebo) and a Scottish HF registry were obtained. The primary end point for both trials was all-cause mortality; composite outcomes were all-cause mortality or hospitalization for COMET and HF-related death or hospitalization for DIG. We performed transportation using regression-based and inverse odds of sampling weights (IOSW) approaches. RESULTS: Registry patients were older, had poorer renal function and received higher-doses of loop-diuretics than trial participants. For each trial, point estimates were similar for the original and IOSW (e.g., DIG composite outcome: OR 0.75 (0.69, 0.82) vs. 0.73 (0.64, 0.83)). Treatment effect estimates were also similar when examining high-risk (0.64 (0.46, 0.89)) and low-risk registry patients (0.73 (0.61, 0.86)). Similar results were obtained using regression-based transportation. CONCLUSION: Regression-based or IOSW approaches can be used to transport trial effect estimates to patients administrative/registry data, with only moderate reductions in precision.

Antidepressants for pain management in adults with chronic pain: a network meta-analysis.

BACKGROUND: Chronic pain is common in adults, and often has a detrimental impact upon physical ability, well-being, and quality of life. Previous reviews have shown that certain antidepressants may be effective in reducing pain with some benefit in improving patients' global impression of change for certain chronic pain conditions. However, there has not been a network meta-analysis (NMA) examining all antidepressants across all chronic pain conditions. OBJECTIVES: To assess the comparative efficacy and safety of antidepressants for adults with chronic pain (except headache). SEARCH METHODS: We searched CENTRAL, MEDLINE, Embase, CINAHL, LILACS, AMED and PsycINFO databases, and clinical trials registries, for randomised controlled trials (RCTs) of antidepressants for chronic pain conditions in January 2022. SELECTION CRITERIA: We included RCTs that examined antidepressants for chronic pain against any comparator. If the comparator was placebo, another medication, another antidepressant, or the same antidepressant at different doses, then we required the study to be double-blind. We included RCTs with active comparators that were unable to be double-blinded (e.g. psychotherapy) but rated them as high risk of bias. We excluded RCTs where the follow-up was less than two weeks and those with fewer than 10 participants in each arm.  DATA COLLECTION AND ANALYSIS: Two review authors separately screened, data extracted, and judged risk of bias. We synthesised the data using Bayesian NMA and pairwise meta-analyses for each outcome and ranked the antidepressants in terms of their effectiveness using the surface under the cumulative ranking curve (SUCRA). We primarily used Confidence in Meta-Analysis (CINeMA) and Risk of Bias due to Missing Evidence in Network meta-analysis (ROB-MEN) to assess the certainty of the evidence. Where it was not possible to use CINeMA and ROB-MEN due to the complexity of the networks, we used GRADE to assess the certainty of the evidence. Our primary outcomes were substantial (50%) pain relief, pain intensity, mood, and adverse events. Our secondary outcomes were moderate pain relief (30%), physical function, sleep, quality of life, Patient Global Impression of Change (PGIC), serious adverse events, and withdrawal. MAIN RESULTS: This review and NMA included 176 studies with a total of 28,664 participants. The majority of studies were placebo-controlled (83), and parallel-armed (141). The most common pain conditions examined were fibromyalgia (59 studies); neuropathic pain (49 studies) and musculoskeletal pain (40 studies). The average length of RCTs was 10 weeks. Seven studies provided no useable data and were omitted from the NMA. The majority of studies measured short-term outcomes only and excluded people with low mood and other mental health conditions. Across efficacy outcomes, duloxetine was consistently the highest-ranked antidepressant with moderate- to high-certainty evidence. In duloxetine studies, standard dose was equally efficacious as high dose for the majority of outcomes. Milnacipran was often ranked as the next most efficacious antidepressant, although the certainty of evidence was lower than that of duloxetine. There was insufficient evidence to draw robust conclusions for the efficacy and safety of any other antidepressant for chronic pain.  Primary efficacy outcomes Duloxetine standard dose (60 mg) showed a small to moderate effect for substantial pain relief (odds ratio (OR) 1.91, 95% confidence interval (CI) 1.69 to 2.17; 16 studies, 4490 participants; moderate-certainty evidence) and continuous pain intensity (standardised mean difference (SMD) -0.31, 95% CI -0.39 to -0.24; 18 studies, 4959 participants; moderate-certainty evidence). For pain intensity, milnacipran standard dose (100 mg) also showed a small effect (SMD -0.22, 95% CI -0.39 to 0.06; 4 studies, 1866 participants; moderate-certainty evidence). Mirtazapine (30 mg) had a moderate effect on mood (SMD -0.5, 95% CI -0.78 to -0.22; 1 study, 406 participants; low-certainty evidence), while duloxetine showed a small effect (SMD -0.16, 95% CI -0.22 to -0.1; 26 studies, 7952 participants; moderate-certainty evidence); however it is important to note that most studies excluded participants with mental health conditions, and so average anxiety and depression scores tended to be in the 'normal' or 'subclinical' ranges at baseline already. Secondary efficacy outcomes Across all secondary efficacy outcomes (moderate pain relief, physical function, sleep, quality of life, and PGIC), duloxetine and milnacipran were the highest-ranked antidepressants with moderate-certainty evidence, although effects were small. For both duloxetine and milnacipran, standard doses were as efficacious as high doses. Safety There was very low-certainty evidence for all safety outcomes (adverse events, serious adverse events, and withdrawal) across all antidepressants. We cannot draw any reliable conclusions from the NMAs for these outcomes. AUTHORS' CONCLUSIONS: Our review and NMAs show that despite studies investigating 25 different antidepressants, the only antidepressant we are certain about for the treatment of chronic pain is duloxetine. Duloxetine was moderately efficacious across all outcomes at standard dose. There is also promising evidence for milnacipran, although further high-quality research is needed to be confident in these conclusions. Evidence for all other antidepressants was low certainty. As RCTs excluded people with low mood, we were unable to establish the effects of antidepressants for people with chronic pain and depression. There is currently no reliable evidence for the long-term efficacy of any antidepressant, and no reliable evidence for the safety of antidepressants for chronic pain at any time point.

Validating the Assumptions of Population Adjustment: Application of Multilevel Network Meta-regression to a Network of Treatments for Plaque Psoriasis.

BACKGROUND: Network meta-analysis (NMA) and indirect comparisons combine aggregate data (AgD) from multiple studies on treatments of interest but may give biased estimates if study populations differ. Population adjustment methods such as multilevel network meta-regression (ML-NMR) aim to reduce bias by adjusting for differences in study populations using individual patient data (IPD) from 1 or more studies under the conditional constancy assumption. A shared effect modifier assumption may also be necessary for identifiability. This article aims to demonstrate how the assumptions made by ML-NMR can be assessed in practice to obtain reliable treatment effect estimates in a target population. METHODS: We apply ML-NMR to a network of evidence on treatments for plaque psoriasis with a mix of IPD and AgD trials reporting ordered categorical outcomes. Relative treatment effects are estimated for each trial population and for 3 external target populations represented by a registry and 2 cohort studies. We examine residual heterogeneity and inconsistency and relax the shared effect modifier assumption for each covariate in turn. RESULTS: Estimated population-average treatment effects were similar across study populations, as differences in the distributions of effect modifiers were small. Better fit was achieved with ML-NMR than with NMA, and uncertainty was reduced by explaining within- and between-study variation. We found little evidence that the conditional constancy or shared effect modifier assumptions were invalid. CONCLUSIONS: ML-NMR extends the NMA framework and addresses issues with previous population adjustment approaches. It coherently synthesizes evidence from IPD and AgD studies in networks of any size while avoiding aggregation bias and noncollapsibility bias, allows for key assumptions to be assessed or relaxed, and can produce estimates relevant to a target population for decision-making. HIGHLIGHTS: Multilevel network meta-regression (ML-NMR) extends the network meta-analysis framework to synthesize evidence from networks of studies providing individual patient data or aggregate data while adjusting for differences in effect modifiers between studies (population adjustment). We apply ML-NMR to a network of treatments for plaque psoriasis with ordered categorical outcomes.We demonstrate for the first time how ML-NMR allows key assumptions to be assessed. We check for violations of conditional constancy of relative effects (such as unobserved effect modifiers) through residual heterogeneity and inconsistency and the shared effect modifier assumption by relaxing this for each covariate in turn.Crucially for decision making, population-adjusted treatment effects can be produced in any relevant target population. We produce population-average estimates for 3 external target populations, represented by the PsoBest registry and the PROSPECT and Chiricozzi 2019 cohort studies.

Using individual participant data to improve network meta-analysis projects.

A network meta-analysis combines the evidence from existing randomised trials about the comparative efficacy of multiple treatments. It allows direct and indirect evidence about each comparison to be included in the same analysis, and provides a coherent framework to compare and rank treatments. A traditional network meta-analysis uses aggregate data (eg, treatment effect estimates and standard errors) obtained from publications or trial investigators. An alternative approach is to obtain, check, harmonise and meta-analyse the individual participant data (IPD) from each trial. In this article, we describe potential advantages of IPD for network meta-analysis projects, emphasising five key benefits: (1) improving the quality and scope of information available for inclusion in the meta-analysis, (2) examining and plotting distributions of covariates across trials (eg, for potential effect modifiers), (3) standardising and improving the analysis of each trial, (4) adjusting for prognostic factors to allow a network meta-analysis of conditional treatment effects and (5) including treatment-covariate interactions (effect modifiers) to allow relative treatment effects to vary by participant-level covariate values (eg, age, baseline depression score). A running theme of all these benefits is that they help examine and reduce heterogeneity (differences in the true treatment effect between trials) and inconsistency (differences in the true treatment effect between direct and indirect evidence) in the network. As a consequence, an IPD network meta-analysis has the potential for more precise, reliable and informative results for clinical practice and even allows treatment comparisons to be made for individual patients and targeted populations conditional on their particular characteristics.

Calibrating a network meta-analysis of diabetes trials of sodium glucose cotransporter 2 inhibitors, glucagon-like peptide-1 receptor analogues and dipeptidyl peptidase-4 inhibitors to a representative routine population: a systematic review protocol.

INTRODUCTION: Participants in randomised controlled trials (trials) are generally younger and healthier than many individuals encountered in clinical practice. Consequently, the applicability of trial findings is often uncertain. To address this, results from trials can be calibrated to more representative data sources. In a network meta-analysis, using a novel approach which allows the inclusion of trials whether or not individual-level participant data (IPD) is available, we will calibrate trials for three drug classes (sodium glucose cotransporter 2 (SGLT2) inhibitors, glucagon-like peptide-1 (GLP1) receptor analogues and dipeptidyl peptidase-4 (DPP4) inhibitors) to the Scottish diabetes register. METHODS AND ANALYSIS: Medline and EMBASE databases, the US clinical trials registry (clinicaltrials.gov) and the Chinese Clinical Trial Registry (chictr.org.cn) will be searched from 1 January 2002. Two independent reviewers will apply eligibility criteria to identify trials for inclusion. Included trials will be phase 3 or 4 trials of SGLT2 inhibitors, GLP1 receptor analogues or DPP4 inhibitors, with placebo or active comparators, in participants with type 2 diabetes, with at least one of glycaemic control, change in body weight or major adverse cardiovascular event as outcomes. Unregistered trials will be excluded.We have identified a target population from the population-based Scottish diabetes register. The chosen cohort comprises people in Scotland with type 2 diabetes who either (1) require further treatment due to poor glycaemic control where any of the three drug classes may be suitable, or (2) who have adequate glycaemic control but are already on one of the three drug classes of interest or insulin. ETHICS AND DISSEMINATION: Ethical approval for IPD use was obtained from the University of Glasgow MVLS College Ethics Committee (Project: 200160070). The Scottish diabetes register has approval from the Scottish A Research Ethics Committee (11/AL/0225) and operates with Public Benefit and Privacy Panel for Health and Social Care approval (1617-0147). PROSPERO REGISTRATION NUMBER: CRD42020184174.

A systematic review and network meta-analysis of topical pharmacological, oral pharmacological, physical and combined treatments for acne vulgaris.

BACKGROUND: Various treatments for acne vulgaris exist, but little is known about their comparative effectiveness in relation to acne severity. OBJECTIVES: To identify best treatments for mild-to-moderate and moderate-to-severe acne, as determined by clinician-assessed morphological features. METHODS: We undertook a systematic review and network meta-analysis of randomized controlled trials (RCTs) assessing topical pharmacological, oral pharmacological, physical and combined treatments for mild-to-moderate and moderate-to-severe acne, published up to May 2020. Outcomes included percentage change in total lesion count from baseline, treatment discontinuation for any reason, and discontinuation owing to side-effects. Risk of bias was assessed using the Cochrane risk-of-bias tool and bias adjustment models. Effects for treatments with ≥ 50 observations each compared with placebo are reported below. RESULTS: We included 179 RCTs with approximately 35 000 observations across 49 treatment classes. For mild-to-moderate acne, the most effective options for each treatment type were as follows: topical pharmacological - combined retinoid with benzoyl peroxide (BPO) [mean difference 26·16%, 95% credible interval (CrI) 16·75-35·36%]; physical - chemical peels, e.g. salicylic or mandelic acid (39·70%, 95% CrI 12·54-66·78%) and photochemical therapy (combined blue/red light) (35·36%, 95% CrI 17·75-53·08%). Oral pharmacological treatments (e.g. antibiotics, hormonal contraceptives) did not appear to be effective after bias adjustment. BPO and topical retinoids were less well tolerated than placebo. For moderate-to-severe acne, the most effective options for each treatment type were as follows: topical pharmacological - combined retinoid with lincosamide (clindamycin) (44·43%, 95% CrI 29·20-60·02%); oral pharmacological - isotretinoin of total cumulative dose ≥ 120 mg kg-1 per single course (58·09%, 95% CrI 36·99-79·29%); physical - photodynamic therapy (light therapy enhanced by a photosensitizing chemical) (40·45%, 95% CrI 26·17-54·11%); combined - BPO with topical retinoid and oral tetracycline (43·53%, 95% CrI 29·49-57·70%). Topical retinoids and oral tetracyclines were less well tolerated than placebo. The quality of included RCTs was moderate to very low, with evidence of inconsistency between direct and indirect evidence. Uncertainty in findings was high, in particular for chemical peels, photochemical therapy and photodynamic therapy. However, conclusions were robust to potential bias in the evidence. CONCLUSIONS: Topical pharmacological treatment combinations, chemical peels and photochemical therapy were most effective for mild-to-moderate acne. Topical pharmacological treatment combinations, oral antibiotics combined with topical pharmacological treatments, oral isotretinoin and photodynamic therapy were most effective for moderate-to-severe acne. Further research is warranted for chemical peels, photochemical therapy and photodynamic therapy for which evidence was more limited. What is already known about this topic? Acne vulgaris is the eighth most common disease globally. Several topical, oral, physical and combined treatments for acne vulgaris exist. Network meta-analysis (NMA) synthesizes direct and indirect evidence and allows simultaneous inference for all treatments forming an evidence network. Previous NMAs have assessed a limited range of treatments for acne vulgaris and have not evaluated effectiveness of treatments for moderate-to-severe acne. What does this study add? For mild-to-moderate acne, topical treatment combinations, chemical peels, and photochemical therapy (combined blue/red light; blue light) are most effective. For moderate-to-severe acne, topical treatment combinations, oral antibiotics combined with topical treatments, oral isotretinoin and photodynamic therapy (light therapy enhanced by a photosensitizing chemical) are most effective. Based on these findings, along with further clinical and cost-effectiveness considerations, National Institute for Health and Care Excellence (NICE) guidance recommends, as first-line treatments, fixed topical treatment combinations for mild-to-moderate acne and fixed topical treatment combinations, or oral tetracyclines combined with topical treatments, for moderate-to-severe acne.

Comparative clinical effectiveness and safety of tobacco cessation pharmacotherapies and electronic cigarettes: a systematic review and network meta-analysis of randomized controlled trials.

AIM: To determine how varenicline, bupropion, nicotine replacement therapy (NRT) and electronic cigarettes compare with respect to their clinical effectiveness and safety. METHOD: Systematic reviews and Bayesian network meta-analyses of randomized controlled trials, in any setting, of varenicline, bupropion, NRT and e-cigarettes (in high, standard and low doses, alone or in combination) in adult smokers and smokeless tobacco users with follow-up duration of 24 weeks or greater (effectiveness) or any duration (safety). Nine databases were searched until 19 February 2019. Primary outcomes were sustained tobacco abstinence and serious adverse events (SAEs). We estimated odds ratios (ORs) and treatment rankings and conducted meta-regression to explore covariates. RESULTS: We identified 363 trials for effectiveness and 355 for safety. Most monotherapies and combination therapies were more effective than placebo at helping participants to achieve sustained abstinence; the most effective of these, estimated with some imprecision, were varenicline standard [OR = 2.83, 95% credible interval (CrI) = 2.34-3.39] and varenicline standard + NRT standard (OR = 5.75, 95% CrI = 2.27-14.88). Estimates were higher in smokers receiving counselling than in those without and in studies with higher baseline nicotine dependence scores than in those with lower scores. Varenicline standard + NRT standard showed a high probability of being ranked best or second-best. For safety, only bupropion at standard dose increased the odds of experiencing SAEs compared with placebo (OR = 1.27, 95% CrI = 1.04-1.58), and we found no evidence of effect modification. CONCLUSIONS: Most tobacco cessation monotherapies and combination therapies are more effective than placebo at helping participants to achieve sustained abstinence, with varenicline appearing to be most effective based on current evidence. There does not appear to be strong evidence of associations between most tobacco cessation pharmacotherapies and adverse events; however, the data are limited and there is a need for improved reporting of safety data.