Hierarchical network meta-analysis models for synthesis of evidence from randomised and non-randomised studies.

BACKGROUND: With the increased interest in the inclusion of non-randomised data in network meta-analyses (NMAs) of randomised controlled trials (RCTs), analysts need to consider the implications of the differences in study designs as such data can be prone to increased bias due to the lack of randomisation and unmeasured confounding. This study aims to explore and extend a number of NMA models that account for the differences in the study designs, assessing their impact on the effect estimates and uncertainty. METHODS: Bayesian random-effects meta-analytic models, including naïve pooling and hierarchical models differentiating between the study designs, were extended to allow for the treatment class effect and accounting for bias, with further extensions allowing for bias terms to vary depending on the treatment class. Models were applied to an illustrative example in type 2 diabetes; using data from a systematic review of RCTs and non-randomised studies of two classes of glucose-lowering medications: sodium-glucose co-transporter 2 inhibitors and glucagon-like peptide-1 receptor agonists. RESULTS: Across all methods, the estimated mean differences in glycated haemoglobin after 24 and 52 weeks remained similar with the inclusion of observational data. The uncertainty around these estimates reduced when conducting naïve pooling, compared to NMA of RCT data alone, and remained similar when applying hierarchical model allowing for class effect. However, the uncertainty around these effect estimates increased when fitting hierarchical models allowing for the differences in study design. The impact on uncertainty varied between treatments when applying the bias adjustment models. Hierarchical models and bias adjustment models all provided a better fit in comparison to the naïve-pooling method. CONCLUSIONS: Hierarchical and bias adjustment NMA models accounting for study design may be more appropriate when conducting a NMA of RCTs and observational studies. The degree of uncertainty around the effectiveness estimates varied depending on the method but use of hierarchical models accounting for the study design resulted in increased uncertainty. Inclusion of non-randomised data may, however, result in inferences that are more generalisable and the models accounting for the differences in the study design allow for more detailed and appropriate modelling of complex data, preventing overly optimistic conclusions.

Double-counting of populations in evidence synthesis in public health: a call for awareness and future methodological development.

BACKGROUND: There is a growing interest in the inclusion of real-world and observational studies in evidence synthesis such as meta-analysis and network meta-analysis in public health. While this approach offers great epidemiological opportunities, use of such studies often introduce a significant issue of double-counting of participants and databases in a single analysis. Therefore, this study aims to introduce and illustrate the nuances of double-counting of individuals in evidence synthesis including real-world and observational data with a focus on public health. METHODS: The issues associated with double-counting of individuals in evidence synthesis are highlighted with a number of case studies. Further, double-counting of information in varying scenarios is discussed with potential solutions highlighted. RESULTS: Use of studies of real-world data and/or established cohort studies, for example studies evaluating the effectiveness of therapies using health record data, often introduce a significant issue of double-counting of individuals and databases. This refers to the inclusion of the same individuals multiple times in a single analysis. Double-counting can occur in a number of manners, such as, when multiple studies utilise the same database, when there is overlapping timeframes of analysis or common treatment arms across studies. Some common practices to address this include synthesis of data only from peer-reviewed studies, utilising the study that provides the greatest information (e.g. largest, newest, greater outcomes reported) or analysing outcomes at different time points. CONCLUSIONS: While common practices currently used can mitigate some of the impact of double-counting of participants in evidence synthesis including real-world and observational studies, there is a clear need for methodological and guideline development to address this increasingly significant issue.

Methods for the inclusion of real-world evidence in network meta-analysis.

BACKGROUND: Network Meta-Analysis (NMA) is a key component of submissions to reimbursement agencies world-wide, especially when there is limited direct head-to-head evidence for multiple technologies from randomised controlled trials (RCTs). Many NMAs include only data from RCTs. However, real-world evidence (RWE) is also becoming widely recognised as a valuable source of clinical data. This study aims to investigate methods for the inclusion of RWE in NMA and its impact on the level of uncertainty around the effectiveness estimates, with particular interest in effectiveness of fingolimod. METHODS: A range of methods for inclusion of RWE in evidence synthesis were investigated by applying them to an illustrative example in relapsing remitting multiple sclerosis (RRMS). A literature search to identify RCTs and RWE evaluating treatments in RRMS was conducted. To assess the impact of inclusion of RWE on the effectiveness estimates, Bayesian hierarchical and adapted power prior models were applied. The effect of the inclusion of RWE was investigated by varying the degree of down weighting of this part of evidence by the use of a power prior. RESULTS: Whilst the inclusion of the RWE led to an increase in the level of uncertainty surrounding effect estimates in this example, this depended on the method of inclusion adopted for the RWE. 'Power prior' NMA model resulted in stable effect estimates for fingolimod yet increasing the width of the credible intervals with increasing weight given to RWE data. The hierarchical NMA models were effective in allowing for heterogeneity between study designs, however, this also increased the level of uncertainty. CONCLUSION: The 'power prior' method for the inclusion of RWE in NMAs indicates that the degree to which RWE is taken into account can have a significant impact on the overall level of uncertainty. The hierarchical modelling approach further allowed for accommodating differences between study types. Consequently, further work investigating both empirical evidence for biases associated with individual RWE studies and methods of elicitation from experts on the extent of such biases is warranted.

Development, content validation, and reliability of the Assessment of Real-World Observational Studies (ArRoWS) critical appraisal tool.

PURPOSE: The objective was to develop and test a pragmatic critical appraisal tool, the Assessment of Real-World Observational Studies (ArRoWS), to quickly and easily assess the quality of real-world evidence studies using electronic health records. METHODS: The initial ArRoWS tool was developed by identifying items frequently found in existing validated assessment instruments and adapting these items to specifically assess real-world evidence studies. The tool was revised based on recommendations from an expert panel of 14 senior academic individuals specializing in epidemiology and content validity was measured. During March 2018-January 2019, 47 large, observational studies related to cardiometabolic medicine were identified through a search algorithm and assessed by three pairs of raters using the ArRoWS tool. RESULTS: The final version of the ArRoWS had 16 items including nine core items and seven study design-specific items with item-specific content validity indexes ranging from 0.64 to 1.00. The scale-level content validity index of the ArRoWS appraisal tool was 0.91. When the ArRoWS tool was pilot tested, the observed agreement between assessor pairs on whether the study provided high-quality real-world evidence was 85.7%, 68.8%, and 58.8%. The prevalence adjusted bias-adjusted kappa for the assessor pairs was 0.71, 0.38, and 0.18. CONCLUSION: The ArRoWS is a simple tool to standardize the assessment of real-world evidence studies.

Efficacy and tolerability of sodium-glucose co-transporter-2 inhibitors and glucagon-like peptide-1 receptor agonists: A systematic review and network meta-analysis.

AIM: To compare the efficacy and tolerability of sodium-glucose co-transporter 2 inhibitors (SGLT-2is) and glucagon-like peptide-1 receptor agonists (GLP-1RAs) in adults with type 2 diabetes. MATERIALS AND METHODS: Electronic databases were searched from inception to 24 April 2019 for randomized controlled trials reporting change in glycated haemoglobin (HbA1c) at approximately 24 and/or 52 weeks for SGLT-2is and/or GLP-1RAs (classified as short- and long-acting). Bayesian network meta-analyses were conducted to compare within and between SGLT-2i and GLP-1RA classes for cardiometabolic efficacy and adverse events (PROSPERO registration number: CRD42018091306). RESULTS: Sixty-four trials (53 trials of 24 weeks; seven trials of 52 weeks; four trials of both 24 and 52 weeks), comprising 31 384 participants were identified. Compared with placebo, all treatments improved HbA1c. Long-acting GLP-1RAs reduced HbA1c compared with short-acting GLP-1RAs and SGLT-2is, with semaglutide showing greater reduction compared with placebo [24 weeks: -1.49% (95% credible interval: -1.76, -1.22); 52 weeks: -1.38% (-2.05, -0.71)] and all other treatments. Long-acting GLP-1RAs showed benefits in body weight and waist circumference reduction, while SGLT-2is reduced blood pressure. SGLT-2is showed increased risk of genital infection in comparison with long-acting GLP-1RAs [odds ratio (95% credible interval): 5.26 (1.45, 25.00)], while GLP-1RAs showed increased risk of diarrhoea in comparison with SGLT-2is [short-acting GLP-1RAs: 1.65 (1.09, 2.49); long-acting GLP-1RAs: 2.23 (1.51, 3.28)]. No other differences were found between SGLT-2is and GLP-1RAs in adverse events. CONCLUSION: Long-acting GLP-1RAs showed superiority in reducing HbA1c levels, body weight and waist circumference. SGLT-2is showed reductions in blood pressure levels. This review provides essential evidence to guide treatment recommendations in the management of type 2 diabetes.

Efficacy and safety of sodium-glucose cotransporter 2 inhibitors (SGLT-2is) and glucagon-like peptide-1 receptor agonists (GLP-1RAs) in patients with type 2 diabetes: a systematic review and network meta-analysis study protocol.

INTRODUCTION: Sodium-glucose cotransporter 2 inhibitors (SGLT-2is) and glucagon-like peptide-1 receptor agonists (GLP-1RAs) are two classes of glucose-lowering drugs gaining popularity in the treatment of type 2 diabetes mellitus (T2DM). Current guidelines suggest patient-centred approaches when deciding between available hyperglycaemia drugs with no indication to which specific drug should be administered. Despite systematic reviews and meta-analyses being conducted within SGLT-2is and GLP-1RAs, differences across these classes of drugs have not been investigated. Therefore, this systematic review and network meta-analysis (NMA) will aim to compare the efficacy and safety profiles across and within SGLT-2is and GLP-1RAs. METHODS: PubMed, the Cochrane Central Register of Controlled Trials and ISI Web of Science will be searched from inception for published randomised controlled trials conducted in patients with T2DM, with at least two arms consisting of SGLT-2is, GLP-1RAs or control/placebo. Title and abstracts will be screened by two independent reviewers with conflicts resolved by a third. Data will be extracted by the primary researcher, a random sample will be checked by an independent reviewer. Risk of bias will be assessed using the Cochrane Risk of Bias Tool and overall quality of evidence will be assessed using the Grading of Recommendations Assessment, Development and Evaluation approach.Study characteristics, participants baseline characteristics, mean change in cardiometabolic outcomes and number of adverse events will be extracted for each study. Primary outcome will be the mean change in glycated haemoglobin (HbA1c) (%, mmol/mol). Initial random-effects pairwise meta-analysis will be conducted for each unique treatment comparison where heterogeneity will be assessed. A Bayesian NMA approach will be adopted where random-effects generalised linear models will be fitted in WinBUGS. Sensitivity analysis will be conducted to assess choices of prior distributions and length of burn-in and sample. ETHICS AND DISSEMINATION: Ethics approval is not required for this study. Results from this study will be published in a peer-review journal. PROSPERO REGISTRATION NUMBER: CRD42018091306.