Grey literature in systematic reviews: a cross-sectional study of the contribution of non-English reports, unpublished studies and dissertations to the results of meta-analyses in child-relevant reviews.

BACKGROUND: Systematic reviews (SRs) are an important source of information about healthcare interventions. A key component of a well-conducted SR is a comprehensive literature search. There is limited evidence on the contribution of non-English reports, unpublished studies, and dissertations and their impact on results of meta-analyses. METHODS: Our sample included SRs from three Cochrane Review Groups: Acute Respiratory Infections (ARI), Infectious Diseases (ID), Developmental Psychosocial and Learning Problems (DPLP) (n = 129). Outcomes included: 1) proportion of reviews that searched for and included each study type; 2) proportion of relevant studies represented by each study type; and 3) impact on results and conclusions of the primary meta-analysis for each study type. RESULTS: Most SRs searched for non-English studies; however, these were included in only 12% of reviews and represented less than 5% of included studies. There was a change in results in only four reviews (total sample = 129); in two cases the change did not have an impact on the statistical or clinical significance of results. Most SRs searched for unpublished studies but the majority did not include these (only 6%) and they represented 2% of included studies. In most cases the impact of including unpublished studies was small; a substantial impact was observed in one case that relied solely on unpublished data. Few reviews in ARI (9%) and ID (3%) searched for dissertations compared to 65% in DPLP. Overall, dissertations were included in only nine SRs and represented less than 2% of included studies. In the majority of cases the change in results was negligible or small; in the case where a large change was noted, the estimate was more conservative without dissertations. CONCLUSIONS: The majority of SRs searched for non-English and unpublished studies; however, these represented a small proportion of included studies and rarely impacted the results and conclusions of the review. Inclusion of these study types may have an impact in situations where there are few relevant studies, or where there are questionable vested interests in the published literature. We found substantial variation in whether SRs searched for dissertations; in most reviews that included dissertations, these had little impact on results.

The contribution of databases to the results of systematic reviews: a cross-sectional study.

BACKGROUND: One of the best sources for high quality information about healthcare interventions is a systematic review. A well-conducted systematic review includes a comprehensive literature search. There is limited empiric evidence to guide the extent of searching, in particular the number of electronic databases that should be searched. We conducted a cross-sectional quantitative analysis to examine the potential impact of selective database searching on results of meta-analyses. METHODS: Our sample included systematic reviews (SRs) with at least one meta-analysis from three Cochrane Review Groups: Acute Respiratory Infections (ARI), Infectious Diseases (ID), Developmental Psychosocial and Learning Problems (DPLP) (n = 129). Outcomes included: 1) proportion of relevant studies indexed in each of 10 databases; and 2) changes in results and statistical significance of primary meta-analysis for studies identified in Medline only and in Medline plus each of the other databases. RESULTS: Due to variation across topics, we present results by group (ARI n = 57, ID n = 38, DPLP n = 34). For ARI, identification of relevant studies was highest for Medline (85 %) and Embase (80 %). Restricting meta-analyses to trials that appeared in Medline + Embase yielded fewest changes in statistical significance: 53/55 meta-analyses showed no change. Point estimates changed in 12 cases; in 7 the change was less than 20 %. For ID, yield was highest for Medline (92 %), Embase (81 %), and BIOSIS (67 %). Restricting meta-analyses to trials that appeared in Medline + BIOSIS yielded fewest changes with 1 meta-analysis changing in statistical significance. Point estimates changed in 8 of 31 meta-analyses; change less than 20 % in all cases. For DPLP, identification of relevant studies was highest for Medline (75 %) and Embase (62 %). Restricting meta-analyses to trials that appeared in Medline + PsycINFO resulted in only one change in significance. Point estimates changed for 13 of 33 meta-analyses; less than 20 % in 9 cases. CONCLUSIONS: Majority of relevant studies can be found within a limited number of databases. Results of meta-analyses based on the majority of studies did not differ in most cases. There were very few cases of changes in statistical significance. Effect estimates changed in a minority of meta-analyses but in most the change was small. Results did not change in a systematic manner (i.e., regularly over- or underestimating treatment effects), suggesting that selective searching may not introduce bias in terms of effect estimates.

How Safe Are Common Analgesics for the Treatment of Acute Pain for Children? A Systematic Review.

Background. Fear of adverse events and occurrence of side effects are commonly cited by families and physicians as obstructive to appropriate use of pain medication in children. We examined evidence comparing the safety profiles of three groups of oral medications, acetaminophen, nonsteroidal anti-inflammatory drugs, and opioids, to manage acute nonsurgical pain in children (<18 years) treated in ambulatory settings. Methods. A comprehensive search was performed to July 2015, including review of national data registries. Two reviewers screened articles for inclusion, assessed methodological quality, and extracted data. Risks (incidence rates) were pooled using a random effects model. Results. Forty-four studies were included; 23 reported on adverse events. Based on limited current evidence, acetaminophen, ibuprofen, and opioids have similar nausea and vomiting profiles. Opioids have the greatest risk of central nervous system adverse events. Dual therapy with a nonopioid/opioid combination resulted in a lower risk of adverse events than opioids alone. Conclusions. Ibuprofen and acetaminophen have similar reported adverse effects and notably less adverse events than opioids. Dual therapy with a nonopioid/opioid combination confers a protective effect for adverse events over opioids alone. This research highlights challenges in assessing medication safety, including lack of more detailed information in registry data, and inconsistent reporting in trials.

Behavioral Programs for Type 1 Diabetes Mellitus: A Systematic Review and Meta-analysis.

BACKGROUND: Whether behavioral approaches for self-management programs benefit individuals with type 1 diabetes mellitus is unclear. PURPOSE: To determine the effects of behavioral programs for patients with type 1 diabetes on behavioral, clinical, and health outcomes and to investigate factors that might moderate effect. DATA SOURCES: 6 electronic databases (1993 to June 2015), trial registries and conference proceedings (2011 to 2014), and reference lists. STUDY SELECTION: 36 prospective, controlled studies involving participants of any age group that compared behavioral programs with usual care, active controls, or other programs. DATA EXTRACTION: One reviewer extracted and another verified data. Two reviewers assessed quality and strength of evidence (SOE). DATA SYNTHESIS: Moderate SOE showed reduction in glycated hemoglobin (HbA1c) at 6 months after the intervention compared with usual care (mean difference, -0.29 [95% CI, -0.45 to -0.13] percentage points) and compared with active controls (-0.44 [CI, -0.69 to -0.19] percentage points). At the end of the intervention and 12-month follow-up or longer, there were no statistically significant differences in HbA1c (low SOE) for comparisons with usual care or active control. Compared with usual care, generic quality of life at program completion did not differ (moderate SOE). Other outcomes had low or insufficient SOE. Adults appeared to benefit more for glycemic control at program completion (-0.28 [CI, -0.57 to 0.01] percentage points) than did youth (-0.12 [CI, -0.43 to 0.19] percentage points). Program intensity appeared not to influence effectiveness; some individual delivery appears beneficial. LIMITATIONS: All studies had medium or high risk of bias. There was scarce evidence for many outcomes. CONCLUSION: Behavioral programs for type 1 diabetes offer some benefit for glycemic control, at least at short-term follow-up, but improvement for other outcomes has not been shown. (PROSPERO registration number: CRD42014010515). PRIMARY FUNDING SOURCE: Agency for Healthcare Research and Quality. (PROSPERD registration number: CRD42014010515).

Behavioral Programs for Type 2 Diabetes Mellitus: A Systematic Review and Network Meta-analysis.

BACKGROUND: Behavioral programs may improve outcomes for individuals with type 2 diabetes mellitus, but there is a large diversity of behavioral interventions and uncertainty about how to optimize the effectiveness of these programs. PURPOSE: To identify factors moderating the effectiveness of behavioral programs for adults with type 2 diabetes. DATA SOURCES: 6 databases (1993 to January 2015), conference proceedings (2011 to 2014), and reference lists. STUDY SELECTION: Duplicate screening and selection of 132 randomized, controlled trials evaluating behavioral programs compared with usual care, active controls, or other behavioral programs. DATA EXTRACTION: One reviewer extracted and another verified data. Two reviewers independently assessed risk of bias. DATA SYNTHESIS: Behavioral programs were grouped on the basis of program content and delivery methods. A Bayesian network meta-analysis showed that most lifestyle and diabetes self-management education and support programs (usually offering ≥ 11 contact hours) led to clinically important improvements in glycemic control (≥ 0.4% reduction in hemoglobin A1c [HbA1c]), whereas most diabetes self-management education programs without added support-especially those offering 10 or fewer contact hours-provided little benefit. Programs with higher effect sizes were more often delivered in person than via technology. Lifestyle programs led to the greatest reductions in body mass index. Reductions in HbA1c seemed to be greater for participants with a baseline HbA1c level of 7.0% or greater, adults younger than 65 years, and minority persons (subgroups with ≥ 75% nonwhite participants). LIMITATIONS: All trials had medium or high risk of bias. Subgroup analyses were indirect, and therefore exploratory. Most outcomes were reported immediately after the interventions. CONCLUSION: Diabetes self-management education offering 10 or fewer hours of contact with delivery personnel provided little benefit. Behavioral programs seem to benefit persons with suboptimal or poor glycemic control more than those with good control. PRIMARY FUNDING SOURCE: Agency for Healthcare Research and Quality. (PROSPERO registration number: CRD42014010515).

A taxonomy of rapid reviews links report types and methods to specific decision-making contexts.

OBJECTIVES: Describe characteristics of rapid reviews and examine the impact of methodological variations on their reliability and validity. STUDY DESIGN AND SETTING: We conducted a literature review and interviews with organizations that produce rapid reviews or related products to identify methods, guidance, empiric evidence, and current practices. RESULTS: We identified 36 rapid products from 20 organizations (production time, 5 minutes to 8 months). Methods differed from systematic reviews at all stages. As time frames increased, methods became more rigorous; however, restrictions on database searching, inclusion criteria, data extracted, and independent dual review remained. We categorized rapid products based on extent of synthesis. "Inventories" list what evidence is available. "Rapid responses" present best available evidence with no formal synthesis. "Rapid reviews" synthesize the quality of and findings from the evidence. "Automated approaches" generate meta-analyses in response to user-defined queries. Rapid products rely on a close relationship with end users and support specific decisions in an identified time frame. Limited empiric evidence exists comparing rapid and systematic reviews. CONCLUSIONS: Rapid products have tremendous methodological variation; categorization based on time frame or type of synthesis reveals patterns. The similarity across rapid products lies in the close relationship with the end user to meet time-sensitive decision-making needs.

Advancing knowledge of rapid reviews: an analysis of results, conclusions and recommendations from published review articles examining rapid reviews.

BACKGROUND: Rapid review (RR) products are inherently appealing as they are intended to be less time-consuming and resource-intensive than traditional systematic reviews (SRs); however, there is concern about the rigor of methods and reliability of results. In 2013 to 2014, a workgroup comprising representatives from the Agency for Healthcare Research and Quality's Evidence-based Practice Center Program conducted a formal evaluation of RRs. This paper summarizes results, conclusions, and recommendations from published review articles examining RRs. METHODS: A systematic literature search was conducted and publications were screened independently by two reviewers. Twelve review articles about RRs were identified. One investigator extracted data about RR methods and how they compared with standard SRs. A narrative summary is presented. RESULTS: A cross-comparison of review articles revealed the following: 1) ambiguous definitions of RRs, 2) varying timeframes to complete RRs ranging from 1 to 12 months, 3) limited scope of RR questions, and 4) significant heterogeneity between RR methods. CONCLUSIONS: RR definitions, methods, and applications vary substantially. Published review articles suggest that RRs should not be viewed as a substitute for a standard SR, although they have unique value for decision-makers. Recommendations for RR producers include transparency of methods used and the development of reporting standards.

A meta-epidemiological study to examine the association between bias and treatment effects in neonatal trials.

BACKGROUND: Randomized controlled trials are considered the gold standard for evidence on therapeutic interventions; however, they are susceptible to bias. The objectives of this observational study were to describe the methodological quality of neonatal randomized controlled trials and quantify the bias related to specific methodological and study-level characteristics. METHODS: Twenty-five systematic reviews yielding 208 neonatal trials were included. Two independent reviewers assessed risk of bias (RoB) on seven domains consisting of nine items. For each domain, meta-analyses with at least one high/unclear and one low risk study were included in the analysis. For the primary outcome within each meta-analysis a ratio of odds ratios with a 95% confidence interval was generated. The ratio of odds ratios for each meta-analysis were combined using meta-analytic techniques with inverse-variance weighting and a random effects model to obtain a summary ratio of odds ratio. RESULTS: None of the studies had an overall low RoB. Most studies had a low RoB for the domain of incomplete outcome data (89%), while 63%, 55% and 46% of trials had low RoB for sequence generation, other sources of bias, and blinding of outcome assessors, respectively. For all other domains (allocation concealment, blinding of parents and investigators and selective outcome reporting), the majority of trials were assessed as unclear. Selective outcome reporting was rated as unclear RoB for 55% and high for 42% of studies. The only domain that showed a statistically significant association with the treatment effect was selective outcome reporting: trials at unclear/high risk of bias for this domain significantly overestimated the treatment effects compared with those assessed at low risk of bias (ROR = 1.87, 95% confidence interval: 1.26-2.78). CONCLUSIONS: This observational study of a sample of neonatal trials showed that most were at high risk of bias, indicating that there is room for improvement in the design, conduct and reporting of neonatal trials to ensure valid results for the most clinically important outcomes. We did not find an association between most risk of bias domains and effect estimates; however, we found that randomized controlled trials at high risk for selective outcome reporting were associated with overestimates of treatment benefits. These results need to be confirmed in larger samples.

The role of biological agents in the management of large vessel vasculitis (LVV): a systematic review and meta-analysis.

BACKGROUND: Giant cell arteritis (GCA) and Takayasu's arteritis (TAA) are large vessel vasculitides (LVV) for which corticosteroids (CS) are the mainstay for treatment. In patients with LVV unable to tolerate CS, biological agents have been used with variable effectiveness. OBJECTIVE: To systematically review the effectiveness and safety of biological agents in patients with LVV. METHODS: We searched 5 electronic databases (inception to October 2012) and conference abstracts with no language restrictions. Two reviewers independently selected studies, extracted data and assessed methodological quality. Our protocol was registered in PROSPERO. RESULTS: We included 25 studies (3 RCTs and 22 case series with ≥2 cases). 95 GCA and 98 TAA patients received biological agents. The RCTs using anti-TNF agents (infliximab, etanercept and adalimumab) did not suggest a benefit in GCA. GCA patients receiving tocilizumab, in case series, achieved remission (19 patients) and reduction of corticosteroid dose (mean difference, -16.55 mg/day (95% CI: -26.24, -6.86)). In case series, 75 patients with refractory TAA treated with infliximab discontinued CS 32% of the time. Remission was variably defined and the studies were clinically heterogeneous which precluded further analysis. CONCLUSION: This systematic review demonstrated a weak evidence base on which to assess the effectiveness of biological treatment in LVV. Evidence from RCTs suggests that anti-TNF agents are not effective for remission or reduction of CS use. Tocilizumab and infliximab may be effective in the management of LVV and refractory TAA, respectively, although the evidence comes from case series. Future analytical studies are needed to confirm these findings.

The value of including observational studies in systematic reviews was unclear: a descriptive study.

OBJECTIVES: To evaluate (1) how often observational studies are included in comparative effectiveness reviews (CERs); (2) the rationale for including observational studies; (3) how data from observational studies are appraised, analyzed, and graded; and (4) the impact of observational studies on strength of evidence (SOE) and conclusions. STUDY DESIGN AND SETTING: Descriptive study of 23 CERs published through the Effective Health Care Program of the U.S. Agency for Healthcare Research and Quality. RESULTS: Authors searched for observational studies in 20 CERs, of which 18 included a median of 11 (interquartile range, 2-31) studies. Sixteen CERs incorporated the observational studies in their SOE assessments. Seventy-eight comparisons from 12 CERs included evidence from both trials and observational studies; observational studies had an impact on SOE and conclusions for 19 (24%) comparisons. There was diversity across the CERs regarding decisions to include observational studies; study designs considered; and approaches used to appraise, synthesize, and grade SOE. CONCLUSION: Reporting and methods guidance are needed to ensure clarity and consistency in how observational studies are incorporated in CERs. It was not always clear that observational studies added value in light of the additional resources needed to search for, select, appraise, and analyze such studies.

Integration of existing systematic reviews into new reviews: identification of guidance needs.

BACKGROUND: An exponential increase in the number of systematic reviews published, and constrained resources for new reviews, means that there is an urgent need for guidance on explicitly and transparently integrating existing reviews into new systematic reviews. The objectives of this paper are: 1) to identify areas where existing guidance may be adopted or adapted, and 2) to suggest areas for future guidance development. METHODS: We searched documents and websites from healthcare focused systematic review organizations to identify and, where available, to summarize relevant guidance on the use of existing systematic reviews. We conducted informational interviews with members of Evidence-based Practice Centers (EPCs) to gather experiences in integrating existing systematic reviews, including common issues and challenges, as well as potential solutions. RESULTS: There was consensus among systematic review organizations and the EPCs about some aspects of incorporating existing systematic reviews into new reviews. Current guidance may be used in assessing the relevance of prior reviews and in scanning references of prior reviews to identify studies for a new review. However, areas of challenge remain. Areas in need of guidance include how to synthesize, grade the strength of, and present bodies of evidence composed of primary studies and existing systematic reviews. For instance, empiric evidence is needed regarding how to quality check data abstraction and when and how to use study-level risk of bias assessments from prior reviews. CONCLUSIONS: There remain areas of uncertainty for how to integrate existing systematic reviews into new reviews. Methods research and consensus processes among systematic review organizations are needed to develop guidance to address these challenges.

Quantifying bias in randomized controlled trials in child health: a meta-epidemiological study.

OBJECTIVE: To quantify bias related to specific methodological characteristics in child-relevant randomized controlled trials (RCTs). DESIGN: Meta-epidemiological study. DATA SOURCES: We identified systematic reviews containing a meta-analysis with 10-40 RCTs that were relevant to child health in the Cochrane Database of Systematic Reviews. DATA EXTRACTION: Two reviewers independently assessed RCTs using items in the Cochrane Risk of Bias tool and other study factors. We used meta-epidemiological methods to assess for differences in effect estimates between studies classified as high/unclear vs. low risk of bias. RESULTS: We included 287 RCTs from 17 meta-analyses. The proportion of studies at high/unclear risk of bias was: 79% sequence generation, 83% allocation concealment, 67% blinding of participants, 47% blinding of outcome assessment, 49% incomplete outcome data, 32% selective outcome reporting, 44% other sources of bias, 97% overall risk of bias, 56% funding, 35% baseline imbalance, 13% blocked randomization in unblinded trials, and 1% early stopping for benefit. We found no significant differences in effect estimates for studies that were high/unclear vs. low risk of bias for any of the risk of bias domains, overall risk of bias, or other study factors. CONCLUSIONS: We found no differences in effect estimates between studies based on risk of bias. A potential explanation is the number of trials included, in particular the small number of studies with low risk of bias. Until further evidence is available, reviewers should not exclude RCTs from systematic reviews and meta-analyses based solely on risk of bias particularly in the area of child health.

Lifestyle interventions for patients with and at risk for type 2 diabetes: a systematic review and meta-analysis.

BACKGROUND: The effect of multifaceted lifestyle interventions on clinically oriented outcomes across a spectrum of metabolic risk factors and abnormal glucose is unclear. PURPOSE: To systematically review the effectiveness of lifestyle interventions on minimizing progression to diabetes in high-risk patients or progression to clinical outcomes (such as cardiovascular disease and death) in patients with type 2 diabetes. DATA SOURCES: 5 electronic databases (1980 to June 2013), reference lists, and gray literature. STUDY SELECTION: Two reviewers independently identified randomized, controlled trials of lifestyle interventions (≥3 months' duration) that included exercise, diet, and at least 1 other component; the comparator was standard care. DATA EXTRACTION: One reviewer extracted and a second verified data. Two reviewers independently assessed methodological quality. DATA SYNTHESIS: Nine randomized, controlled trials with patients who were at risk for diabetes and 11 with patients who had diabetes were included. Seven studies reported that lifestyle interventions decreased the risk for diabetes from the end of intervention up to 10 years after it. In patients with diabetes, 2 randomized, controlled trials (which included pharmacotherapy) reported no improvement in all-cause mortality (risk ratio, 0.75 [95% CI, 0.53 to 1.06]). Composite outcomes for cardiovascular disease were too heterogeneous to pool. One trial reported improvement in microvascular outcomes at 13-year follow-up. LIMITATION: Most trials focused on surrogate measures (such as weight change, blood pressure, and lipids) for which clinical relevance was unclear. CONCLUSION: Comprehensive lifestyle interventions effectively decrease the incidence of type 2 diabetes in high-risk patients. In patients who already have type 2 diabetes, there is no evidence of reduced all-cause mortality and insufficient evidence to suggest benefit on cardiovascular and microvascular outcomes. PRIMARY FUNDING SOURCE: Agency for Healthcare Research and Quality.

Screening tests for gestational diabetes: a systematic review for the U.S. Preventive Services Task Force.

BACKGROUND: A 50-g oral glucose challenge test (OGCT) is a widely accepted screening method for gestational diabetes mellitus (GDM), but other options are being considered. PURPOSE: To systematically review the test characteristics of various screening methods for GDM across a range of recommended diagnostic glucose thresholds. DATA SOURCES: 15 electronic databases from 1995 to May 2012, reference lists, Web sites of relevant organizations, and gray literature. STUDY SELECTION: Two reviewers independently identified English-language prospective studies that compared any screening test for GDM with any reference standard. DATA EXTRACTION: One reviewer extracted and a second reviewer verified data from 51 cohort studies. Two reviewers independently assessed methodological quality. DATA SYNTHESIS: The sensitivity, specificity, and positive and negative likelihood ratios for the OGCT at a threshold of 7.8 mmol/L (140 mg/dL) were 70% to 88%, 69% to 89%, 2.6 to 6.5, and 0.16 to 0.33, respectively. At a threshold of 7.2 mmol/L (130 mg/dL), the test characteristics were 88% to 99%, 66% to 77%, 2.7 to 4.2, and 0.02 to 0.14, respectively. For a fasting plasma glucose threshold of 4.7 mmol/L (85 mg/dL), they were 87%, 52%, 1.8, and 0.25, respectively. Glycated hemoglobin level had poorer test characteristics than fasting plasma glucose level or the OGCT. No studies compared the OGCT with International Association of the Diabetes and Pregnancy Study Groups (IADPSG) diagnostic criteria. LIMITATIONS: The lack of a gold standard for confirming GDM limits comparisons. Few data exist for screening tests before 24 weeks' gestation. CONCLUSION: The OGCT and fasting plasma glucose level (at a threshold of 4.7 mmol/L [85 mg/dL]) by 24 weeks' gestation are good at identifying women who do not have GDM. The OGCT is better at identifying women who have GDM. The OGCT has not been validated for the IADPSG diagnostic criteria.

Benefits and harms of treating gestational diabetes mellitus: a systematic review and meta-analysis for the U.S. Preventive Services Task Force and the National Institutes of Health Office of Medical Applications of Research.

BACKGROUND: Outcomes of treating gestational diabetes mellitus (GDM) are not well-established. PURPOSE: To summarize evidence about the maternal and neonatal benefits and harms of treating GDM. DATA SOURCES: 15 electronic databases from 1995 to May 2012, gray literature, Web sites of relevant organizations, trial registries, and reference lists. STUDY SELECTION: English-language randomized, controlled trials (n = 5) and cohort studies (n = 6) of women without known preexisting diabetes. DATA EXTRACTION: One reviewer extracted data, and a second reviewer verified them. Two reviewers independently assessed methodological quality and evaluated strength of evidence for primary outcomes by using a Grading of Recommendations Assessment, Development and Evaluation approach. DATA SYNTHESIS: All studies compared diet modification, glucose monitoring, and insulin as needed with no treatment. Women who were treated had more prenatal visits than those in control groups. Moderate evidence showed fewer cases of preeclampsia, shoulder dystocia, and macrosomia in the treated group. Evidence was insufficient for maternal weight gain and birth injury. Low evidence showed no difference between groups for neonatal hypoglycemia. Evidence was insufficient for long-term metabolic outcomes among offspring. No difference was found for cesarean delivery (low evidence), induction of labor (insufficient evidence), small-for-gestational-age neonates (moderate evidence), or admission to a neonatal intensive care unit (low evidence). LIMITATIONS: Evidence is low or insufficient for many outcomes of greatest clinical importance. The strongest evidence supports reductions in intermediate outcomes; however, other factors (for example, maternal weight and gestational weight gain) may impart greater risk than GDM, particularly when glucose levels are modestly elevated. CONCLUSION: Treating GDM results in less preeclampsia, shoulder dystocia, and macrosomia; however, current evidence does not show an effect on neonatal hypoglycemia or future poor metabolic outcomes. There is little evidence of short-term harm of treating GDM other than an increased demand for services.

Testing the Newcastle Ottawa Scale showed low reliability between individual reviewers.

OBJECTIVES: To assess inter-rater reliability and validity of the Newcastle Ottawa Scale (NOS) used for methodological quality assessment of cohort studies included in systematic reviews. STUDY DESIGN AND SETTING: Two reviewers independently applied the NOS to 131 cohort studies included in eight meta-analyses. Inter-rater reliability was calculated using kappa (κ) statistics. To assess validity, within each meta-analysis, we generated a ratio of pooled estimates for each quality domain. Using a random-effects model, the ratios of odds ratios for each meta-analysis were combined to give an overall estimate of differences in effect estimates. RESULTS: Inter-rater reliability varied from substantial for length of follow-up (κ = 0.68, 95% confidence interval [CI] = 0.47, 0.89) to poor for selection of the nonexposed cohort and demonstration that the outcome was not present at the outset of the study (κ = -0.03, 95% CI = -0.06, 0.00; κ = -0.06, 95% CI = -0.20, 0.07). Reliability for overall score was fair (κ = 0.29, 95% CI = 0.10, 0.47). In general, reviewers found the tool difficult to use and the decision rules vague even with additional information provided as part of this study. We found no association between individual items or overall score and effect estimates. CONCLUSION: Variable agreement and lack of evidence that the NOS can identify studies with biased results underscore the need for revisions and more detailed guidance for systematic reviewers using the NOS.

Efficacy of supplemental peripheral nerve blockade for hip fracture surgery: multiple treatment comparison.

PURPOSE: This study was designed to determine the most effective peripheral nerve block supplement to standard anesthesia management for hip fracture patients. METHODS: We systematically reviewed randomized controlled trials (RCTs) published from 1990 to 2010 and conducted multiple treatment comparisons using direct and indirect evidence for two outcomes, i.e., acute pain intensity and delirium. We combined trials by type of injection (regardless of time of insertion during the perioperative phase, use of nerve block catheter, local anesthetic type, additives, or duration of treatment). RESULTS: Twenty-one RCTs comprising 1,422 participants were included. In most cases, the trials were conducted in Europe; they excluded patients with cognitive impairment, and they were unclear or at high risk of bias. The combination of obturator and lateral femoral cutaneous nerve blockade had the highest probability of being the most effective against acute postoperative pain. Fascia iliaca blockade had the highest probability of being the most effective against delirium. There was no difference in outcomes among other nerve blocks. CONCLUSION: Not all nerve blocks are equally effective in improving outcomes after hip fracture. Multiple treatment comparison, a tool to compare the effectiveness of multiple treatments simultaneously, provides useful guidance to anesthesia providers seeking effective treatment when faced with a body of RCTs wherein each investigates one treatment. More RCTs comparing multiple nerve blocks in hip fracture are needed.

Testing the risk of bias tool showed low reliability between individual reviewers and across consensus assessments of reviewer pairs.

OBJECTIVES: To assess the reliability of the Cochrane Risk of Bias (ROB) tool between individual raters and across consensus agreements of pairs of reviewers and examine the impact of study-level factors on reliability. STUDY DESIGN AND SETTING: Two reviewers assessed risk of bias for 154 randomized controlled trials (RCTs). For 30 RCTs, two reviewers from each of four centers assessed risk of bias and reached consensus. We assessed interrater agreement using kappas and the impact of study-level factors through subgroup analyses. RESULTS: Reliability between two reviewers was fair for most domains (κ=0.24-0.37), except sequence generation (κ=0.79, substantial). Reliability results across reviewer pairs: sequence generation, moderate (κ=0.60); allocation concealment and "other sources of bias," fair (κ=0.37-0.27); and other domains, slight (κ=0.05-0.09). Reliability was influenced by the nature of the outcome, nature of the intervention, study design, trial hypothesis, and funding source. Variability resulted from different interpretation of the tool rather than different information identified in the study reports. CONCLUSION: Low agreement has implications for interpreting systematic reviews. These findings suggest the need for detailed guidance in assessing the risk of bias.

A descriptive analysis of overviews of reviews published between 2000 and 2011.

BACKGROUND: Overviews of systematic reviews compile data from multiple systematic reviews (SRs) and are a new method of evidence synthesis. OBJECTIVES: To describe the methodological approaches in overviews of interventions. DESIGN: Descriptive study. METHODS: We searched 4 databases from 2000 to July 2011; we handsearched Evidence-based Child Health: A Cochrane Review Journal. We defined an overview as a study that: stated a clear objective; examined an intervention; used explicit methods to identify SRs; collected and synthesized outcome data from the SRs; and intended to include only SRs. We did not restrict inclusion by population characteristics (e.g., adult or children only). Two researchers independently screened studies and applied eligibility criteria. One researcher extracted data with verification by a second. We conducted a descriptive analysis. RESULTS: From 2,245 citations, 75 overviews were included. The number of overviews increased from 1 in 2000 to 14 in 2010. The interventions were pharmacological (n = 20, 26.7%), non-pharmacological (n = 26, 34.7%), or both (n = 29, 38.7%). Inclusion criteria were clearly stated in 65 overviews. Thirty-three (44%) overviews searched at least 2 databases. The majority reported the years and databases searched (n = 46, 61%), and provided key words (n = 58, 77%). Thirty-nine (52%) overviews included Cochrane SRs only. Two reviewers independently screened and completed full text review in 29 overviews (39%). Methods of data extraction were reported in 45 (60%). Information on quality of individual studies was extracted from the original SRs in 27 (36%) overviews. Quality assessment of the SRs was performed in 28 (37%) overviews; at least 9 different tools were used. Quality of the body of evidence was assessed in 13 (17%) overviews. Most overviews provided a narrative or descriptive analysis of the included SRs. One overview conducted indirect analyses and the other conducted mixed treatment comparisons. Publication bias was discussed in 18 (24%) overviews. CONCLUSIONS: This study shows considerable variation in the methods used for overviews. There is a need for methodological rigor and consistency in overviews, as well as empirical evidence to support the methods employed.

Systematic review of knowledge translation strategies in the allied health professions.

BACKGROUND: Knowledge translation (KT) aims to close the research-practice gap in order to realize and maximize the benefits of research within the practice setting. Previous studies have investigated KT strategies in nursing and medicine; however, the present study is the first systematic review of the effectiveness of a variety of KT interventions in five allied health disciplines: dietetics, occupational therapy, pharmacy, physiotherapy, and speech-language pathology. METHODS: A health research librarian developed and implemented search strategies in eight electronic databases (MEDLINE, CINAHL, ERIC, PASCAL, EMBASE, IPA, Scopus, CENTRAL) using language (English) and date restrictions (1985 to March 2010). Other relevant sources were manually searched. Two reviewers independently screened the titles and abstracts, reviewed full-text articles, performed data extraction, and performed quality assessment. Within each profession, evidence tables were created, grouping and analyzing data by research design, KT strategy, targeted behaviour, and primary outcome. The published descriptions of the KT interventions were compared to the Workgroup for Intervention Development and Evaluation Research (WIDER) Recommendations to Improve the Reporting of the Content of Behaviour Change Interventions. RESULTS: A total of 2,638 articles were located and the titles and abstracts were screened. Of those, 1,172 full-text articles were reviewed and subsequently 32 studies were included in the systematic review. A variety of single (n = 15) and multiple (n = 17) KT interventions were identified, with educational meetings being the predominant KT strategy (n = 11). The majority of primary outcomes were identified as professional/process outcomes (n = 25); however, patient outcomes (n = 4), economic outcomes (n = 2), and multiple primary outcomes (n = 1) were also represented. Generally, the studies were of low methodological quality. Outcome reporting bias was common and precluded clear determination of intervention effectiveness. In the majority of studies, the interventions demonstrated mixed effects on primary outcomes, and only four studies demonstrated statistically significant, positive effects on primary outcomes. None of the studies satisfied the four WIDER Recommendations. CONCLUSIONS: Across five allied health professions, equivocal results, low methodological quality, and outcome reporting bias limited our ability to recommend one KT strategy over another. Further research employing the WIDER Recommendations is needed to inform the development and implementation of effective KT interventions in allied health.