Predictors of Higher Quality of Systematic Reviews Addressing Nutrition and Cancer Prevention.

Systematic reviews/meta-analyses (SR/MAs) are considered a reliable source of information in healthcare. We aimed to explore the association of several characteristics of SR/MAs addressing nutrition in cancer prevention and their quality/risk of bias (using assessments from AMSTAR-2 and ROBIS tools). The analysis included 101 SR/MAs identified in a systematic survey. Associations of each specified characteristic (e.g., information about the protocol, publication year, reported use of GRADE, or other methods for assessing overall certainty of evidence) with the number of AMSTAR-2 not met ('No' responses) and the number of ROBIS items met ('Probably Yes' or "Yes' responses) were examined. Poisson regression was used to identify predictors of the number of 'No' answers (indicating lower quality) for all AMSTAR-2 items and the number of 'Yes' or 'Probably Yes' answers (indicating higher quality/lower concern for bias) for all ROBIS items. Logistic regression was used to identify variables associated with at least one domain assessed as 'low concern for bias' in the ROBIS tool. In multivariable analysis, SR/MAs not reporting use of any quality/risk of bias assessment instrument for primary studies were associated with a higher number of 'No' answers for all AMSTAR-2 items (incidence rate ratio (IRR) 1.26, 95% confidence interval (CI) 1.09-1.45), and a lower number of 'Yes' or 'Probably Yes' answers for all ROBIS items (IRR 0.76, 95% CI 0.66-0.87). Providing information about the protocol and search for unpublished studies was associated with a lower number of 'No' answers (IRR 0.73, 95% CI 0.56-0.97 and IRR 0.75, 95% CI 0.59-0.95, respectively) and a higher number of 'Yes' or 'Probably Yes' answers (IRR 1.43, 95% CI 1.17-1.74 and IRR 1.28, 95% CI 1.07-1.52, respectively). Not using at least one quality/risk of bias assessment tool for primary studies within an SR/MA was associated with lower odds that a study would be assessed as 'low concern for bias' in at least one ROBIS domain (odds ratio 0.061, 95% CI 0.007-0.527). Adherence to methodological standards in the development of SR/MAs was associated with a higher overall quality of SR/MAs addressing nutrition for cancer prevention.

Are systematic reviews addressing nutrition for cancer prevention trustworthy? A systematic survey of quality and risk of bias.

CONTEXT: The last 30 years have yielded a vast number of systematic reviews and/or meta-analyses addressing the link between nutrition and cancer risk. OBJECTIVE: The aim of this survey was to assess overall quality and potential for risk of bias in systematic reviews and meta-analyses (SRMAs) that examined the role of nutrition in cancer prevention. DATA SOURCES: MEDLINE, Embase, and the Cochrane Library databases were searched (last search performed November 2018). STUDY SELECTION: Studies identified as SRMAs that investigated a nutritional or dietary intervention or exposure for cancer prevention in the general population or in people at risk of cancer and in which primary studies had a comparison group were eligible for inclusion. Screening, data extraction, and quality assessment were conducted independently by 2 reviewers. DATA EXTRACTION: Altogether, 101 studies were randomly selected for analysis. The methodological quality and risk of bias were evaluated using the AMSTAR-2 and ROBIS tools, respectively. RESULTS: Most SRMAs included observational studies. Less than 10% of SRMAs reported a study protocol, and only 51% of SRMAs assessed the risk of bias in primary studies. Most studies conducted subgroup analyses, but only a few reported tests of interaction or specified subgroups of interest a priori. Overall, according to AMSTAR-2, only 1% of SRMAs were of high quality, while 97% were of critically low quality. Only 3% had a low risk of bias, according to ROBIS. CONCLUSIONS: This systematic survey revealed substantial limitations with respect to quality and risk of bias of SRMAs. SRMAs examining nutrition and cancer prevention cannot be considered trustworthy, and results should be interpreted with caution. Peer reviewers as well as users of SRMAs should be advised to use the AMSTAR-2 and/or ROBIS instruments to help to determine the overall quality and risk of bias of SRMAs. SYSTEMATIC REVIEW REGISTRATION: PROSPERO registration number CRD42019121116.

Similarities, reliability and gaps in assessing the quality of conduct of systematic reviews using AMSTAR-2 and ROBIS: systematic survey of nutrition reviews.

BACKGROUND: AMSTAR-2 ('A Measurement Tool to Assess Systematic Reviews, version 2') and ROBIS ('Risk of Bias in Systematic Reviews') are independent instruments used to assess the quality of conduct of systematic reviews/meta-analyses (SR/MAs). The degree of overlap in methodological constructs together with the reliability and any methodological gaps have not been systematically assessed and summarized in the field of nutrition. METHODS: We performed a systematic survey of MEDLINE, EMBASE, and the Cochrane Library for SR/MAs published between January 2010 and November 2018 that examined the effects of any nutritional intervention/exposure for cancer prevention. We followed a systematic review approach including two independent reviewers at each step of the process. For AMSTAR-2 (16 items) and ROBIS (21 items), we assessed the similarities, the inter-rater reliability (IRR) and any methodological limitations of the instruments. Our protocol for the survey was registered in PROSPERO (CRD42019121116). RESULTS: We found 4 similar domain constructs based on 11 comparisons from a total of 12 AMSTAR-2 and 14 ROBIS items. Ten comparisons were considered fully overlapping. Based on Gwet's agreement coefficients, six comparisons provided almost perfect (> 0.8), three substantial (> 0.6), and one a moderate level of agreement (> 0.4). While there is considerable overlap in constructs, AMSTAR-2 uniquely addresses explaining the selection of study designs for inclusion, reporting on excluded studies with justification, sources of funding of primary studies, and reviewers' conflict of interest. By contrast, ROBIS uniquely addresses appropriateness and restrictions within eligibility criteria, reducing risk of error in risk of bias (RoB) assessments, completeness of data extracted for analyses, the inclusion of all necessary studies for analyses, and adherence to predefined analysis plan. CONCLUSIONS: Among the questions on AMSTAR-2 and ROBIS, 70.3% (26/37 items) address the same or similar methodological constructs. While the IRR of these constructs was moderate to perfect, there are unique methodological constructs that each instrument independently addresses. Notably, both instruments do not address the reporting of absolute estimates of effect or the overall certainty of the evidence, items that are crucial for users' wishing to interpret the importance of SR/MA results.