A snapshot of the characteristics, quality and volume of the COVID-19 evidence synthesis infodemic: systematic review*

ABSTRACT

and full text screening were undertaken by two independent reviewers. Descriptive information on review type, purpose, population, size, citation and attention metrics were extracted along with whether the review met six key methodological criteria. For reviews meeting all six methodological criteria, additional data were extracted on methods and publication metrics and AMSTAR-2 was used to assess the quality of the reported methods. Registration PROSPERO CRD42020188822ResultsSearches returned 2334 unique records. After applying eligibility criteria we included 280 reviews. Less than half reported undertaking critical appraisal and a third had no reproducible search strategy. There was considerable overlap in topics, with discordant findings. Eighty-eight of the 280 reviews met all six methodological criteria. Of these, 3 were rated as of moderate or high quality on AMSTAR-2, with the majority having critical flaws only a third reported registering a protocol, and less than one in five searched named COVID-19 databases. Review conduct and publication was rapid, with 56 of the 88 systematic review reported as being conducted within three weeks, and half published within three weeks of submission. Despite being of low quality and many lacking robust methods, the reviews received substantial attention across both academic and public platforms, and the attention was not related to the quality of review methods.ConclusionMethodological flaws limit the validity of systematic reviews and the generalisability of their findings. Yet by being reported as ‘systematic reviews’, many readers may well regard them as high quality evidence, irrespective of the methods undertaken. To maintain trustworthiness, researchers, peer-reviewers and journal editors need to ensure systematic reviews adhere to guidelines of best practice.