Principles for good scholarship in systematic reviews.

Many sources document problems that jeopardize the trustworthiness of systematic reviews. This is a major concern given their potential to influence patient care and impact people's lives. Responsibility for producing trustworthy conclusions on the evidence in systematic reviews is borne primarily by authors who need the necessary training and resources to correctly report on the current knowledge base. Peer reviewers and editors are also accountable; they must ensure that systematic reviews are accurate by demonstrating proper methods. To support all these stakeholders, we attempt to distill the sprawling guidance that is currently available in our recent co-publication about best tools and practices for systematic reviews. We specifically address how to meet methodological conduct standards applicable to key components of systematic reviews. In this complementary invited review, we place these standards in the context of good scholarship principles for systematic review development. Our intention is to reach a broad audience and potentially improve the trustworthiness of evidence syntheses published in the developmental medicine literature and beyond.

Guidance to best tools and practices for systematic reviews1.

Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy.A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work.Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

Quality appraisal of systematic reviews of interventions for children with cerebral palsy reveals critically low confidence.

AIM: To evaluate the methodological quality of recent systematic reviews of interventions for children with cerebral palsy in order to determine the level of confidence in the reviews' conclusions. METHOD: A comprehensive search of 22 databases identified eligible systematic reviews with and without meta-analysis published worldwide from 2015 to 2019. We independently extracted data and used A Measurement Tool to Assess Systematic Reviews-2 (AMSTAR-2) to appraise methodological quality. RESULTS: Eighty-three systematic reviews met strict eligibility criteria. Most were from Europe and Latin America and reported on rehabilitative interventions. AMSTAR-2 appraisal found critically low confidence in 88% (n=73) because of multiple and varied deficiencies. Only 7% (n=6) had no AMSTAR-2 critical domain deficiency. The number of systematic reviews increased fivefold from 2015 to 2019; however, quality did not improve over time. INTERPRETATION: Most of these systematic reviews are considered unreliable according to AMSTAR-2. Current recommendations for treating children with CP based on these flawed systematic reviews need re-evaluation. Findings are comparable to reports from other areas of medicine, despite the general perception that systematic reviews are high-level evidence. The required use of current widely accepted guidance for conducting and reporting systematic reviews by authors, peer reviewers, and editors is critical to ensure reliable, unbiased, and transparent systematic reviews. What this paper adds Confidence was critically low in the conclusions of 88% of systematic reviews about interventions for children with cerebral palsy (CP). Quality issues in the sample were not limited to systematic reviews of non-randomized trials, or to those about certain populations of CP or interventions. The inclusion of meta-analysis did not improve the level of confidence in these systematic reviews. Numbers of systematic reviews on this topic increased over the 5 search years but their methodological quality did not improve.

Single-subject research designs in pediatric rehabilitation: a valuable step towards knowledge translation.

Knowledge translation may be particularly challenging in pediatric rehabilitation, where study findings are often ambiguous owing to low statistical power or inconsistent responses to intervention. Disconnection between research protocols and clinical practicality, as well as variability of responsiveness in heterogeneous pediatric populations, may further impede integration of research findings into everyday practice. Use of single-subject research designs (SSRDs) may bridge the gap between research and practice, with robust design options that better identify and preserve patterns of responsiveness to specific interventions and offer protocols that are more readily implemented in practice settings than can be done in traditional randomized controlled trials. This review defines SSRD, provides examples of research questions that can be answered using SSRD, details the experimental designs that can be used and the level of evidence of each design, and describes statistical analysis approaches and clinical application. This analysis will aid researchers, reviewers, clinicians, and others in better understanding SSRD methodology and its application in everyday practice.

Single-subject research design: recommendations for levels of evidence and quality rating.

The aim of this article is to present a set of evidence levels, accompanied by 14 quality or rigor questions, to foster a critical review of published single-subject research articles. In developing these guidelines, we reviewed levels of evidence and quality/rigor criteria that are in wide use for group research designs, e.g. randomized controlled trials, such as those developed by the Treatment Outcomes Committee of the American Academy for Cerebral Palsy and Developmental Medicine. We also reviewed methodological articles on how to conduct and critically evaluate single-subject research designs (SSRDs). We then subjected the quality questions to interrater agreement testing and refined them until acceptable agreement was reached. We recommend that these guidelines be implemented by clinical researchers who plan to conduct single-subject research or who incorporate SSRD studies into systematic reviews, and by clinicians who aim to practise evidence-based medicine and who wish to critically review pediatric single-subject research.