ABSTRACT
BACKGROUND: Several systematic reviews have addressed digital technology use for treatment and monitoring of chronic obstructive pulmonary disease (COPD). OBJECTIVE: This study aimed to assess if systematic reviews considered the effects of sex, gender, or age on the outcomes of digital technologies for treatment and monitoring of COPD through an overview of such systematic reviews. The objectives of this overview were to (1) describe the definitions of sex or gender used in reviews; (2) determine whether the consideration of sex, gender, or age was planned in reviews; (3) determine whether sex, gender, or age was reported in review results; (4) determine whether sex, gender, or age was incorporated in implications for clinical practice in reviews; and (5) create an evidence map for development of individualized clinical recommendations for COPD based on sex, gender, or age diversity. METHODS: MEDLINE, the Cochrane Library, Epistemonikos, Web of Science, and the bibliographies of the included systematic reviews were searched to June 2022. Inclusion was based on the PICOS framework: (1) population (COPD), (2) intervention (any digital technology), (3) comparison (any), (4) outcome (any), and (5) study type (systematic review). Studies were independently selected by 2 authors based on title and abstract and full-text screening. Data were extracted by 1 author and checked by another author. Data items included systematic review characteristics; PICOS criteria; and variables related to sex, gender, or age. Systematic reviews were appraised using A Measurement Tool to Assess Systematic Reviews, version 2 (AMSTAR 2). Data were synthesized using descriptive statistics. RESULTS: Of 1439 records, 30 systematic reviews published between 2010 and 2022 were included in this overview. The confidence in the results of 25 of the 30 (83%) reviews was critically low according to AMSTAR 2. The reviews focused on user outcomes that potentially depend on sex, gender, or age, such as efficacy or effectiveness (25/30, 83%) and acceptance, satisfaction, or adherence (3/30, 10%) to digital technologies for COPD. Reviews reported sex or gender (19/30 systematic reviews) or age (25/30 systematic reviews) among primary study characteristics. However, only 1 of 30 reviews included age in a subgroup analysis, and 3 of 30 reviews identified the effects of sex, gender, or age as evidence gaps. CONCLUSIONS: This overview shows that the effects of sex, gender, or age were rarely considered in 30 systematic reviews of digital technologies for COPD treatment and monitoring. Furthermore, systematic reviews did not incorporate sex, gender, nor age in their implications for clinical practice. We recommend that future systematic reviews should (1) evaluate the effects of sex, gender, or age on the outcomes of digital technologies for treatment and monitoring of COPD and (2) better adhere to reporting guidelines to improve the confidence in review results. TRIAL REGISTRATION: PROSPERO CRD42022322924; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=322924. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): RR2-10.2196/40538.
Subject(s)
Digital Technology , Pulmonary Disease, Chronic Obstructive , Humans , Systematic Reviews as Topic , Evidence Gaps , Pulmonary Disease, Chronic Obstructive/therapyABSTRACT
INTRODUCTION: Migraine is a neurological disorder characterized by recurrent, severe headaches that are often accompanied by other symptoms. There are various factors that can trigger a migraine in sufferers. Stress can be such a trigger. Drug and nondrug treatments are available for the preventive treatment of migraine. According to a German guideline, mindfulness can be recommended for the prophylaxis of migraine. Therefore, the aim was to investigate the effectiveness of mindfulness-based stress reduction (MBSR) in relation to patient-relevant outcomes in adult patients with migraine. Patient-relevant outcomes in this context are migraine frequency, headache intensity during a migraine attack, depressive symptoms, and quality of life. MATERIAL AND METHODS: The conduct of this study was guided by the PRISMA 2020 statement. A systematic literature search for randomized controlled trials (RCTs) of the effectiveness of MBSR in adult migraine patients was conducted in December 2021 in three databases: MEDLINE via PubMed, the Cochrane Library, and Web of Science. In addition, a review of reference lists and a search of study registries were performed. The last search was conducted on October 7, 2022. In a two-step process, studies were selected based on predefined inclusion and exclusion criteria. The potential for bias was assessed using the Cochrane Risk of Bias Tool 2. The results were summarized descriptively and by means of quantitative synthesis. RESULTS: Four RCTs with a total of 275 patients and the follow-up publication of one of these studies were included. The risk of bias in one study each was judged to be low or of some concern and high in two studies. Four studies were included in the quantitative analysis. For the endpoint migraine frequency, the meta-analytic summary of three studies failed to show a statistically significant benefit for MBSR (SMD: -0.23; 95% CI: -0.79 to 0.32). For the endpoint depressive symptoms, a meta-analytic summary of three studies showed a statistically significant benefit for MBSR (SMD: -0.59; 95% CI: -0.93 to -0.25). No study had examined the severity of headaches during a migraine episode. CONCLUSION: Some results suggest that migraine patients may benefit from MBSR. However, the evidence base is currently insufficient for recommendations on the use of MBSR as a nondrug treatment option. Further adequately powered, high-quality RCTs are needed.
Subject(s)
Migraine Disorders , Mindfulness , Adult , Humans , Treatment Outcome , Migraine Disorders/prevention & control , HeadacheABSTRACT
OBJECTIVES: A measurement tool to assess systematic reviews 2 (AMSTAR 2) was originally developed for systematic reviews (SRs) of health-care interventions. The aim of this study was to assess the applicability of AMSTAR 2 to SRs of non-intervention studies. STUDY DESIGN AND SETTING: This was a meta-research study. We used 20 SRs for each of the following four types of SRs: Diagnostic Test Accuracy reviews, Etiology and/or Risk reviews, Prevalence and/or Incidence reviews, and Prognostic reviews (80 in total). Three authors applied AMSTAR 2 independently to each included SRs. Then, the authors assessed the applicability of each item to that SR type and any SR type. RESULTS: Researchers unanimously indicated that 7 of 16 AMSTAR 2 items were applicable for all four specific SR types and any SR type (items 2, 5, 6, 7, 10, 14 and 16), but 8 of 16 items for any SR type. These items could cover generic SR methods that do not depend on a specific SR type. CONCLUSION: AMSTAR 2 is only partially applicable for non-intervention SRs. There is a need to adapt/extend AMSTAR 2 for SRs of non-intervention studies. Our study can help to further define generic methodological aspects shared across SR types and methodological expectations for non-intervention SRs.
Subject(s)
Evidence-Based Medicine , Research Design , Humans , Systematic Reviews as TopicABSTRACT
BACKGROUND: Appropriate dissemination of public health evidence is of high importance to ensure that scientific knowledge reaches potential stakeholders and relevant population groups. A wide distrust towards science and its findings indicates that communication thereof remains below its potential. Cochrane Public Health provides an important source of high-quality scientific evidence in the field of public health via reviews with systematic methodology. The aims of this study were to identify (1) dissemination strategies and (2) stakeholders of Cochrane Public Health reviews. METHODS: This is a bibliographic study with a cross-sectional design. All 68 records (reviews or review protocols) listed on the Cochrane Public Health website ( https://ph.cochrane.org/cph-reviews-and-topics ) up to 8 March 2022 were included. Record characteristics, dissemination strategies, and potential stakeholder details were coded by one author, and 10% of records were checked by another author. Data were analyzed using descriptive statistics or narratively into common themes. RESULTS: The 68 records were published between 2010 and 2022 and included 15 review protocols and 53 reviews with systematic methodology (46 systematic, 6 rapid, and 1 scoping review). All 53 reviews were disseminated via open-access plain language summaries (PLS) in English with translations into 3-13 other languages. Other dissemination strategies included information on Cochrane websites (e.g., clinical answers or guidelines) available for 41/53 reviews and Cochrane news or blogs that mentioned 19/53 reviews. Overall, 23/68 records mentioned the actual stakeholder involvement in review production, protocol development, or formulation of dissemination plans. The potential stakeholders included several highly diverse groups, such as the general population or specific communities (e.g., racial minority groups), policy and decision makers, and researchers and professionals in various fields (e.g., nutrition, physical activity, education, or care). CONCLUSIONS: This study shows that Cochrane Public Health reviews are disseminated predominantly via PLS in different languages and via review information on Cochrane websites. Planned dissemination strategies were rarely reported although actual stakeholders were involved in the planning and production of some reviews. The relevance of Cochrane Public Health reviews for non-academic stakeholders and the general population highlights the need for the dissemination of evidence from such reviews beyond academia. SYSTEMATIC REVIEW REGISTRATION: The study was prospectively registered at the Open Science Framework ( https://osf.io/ga9pt/ ).
Subject(s)
Communication , Public Health , Humans , Cross-Sectional Studies , Educational Status , Quality of Health Care , Review Literature as TopicABSTRACT
INTRODUCTION: Mother's own milk is the best nutrition for every newborn and especially for vulnerable infants such as preterm infants with a very low birth weight below 1,500 grams (VLBW). If no MOM is available, human donor milk is the alternative of choice. Mothers of preterm born infants face challenging conditions that impair sufficient milk production. For this reason, it is particularly important to provide structural lactation support and, at the same time, to promote the establishment of human donor milk banks. METHODS AND ANALYSIS: Via a multidisciplinary approach the Neo-MILK study will develop an intervention for structured breastfeeding and lactation support. This will be based on a comprehensive status quo and needs assessment. In addition, the implementation of human donor milk banks (HDMB) will be supported by the development of standards. ETHICS AND DISSEMINATION: Intervention development is participatory, involving different disciplines and stakeholders. All surveys are subject to approval by the ethics committee. During the course of the project, the results will be communicated to the scientific community and the general public via publications, the project homepage and social media. TRIAL REGISTRATION NUMBER: DRKS00024799 (German Clinical Trials Register).
Subject(s)
Breast Feeding , Infant, Premature , Female , Humans , Infant , Infant, Newborn , Infant Nutritional Physiological Phenomena , Infant, Very Low Birth Weight , Intensive Care Units, Neonatal , Lactation , Milk, Human , Mothers , Needs AssessmentABSTRACT
High-quality systematic reviews (SRs) can strengthen the evidence base for prevention and health promotion. A 16-item AMSTAR 2 tool allows the appraisal of SRs by deriving a confidence rating in their results. In this cross-sectional study, we aimed to assess and compare two approaches to appraising 30 SRs of digital interventions for physical activity (PA) promotion using AMSTAR 2. Approach 1 (appraisals with 2/16 items) was used to identify SRs with critically low confidence ratings. Approach 2 (appraisals with all 16 items) was used (1) to derive the confidence ratings, (2) to identify SR strengths and weaknesses, and (3) to compare SR strengths among subgroups of SRs. The appraisal outcomes were summarized and compared using descriptive statistics. Approach 1 was quick (mean of 5 min/SR) at identifying SRs with critically low confidence ratings. Approach 2 was slower (mean of 20 min/SR), but allowed to identify SR strengths and weaknesses. Approach 2 showed that confidence ratings were low to critically low in 29/30 SRs. More strengths were identified in SRs with review protocols relative to SRs without review protocols and in newer SRs (published after AMSTAR 2 release) relative to older SRs. Only two items on AMSTAR 2 can quickly identify SRs with critical weaknesses. Although most SRs received low to critically low confidence ratings, SRs with review protocols and newer SRs tended to have more strengths. Future SRs require review protocols and better adherence to reporting guidelines to improve the confidence in their results.
Subject(s)
Exercise , Frailty , Humans , Cross-Sectional StudiesABSTRACT
BACKGROUND: 'A Measurement Tool to Assess Systematic Reviews, version 2' (AMSTAR 2) is a validated 16-item scale designed to appraise systematic reviews (SRs) of healthcare interventions and to rate the overall confidence in their results. This commentary aims to describe the challenges with rating of the individual items and the application of AMSTAR 2 from the user perspective. DISCUSSION: A group of six experienced users (methodologists working in different clinical fields for at least 10 years) identified and discussed the challenges in rating of each item and the general use of AMSTAR 2 to appraise SRs. A group discussion was used to develop recommendations on how users could deal with the identified challenges. We identified various challenges with the content of items 2-16 and with the derivation of the overall confidence ratings on AMSTAR 2. These challenges include the need (1) to provide additional definitions (e.g., what constitutes major deviations from SR protocol on item 2), (2) to choose a rating strategy for multiple conditions on single items (e.g., how to rate item 5 if studies were selected in duplicate, but consensus between two authors was not reported), and (3) to determine rules for deriving the confidence ratings (e.g., what items are critical for such ratings). Based on these challenges we formulated specific recommendations for items 2-16 that AMSTAR 2 users could consider before applying the tool. Our commentary adds to the existing literature by providing the first in-depth examination of the AMSTAR 2 tool from the user perspective. The identified challenges could be addressed by additional decision rules including definitions for ambiguous items and guidance for rating of complex items and derivation of confidence ratings. We recommend that a team consensus regarding such decision rules is required before appraisal procedure begins. TRIAL REGISTRATION: Not applicable.
Subject(s)
Evidence-Based Medicine , Humans , Evidence-Based Medicine/methods , Reproducibility of ResultsABSTRACT
BACKGROUND: Chronic obstructive pulmonary disease (COPD) is a common chronic disease that can be treated and monitored with various digital technologies. Digital technologies offer unique opportunities for treating and monitoring people with chronic diseases, but little is known about whether the outcomes of such technologies depend on sex, gender, or age in people with COPD. OBJECTIVE: The general objective of this study is to assess the possible influence of sex, gender, or age on outcomes of digital technologies for treatment and monitoring of COPD through an overview of systematic reviews. METHODS: The study is planned as an overview of systematic reviews. Study reporting is based on the PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) 2020 guidelines because guidelines for overviews are not available as of this writing. The information sources for the overview will include 4 bibliographic databases (MEDLINE, Cochrane Library, Epistemonikos, and Web of Science) as well as the bibliographies of the included systematic reviews. The electronic search strategy will be developed and conducted in collaboration with an experienced database specialist. The search results will be presented in accordance with the PRISMA 2020 guidelines. The eligibility of studies is based on the population, intervention, comparison, outcomes, and study design (PICOS) criteria: (1) people with COPD (population), (2) digital technology intervention for treatment or monitoring (intervention), (3) any control group or no control group (comparison), (4) any outcome, and (5) systematic review of randomized controlled trials or non-randomized controlled trials with or without a meta-analysis (study design). Critical appraisal of the included systematic reviews will be performed using A Measurement Tool to Assess Systematic Reviews, version 2 (AMSTAR 2). Data will be extracted using a standardized data extraction sheet. RESULTS: The literature search is scheduled for June 2022. We expect to select the relevant systematic reviews, code the data, and appraise the systematic reviews by December 2022. CONCLUSIONS: There is a growing recognition that the influence of sex, gender, or age should be considered in research design and outcome reporting in the context of health care interventions. Our overview will identify systematic reviews of various digital technologies for treatment or monitoring of COPD. The most interesting aspect of the overview will be to investigate if any systematic reviews considered the influence of sex, gender, or age on the outcomes of such digital technologies in COPD. Evidence from the overview could be used to guide more individualized (sex, gender, or age-based) recommendations for the use of digital technologies among people with COPD. TRIAL REGISTRATION: PROSPERO International Prospective Register of Systematic Reviews CRD42022322924; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=322924.
ABSTRACT
OBJECTIVE: To develop a reporting guideline for overviews of reviews of healthcare interventions. DESIGN: Development of the preferred reporting items for overviews of reviews (PRIOR) statement. PARTICIPANTS: Core team (seven individuals) led day-to-day operations, and an expert advisory group (three individuals) provided methodological advice. A panel of 100 experts (authors, editors, readers including members of the public or patients) was invited to participate in a modified Delphi exercise. 11 expert panellists (chosen on the basis of expertise, and representing relevant stakeholder groups) were invited to take part in a virtual face-to-face meeting to reach agreement (≥70%) on final checklist items. 21 authors of recently published overviews were invited to pilot test the checklist. SETTING: International consensus. INTERVENTION: Four stage process established by the EQUATOR Network for developing reporting guidelines in health research: project launch (establish a core team and expert advisory group, register intent), evidence reviews (systematic review of published overviews to describe reporting quality, scoping review of methodological guidance and author reported challenges related to undertaking overviews of reviews), modified Delphi exercise (two online Delphi surveys to reach agreement (≥70%) on relevant reporting items followed by a virtual face-to-face meeting), and development of the reporting guideline. RESULTS: From the evidence reviews, we drafted an initial list of 47 potentially relevant reporting items. An international group of 52 experts participated in the first Delphi survey (52% participation rate); agreement was reached for inclusion of 43 (91%) items. 44 experts (85% retention rate) completed the second Delphi survey, which included the four items lacking agreement from the first survey and five new items based on respondent comments. During the second round, agreement was not reached for the inclusion or exclusion of the nine remaining items. 19 individuals (6 core team and 3 expert advisory group members, and 10 expert panellists) attended the virtual face-to-face meeting. Among the nine items discussed, high agreement was reached for the inclusion of three and exclusion of six. Six authors participated in pilot testing, resulting in minor wording changes. The final checklist includes 27 main items (with 19 sub-items) across all stages of an overview of reviews. CONCLUSIONS: PRIOR fills an important gap in reporting guidance for overviews of reviews of healthcare interventions. The checklist, along with rationale and example for each item, provides guidance for authors that will facilitate complete and transparent reporting. This will allow readers to assess the methods used in overviews of reviews of healthcare interventions and understand the trustworthiness and applicability of their findings.
Subject(s)
Checklist , Health Facilities , Consensus , Delivery of Health Care , Delphi Technique , Humans , Research Design , Surveys and QuestionnairesABSTRACT
BACKGROUND: Digital interventions are interventions supported by digital tools or technologies, such as mobile apps, wearables, or web-based software. Digital interventions in the context of public health are specifically designed to promote and improve health. Recent reviews have shown that many digital interventions target physical activity promotion; however, it is unclear how such digital interventions are evaluated. OBJECTIVE: We aimed to investigate evaluation strategies in the context of digital interventions for physical activity promotion using a scoping review of published reviews. We focused on the target (ie, user outcomes or tool performance), methods (ie, tool data or self-reported data), and theoretical frameworks of the evaluation strategies. METHODS: A protocol for this study was preregistered and published. From among 300 reviews published up to March 19, 2021 in Medline, PsycINFO, and CINAHL databases, 40 reviews (1 rapid, 9 scoping, and 30 systematic) were included in this scoping review. Two authors independently performed study selection and data coding. Consensus was reached by discussion. If applicable, data were coded quantitatively into predefined categories or qualitatively using definitions or author statements from the included reviews. Data were analyzed using either descriptive statistics, for quantitative data (relative frequencies out of all studies), or narrative synthesis focusing on common themes, for qualitative data. RESULTS: Most reviews that were included in our scoping review were published in the period from 2019 to 2021 and originated from Europe or Australia. Most primary studies cited in the reviews included adult populations in clinical or nonclinical settings, and focused on mobile apps or wearables for physical activity promotion. The evaluation target was a user outcome (efficacy, acceptability, usability, feasibility, or engagement) in 38 of the 40 reviews or tool performance in 24 of the 40 reviews. Evaluation methods relied upon objective tool data (in 35/40 reviews) or other data from self-reports or assessments (in 28/40 reviews). Evaluation frameworks based on behavior change theory, including goal setting, self-monitoring, feedback on behavior, and educational or motivational content, were mentioned in 22 out of 40 reviews. Behavior change theory was included in the development phases of digital interventions according to the findings of 20 out of 22 reviews. CONCLUSIONS: The evaluation of digital interventions is a high priority according to the reviews included in this scoping review. Evaluations of digital interventions, including mobile apps or wearables for physical activity promotion, typically target user outcomes and rely upon objective tool data. Behavior change theory may provide useful guidance not only for development of digital interventions but also for the evaluation of user outcomes in the context of physical activity promotion. Future research should investigate factors that could improve the efficacy of digital interventions and the standardization of terminology and reporting in this field. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): RR2-10.2196/35332.
Subject(s)
Mobile Applications , Adult , Australia , Europe , Exercise , Humans , TechnologyABSTRACT
BACKGROUND: Digital interventions (DIs) could support physical activity (PA) promotion, according to recent reviews. However, it remains unclear if and how DIs for PA promotion are evaluated; thus, it is unclear if they support behavior change in real-world settings. A mapping of evidence from published reviews is required to focus on the evaluation of DIs for PA promotion. OBJECTIVE: The aim of our study is to investigate evaluation strategies for any outcome in the context of DIs for PA promotion by conducting a scoping review of published reviews. METHODS: Our scoping review adheres to the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines. The information sources include bibliographic databases (MEDLINE, PsycINFO, and CINAHL) and the bibliographies of the selected studies. The electronic search strategy was developed and conducted in collaboration with an experienced database specialist. The electronic search was conducted in English with no limits up to March 19, 2021, for sources with the terms digital intervention AND evaluation AND physical activity in titles or abstracts. After deduplication, 300 reviews selected from 4912 search results were assessed for eligibility by 2 authors working independently. The inclusion criteria were (1) healthy or clinical samples (population), (2) DIs for PA promotion (intervention), (3) comparisons to any other intervention or no intervention (comparison), (4) evaluation strategies (methods, results, or frameworks) for any outcome in the context of DIs for PA promotion (outcome), and (5) any published review (study type). According to the consensus reached during a discussion, 40 reviews met the inclusion criteria-36 from the electronic search and 4 from the manual search of the bibliographies of the 36 reviews. All reviews reported the evaluation strategies for any outcomes in the context of DIs for PA promotion in healthy or clinical samples. Data coding and the quality appraisal of systematic reviews are currently being performed independently by 2 authors. RESULTS: Our scoping review includes data from 40 published reviews (1 rapid review, 9 scoping reviews, and 30 systematic reviews). The focus of data coding is on evaluation strategies in the context of DIs for PA promotion and on the critical appraisal of the included systematic reviews. The final consensus regarding all data is expected in early 2022. CONCLUSIONS: Interventions for PA promotion that are supported by digital technologies require evaluation to ensure their efficacy in real-world settings. Our scoping review is needed because it addresses novel objectives that focus on such evaluations and are not answered in the published reviews identified in our search. The evaluation strategies addressing DIs for PA promotion will be mapped to synthesize the results that have been reported in published reviews so far. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/35332.
ABSTRACT
'A measurement tool to assess systematic reviews, version 2' (AMSTAR2) is a 16-item tool to critically appraise systematic reviews (SRs) of healthcare interventions. This study aimed to assess the methods and outcomes of AMSTAR2 appraisals in overviews of SRs of interventions for mental and behavioural disorders. The cross-sectional study was conducted using 32 overviews of SRs selected from three electronic databases in January 2021. Data items included overview and SR characteristics and AMSTAR2 appraisal methods and outcomes. Data were extracted by two authors independently and narratively synthesised using descriptive statistics (means ± SD and relative frequencies). SR characteristics were compared based on AMSTAR2 appraisal outcomes using chi-square tests. The 32 overviews appraised SRs of predominantly non-pharmacological interventions for mental disorders. AMSTAR2 appraisals were reported as confidence ratings in 25/32 overviews or individual item scores in 24/32 overviews. Most SRs/overview were non-Cochrane (mean = 94%), included RCTs only (mean = 77%) and were published before AMSTAR2 release (mean = 79%). The confidence ratings derived in 25 overviews for 349 SRs were predominantly critically low (68%). Confidence ratings were similar for SRs with RCTs only versus RCTs+non-RCTs or SRs published before versus after AMSTAR2 release, while Cochrane SRs received more high+moderate than low+critically low confidence ratings (p < 0.01). Confidence ratings derived based on AMSTAR2 do not differentiate among SRs of healthcare interventions except for Cochrane SRs that fulfil the criteria for high confidence ratings. AMSTAR2 items should be consulted to avoid common weaknesses in future SRs.
Subject(s)
Mental Disorders , Publications , Cross-Sectional Studies , Delivery of Health Care , Humans , Mental Disorders/therapy , Systematic Reviews as TopicABSTRACT
OBJECTIVES: A measurement tool to assess systematic reviews (SRs) 2 (AMSTAR 2) allows for deriving the overall confidence in an SR. We investigated how authors derived the overall confidence rating and whether different schemes lead to different results. STUDY DESIGN AND SETTING: We compared three different schemes (original 7-item scheme, a self-developed 5-item scheme, and the AMSTAR Web site) to derive the overall confidence in AMSTAR 2 using two distinct samples of SRs. Multiple bibliographic databases were searched for articles to analyze how AMSTAR 2 was applied by others. RESULTS: In both samples (n = 60 and n = 58), the Friedman test revealed a significant difference between the schemes (P < 0.001). The Web site scheme was the least strict one, whereas between the 5-item and 7-item scheme, no differences were found in post hoc analyses. We included 53 publications applying AMSTAR 2 identified in our literature search. Only 37 of them (70%) used the original 7-item scheme. Less than half of them (18 of 37) reported how they derived the overall rating. CONCLUSION: Authors should clearly report how they have derived the overall rating when applying AMSTAR 2. Reporting should allow for reproducing the overall ratings for editors, peer reviewers, and readers.
Subject(s)
Research Design/standards , Systematic Reviews as Topic , Cross-Sectional Studies , Evidence-Based Medicine , Humans , Peer Review/methods , Publishing/standards , Reproducibility of Results , Systematic Reviews as Topic/methods , Systematic Reviews as Topic/standardsABSTRACT
BACKGROUND: Several standards have been developed to assess methodological quality of systematic reviews (SR). One widely used tool is the AMSTAR. A recent update - AMSTAR 2 - is a 16 item evaluation tool that enables a detailed assessment of SR that include randomised (RCT) or non-randomised studies (NRS) of healthcare interventions. METHODS: A cross-sectional study of SR on pharmacological or psychological interventions in major depression in adults was conducted. SR published during 2012-2017 were sampled from MEDLINE, EMBASE and the Cochrane Database of SR. Methodological quality was assessed using AMSTAR 2. Potential predictive factors associated with quality were examined. RESULTS: In rating overall confidence in the results of 60 SR four reviews were rated "high", two were "moderate", one was "low" and 53 were "critically low". The mean AMSTAR 2 percentage score was 45.3% (standard deviation 22.6%) in a wide range from 7.1% to 93.8%. Predictors of higher quality were: type of review (higher quality in Cochrane Reviews), SR including only randomized trials and higher journal impact factor. LIMITATIONS: AMSTAR 2 is not intended to be used for the generation of a percentage score. CONCLUSIONS: According to AMSTAR 2 the overall methodological quality of SR on the treatment of adult major depression needs improvement. Although there is a high need for summarized information in the field of mental health, this work demonstrates the need to critically assess SR before using their findings. Better adherence to established reporting guidelines for SR is needed.
ABSTRACT
BACKGROUND: Guideline developers can: (1) adopt existing recommendations from others; (2) adapt existing recommendations to their own context; or (3) create recommendations de novo. Monetary and nonmonetary resources, credibility, maximization of uptake, as well as logical arguments should guide the choice of the approach and processes. OBJECTIVES: To describe a potentially efficient model for guideline production based on adoption, adaptation, and/or de novo development of recommendations utilizing the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Evidence to Decision (EtD) frameworks. STUDY DESIGN AND SETTING: We applied the model in a new national guideline program producing 22 practice guidelines. We searched for relevant evidence that informs the direction and strength of a recommendation. We then produced GRADE EtDs for guideline panels to develop recommendations. RESULTS: In two waves, a total of 80 EtD frameworks was produced approximately 4 months and 146 EtDs in about 6 months. Use of the EtD frameworks allowed panel members to understand judgments of other guideline groups about the criteria that bear on guideline recommendations and then make their own judgments about those criteria in a systematic approach. CONCLUSION: The "GRADE-ADOLOPMENT" approach to guideline production combines adoption, adaptation, and, as needed, de novo development of recommendations. If guideline developers apply EtD criteria more widely and make their work publically available, this approach should prove even more useful.
Subject(s)
Decision Making , Evidence-Based Medicine , Practice Guidelines as Topic , Decision Support Techniques , Delivery of Health Care , Evidence-Based Medicine/methods , Evidence-Based Medicine/standards , Germany , Health Resources , Humans , Practice Guidelines as Topic/standardsABSTRACT
OBJECTIVES: The objectives of this study were to determine the interrater reliability (IRR) of assessment of multiple systematic reviews (AMSTAR) 2 for reviews of pharmacological or psychological interventions for the treatment of major depression, to compare it to that of AMSTAR and risk of bias in systematic reviews (ROBIS), and to assess the convergent validity between the appraisal tools. STUDY DESIGN AND SETTING: Two groups of four raters were each assigned one of two samples of 30 systematic reviews. All eight raters applied AMSTAR 2 to their sample. Each group also applied either AMSTAR or ROBIS. Fleiss' kappa and Gwet's AC1 were calculated, and agreement between the tools was assessed. RESULTS: The median kappa values as a measure of IRR indicated a moderate agreement for AMSTAR 2 (median = 0.51), a substantial agreement for AMSTAR (median = 0.62), and a fair agreement for ROBIS (median = 0.27). Validity results showed a positive association for AMSTAR and AMSTAR 2 (r = 0.91) as well as ROBIS and AMSTAR 2 (r = 0.84). For the overall rating, AMSTAR 2 showed a high concordance with ROBIS and a lower concordance with AMSTAR. CONCLUSION: The IRR of AMSTAR 2 was found to be slightly lower than the IRR of AMSTAR and higher than the IRR of ROBIS. Validity measurements indicate that AMSTAR 2 is closely related to both ROBIS and AMSTAR.
Subject(s)
Depressive Disorder, Major/therapy , Systematic Reviews as Topic , Antidepressive Agents/therapeutic use , Bias , Calibration , Cross-Sectional Studies , Data Analysis , Humans , Non-Randomized Controlled Trials as Topic , Observer Variation , Psychometrics , Psychotherapy , Randomized Controlled Trials as Topic , Reproducibility of ResultsABSTRACT
INTRODUCTION: Minimum volume standards have been implemented in various countries for quality or safety policies. We present minimum volume standards in an international comparison, focusing on regulatory approaches, selected sets of procedures and thresholds as well as predetermined consequences of non-compliance. MATERIALS AND METHODS: We combined a comprehensive literature search in electronic databases in March 2016 with a hand-search of governmental and related organisations' webpages. We also contacted international experts to verify the information we found in the literature and to obtain additional data. RESULTS: Minimum volume standards have been introduced in different countries predominantly for highly specialized surgical procedures. The same evidence has led to different definitions and ways of implementation of minimum volume standards in Germany, Canada (Ontario), the Netherlands, Switzerland, and Austria. The regulatory approaches to minimum volume standards and the predetermined consequences of non-compliance differ across the countries. CONCLUSION: The sets of procedures for which minimum volume standards and corresponding thresholds have been introduced vary across countries, possibly due to different regulatory approaches. In addition, key attributes of the health care system might affect the development and implementation of minimum volume standards. Therefore, it is not feasible to formulate uniform recommendations that are applicable to all countries. Our results provide a comprehensive overview of international minimum volume standards and can be used to inform policy decisions.
