Increasing the efficiency of study selection for systematic reviews using prioritization tools and a single-screening approach.

BACKGROUND: Systematic literature screening is a key component in systematic reviews. However, this approach is resource intensive as generally two persons independently of each other (double screening) screen a vast number of search results. To develop approaches for increasing efficiency, we tested the use of text mining to prioritize search results as well as the involvement of only one person (single screening) in the study selection process. METHOD: Our study is based on health technology assessments (HTAs) of drug and non-drug interventions. Using a sample size calculation, we consecutively included 11 searches resulting in 33 study selection processes. Of the three screeners for each search, two used screening tools with prioritization (Rayyan, EPPI Reviewer) and one a tool without prioritization. For each prioritization tool, we investigated the proportion of citations classified as relevant at three cut-offs or STOP criteria (after screening 25%, 50% and 75% of the citation set). For each STOP criterion, we measured sensitivity (number of correctly identified relevant studies divided by the total number of relevant studies in the study pool). In addition, we determined the number of relevant studies identified per single screening round and investigated whether missed studies were relevant to the HTA conclusion. RESULTS: Overall, EPPI Reviewer performed better than Rayyan and identified the vast majority (88%, Rayyan 66%) of relevant citations after screening half of the citation set. As long as additional information sources were screened, it was sufficient to apply a single-screening approach to identify all studies relevant to the HTA conclusion. Although many relevant publications (n = 63) and studies (n = 29) were incorrectly excluded, ultimately only 5 studies could not be identified at all in 2 of the 11 searches (1x 1 study, 1x 4 studies). However, their omission did not change the overall conclusion in any HTA. CONCLUSIONS: EPPI Reviewer helped to identify relevant citations earlier in the screening process than Rayyan. Single screening would have been sufficient to identify all studies relevant to the HTA conclusion. However, this requires screening of further information sources. It also needs to be considered that the credibility of an HTA may be questioned if studies are missing, even if they are not relevant to the HTA conclusion.

Negative pressure wound therapy in patients with wounds healing by secondary intention: a systematic review and meta-analysis of randomised controlled trials.

BACKGROUND: Negative pressure wound therapy (NPWT) is a widely used method of wound treatment. We performed a systematic review of randomised controlled trials (RCTs) comparing the patient-relevant benefits and harms of NPWT with standard wound therapy (SWT) in patients with wounds healing by secondary intention. METHODS: We searched for RCTs in MEDLINE, Embase, the Cochrane Central Register of Controlled Trials, and study registries (last search: July 2018) and screened reference lists of relevant systematic reviews and health technology assessments. Manufacturers and investigators were asked to provide unpublished data. Eligible studies investigated at least one patient-relevant outcome (e.g. wound closure). We assessed publication bias and, if feasible, performed meta-analyses, grading the results into different categories (hint, indication or proof of a greater benefit or harm). RESULTS: We identified 48 eligible studies of generally low quality with evaluable data for 4315 patients and 30 eligible studies with missing data for at least 1386 patients. Due to potential publication bias (proportion of inaccessible data, 24%), we downgraded our conclusions. A meta-analysis of all wound healing data showed a significant effect in favour of NPWT (OR 1.56, 95% CI 1.15 to 2.13, p = 0.008). As further analyses of different definitions of wound closure did not contradict that analysis, we inferred an indication of a greater benefit of NPWT. A meta-analysis of hospital stay (in days) showed a significant difference in favour of NPWT (MD - 4.78, 95% CI - 7.79 to - 1.76, p = 0.005). As further analyses of different definitions of hospital stay/readmission did not contradict that analysis, we inferred an indication of a greater benefit of NPWT. There was neither proof (nor indication nor hint) of greater benefit or harm of NPWT for other patient-relevant outcomes such as mortality and adverse events. CONCLUSIONS: In summary, low-quality data indicate a greater benefit of NPWT versus SWT for wound closure in patients with wounds healing by secondary intention. The length of hospital stay is also shortened. The data show no advantages or disadvantages of NPWT for other patient-relevant outcomes. Publication bias is an important problem in studies on NPWT, underlining that all clinical studies need to be fully reported.

Effective study selection using text mining or a single-screening approach: a study protocol.

BACKGROUND: Systematic information retrieval generally requires a two-step selection process for studies, which is conducted by two persons independently of one another (double-screening approach). To increase efficiency, two methods seem promising, which will be tested in the planned study: the use of text mining to prioritize search results as well as the involvement of only one person in the study selection process (single-screening approach). The aim of the present study is to examine the following questions related to the process of study selection: Can the use of the Rayyan or EPPI Reviewer tools to prioritize the results of study selection increase efficiency? How accurately does a single-screening approach identify relevant studies? Which advantages or disadvantages (e.g., shortened screening time or increase in the number of full texts ordered) does a single-screening versus a double-screening approach have? METHODS: Our study is a prospective analysis of study selection processes based on benefit assessments of drug and non-drug interventions. It consists of two parts: firstly, the evaluation of a single-screening approach based on a sample size calculation (11 study selection processes, including 33 single screenings) and involving different screening tools and, secondly, the evaluation of the conventional double-screening approach based on five conventional study selection processes. In addition, the advantages and disadvantages of the single-screening versus the double-screening approach with regard to the outcomes "number of full texts ordered" and "time required for study selection" are analyzed. The previous work experience of the screeners is considered as a potential effect modifier. DISCUSSION: No study comparing the features of prioritization tools is currently available. Our study can thus contribute to filling this evidence gap. This study is also the first to investigate a range of questions surrounding the screening process and to include an a priori sample size calculation, thus enabling statistical conclusions. In addition, the impact of missing studies on the conclusion of a benefit assessment is calculated. SYSTEMATIC REVIEW REGISTRATION: Not applicable.