Potential impact of missing outcome data on treatment effects in systematic reviews: imputation study.

OBJECTIVE: To assess the risk of bias associated with missing outcome data in systematic reviews. DESIGN: Imputation study. SETTING: Systematic reviews. POPULATION: 100 systematic reviews that included a group level meta-analysis with a statistically significant effect on a patient important dichotomous efficacy outcome. MAIN OUTCOME MEASURES: Median percentage change in the relative effect estimate when applying each of the following assumption (four commonly discussed but implausible assumptions (best case scenario, none had the event, all had the event, and worst case scenario) and four plausible assumptions for missing data based on the informative missingness odds ratio (IMOR) approach (IMOR 1.5 (least stringent), IMOR 2, IMOR 3, IMOR 5 (most stringent)); percentage of meta-analyses that crossed the threshold of the null effect for each method; and percentage of meta-analyses that qualitatively changed direction of effect for each method. Sensitivity analyses based on the eight different methods of handling missing data were conducted. RESULTS: 100 systematic reviews with 653 randomised controlled trials were included. When applying the implausible but commonly discussed assumptions, the median change in the relative effect estimate varied from 0% to 30.4%. The percentage of meta-analyses crossing the threshold of the null effect varied from 1% (best case scenario) to 60% (worst case scenario), and 26% changed direction with the worst case scenario. When applying the plausible assumptions, the median percentage change in relative effect estimate varied from 1.4% to 7.0%. The percentage of meta-analyses crossing the threshold of the null effect varied from 6% (IMOR 1.5) to 22% (IMOR 5) of meta-analyses, and 2% changed direction with the most stringent (IMOR 5). CONCLUSION: Even when applying plausible assumptions to the outcomes of participants with definite missing data, the average change in pooled relative effect estimate is substantive, and almost a quarter (22%) of meta-analyses crossed the threshold of the null effect. Systematic review authors should present the potential impact of missing outcome data on their effect estimates and use this to inform their overall GRADE (grading of recommendations assessment, development, and evaluation) ratings of risk of bias and their interpretation of the results.

Meta-Analyses Proved Inconsistent in How Missing Data Were Handled Across Their Included Primary Trials: A Methodological Survey.

BACKGROUND: How systematic review authors address missing data among eligible primary studies remains uncertain. OBJECTIVE: To assess whether systematic review authors are consistent in the way they handle missing data, both across trials included in the same meta-analysis, and with their reported methods. METHODS: We first identified 100 eligible systematic reviews that included a statistically significant meta-analysis of a patient-important dichotomous efficacy outcome. Then, we successfully retrieved 638 of the 653 trials included in these systematic reviews' meta-analyses. From each trial report, we extracted statistical data used in the analysis of the outcome of interest to compare with the data used in the meta-analysis. First, we used these comparisons to classify the "analytical method actually used" for handling missing data by the systematic review authors for each included trial. Second, we assessed whether systematic reviews explicitly reported their analytical method of handling missing data. Third, we calculated the proportion of systematic reviews that were consistent in their "analytical method actually used" across trials included in the same meta-analysis. Fourth, among systematic reviews that were consistent in the "analytical method actually used" across trials and explicitly reported on a method for handling missing data, we assessed whether the "analytical method actually used" and the reported methods were consistent. RESULTS: We were unable to determine the "analytical method reviews actually used" for handling missing outcome data among 397 trials. Among the remaining 241, systematic review authors most commonly conducted "complete case analysis" (n=128, 53%) or assumed "none of the participants with missing data had the event of interest" (n=58, 24%). Only eight of 100 systematic reviews were consistent in their approach to handling missing data across included trials, but none of these reported methods for handling missing data. Among seven reviews that did explicitly report their analytical method of handling missing data, only one was consistent in their approach across included trials (using complete case analysis), and their approach was inconsistent with their reported methods (assumed all participants with missing data had the event). CONCLUSION: The majority of systematic review authors were inconsistent in their approach towards reporting and handling missing outcome data across eligible primary trials, and most did not explicitly report their methods to handle missing data. Systematic review authors should clearly identify missing outcome data among their eligible trials, specify an approach for handling missing data in their analyses, and apply their approach consistently across all primary trials.

Potentially missing data are considerably more frequent than definitely missing data: a methodological survey of 638 randomized controlled trials.

BACKGROUND AND OBJECTIVE: Missing data for the outcomes of participants in randomized controlled trials (RCTs) are a key element of risk of bias assessment. However, it is not always clear from RCT reports whether some categories of participants were followed-up or not (i.e., do or do not have missing data) nor how the RCT authors dealt with missing data in their analyses. Our objectives were to describe how RCT authors (1) report on different categories of participants that might have missing data, (2) handle these categories in the analysis, and (3) judge the risk of bias associated with missing data. METHODS: We surveyed all RCT reports included in 100 clinical intervention systematic reviews (SRs), half of which were Cochrane SRs. Eligible SRs reported a group-level meta-analysis of a patient-important dichotomous efficacy outcome, with a statistically significant effect estimate. Eleven reviewers, working in pairs, independently extracted data from the primary RCT reports included in the SRs. We predefined 19 categories of participants that might have missing data. Then, we classified these participants as follows: "explicitly followed-up," "explicitly not followed-up" (i.e., definitely missing data), or "unclear follow-up status" (i.e., potentially missing data). RESULTS: Of 638 eligible RCTs, 400 (63%) reported on at least one of the predefined categories of participants that might have missing data. The median percentage of participants who were explicitly not followed-up was 5.8% (interquartile range 2.2-14.8%); it was 9.7% (4.1-14.9%) for participants with unclear follow up status; and 11.7% (interquartile range 5.6-23.7%) for participants who were explicitly not followed-up and with unclear follow-up status. When authors explicitly reported not following-up participants, they most often conducted complete case analysis (54%). Most RCTs neither reported on missing data separately for different outcomes (99%) nor reported using a method for judging risk of bias associated with missing data (95%). CONCLUSION: "Potentially missing data" are considerably more frequent than "definitely missing data." Adequate reporting of missing data will require development of explicit standards on which editors insist and to which RCT authors adhere.

Systematic reviews do not adequately report or address missing outcome data in their analyses: a methodological survey.

OBJECTIVES: To describe how systematic review authors report and address categories of participants with potential missing outcome data of trial participants. STUDY DESIGN AND SETTING: Methodological survey of systematic reviews reporting a group-level meta-analysis. RESULTS: We included a random sample of 50 Cochrane and 50 non-Cochrane systematic reviews. Of these, 25 reported in their methods section a plan to consider at least one of the 10 categories of missing outcome data; 42 reported in their results, data for at least one category of missing data. The most reported category in the methods and results sections was "unexplained loss to follow-up" (n = 34 in methods section and n = 6 in the results section). Only 19 reported a method to handle missing data in their primary analyses, which was most often complete case analysis. Few reviews (n = 9) reported in the methods section conducting sensitivity analysis to judge risk of bias associated with missing outcome data at the level of the meta-analysis; and only five of them presented the results of these analyses in the results section. CONCLUSION: Most systematic reviews do not explicitly report sufficient information on categories of trial participants with potential missing outcome data or address missing data in their primary analyses.

Use of safety-engineered devices by healthcare workers for intravenous and/or phlebotomy procedures in healthcare settings: a systematic review and meta-analysis.

BACKGROUND: The acquisition of needle-stick injuries (NSI) in a healthcare setting poses an occupational hazard of transmitting blood-borne pathogens from patients to healthcare workers (HCWs). The objective of this study was to systematically review the evidence about the efficacy and safety of using safety-engineered intravenous devices and safety-engineered phlebotomy devices by HCWs. METHODS: We included randomized and non-randomized studies comparing safety-engineered devices to conventional/standard devices that lack safety features for delivering intravenous injections and/or for blood-withdrawal procedures (phlebotomy). The outcomes of interest included NSI rates, and blood-borne infections rates among HCWs and patients. We conducted an extensive literature search strategy using the OVID interface in October 2013. We followed the standard methods for study selection and data abstraction. When possible, we conducted meta-analyses using a random-effects model. We used the GRADE methodology to assess the quality of evidence by outcome. RESULTS: We identified twenty-two eligible studies: Twelve assessed safety-engineered devices for intravenous procedures, five for phlebotomy procedures, and five for both. Twenty-one of those studies were observational while one was a randomized trial. All studies assessed the reduction in NSIs among HCWs. For safety-engineered intravenous devices, the pooled relative risk for NSI per HCW was 0.28 [0.13, 0.59] (moderate quality evidence). The pooled relative risk for NSI per device used or procedure performed was 0.34 [0.08,1.49] (low quality evidence). For safety-engineered phlebotomy devices, the pooled relative risk for NSI per HCW was 0.57 [0.38, 0.84] (moderate quality evidence). The pooled relative risk for NSI per device used or procedure performed was 0.53 [0.43,0.65] (moderate quality evidence). We identified no studies assessing the outcome of blood-borne infections among healthcare workers or patients. CONCLUSION: There is moderate-quality evidence that the use of safety-engineered devices in intravenous injections and infusions, and phlebotomy (blood-drawing) procedures reduces NSI rates of HCWs.

Knowledge, attitudes, beliefs, values, preferences, and feasibility in relation to the use of injection safety devices in healthcare settings: a systematic review.

BACKGROUND: Adopting technologies such as injection safety devices in healthcare settings can enhance injection safety. Developing guidelines for appropriate adoption of such technologies need to consider factors beyond evidence for their health effects. The objective of this study is to systematically review the published literature for evidence among healthcare workers and patients about knowledge, attitudes, beliefs, values, preferences, and feasibility in relation to the use of injection safety devices in healthcare settings. METHODS: We included both qualitative and quantitative studies conducted with the general public, patients, and healthcare workers, administrators, or policy makers. We searched MEDLINE, EMBASE, CINHAL and CENTRAL. We used a duplicate and independent approach to title and abstract screening, full text screening, data abstraction and risk of bias assessment. RESULTS: Out of a total of 6568 identified citations, we judged fourteen studies as eligible for this systematic review. All these studies were surveys, conducted with healthcare workers in high-income countries. We did not identify any qualitative study, or a study of the general public, patients, healthcare administrators or policy makers. We did not identify any study assessing knowledge, or values assigned to outcomes relevant to injection safety devices. Each of the included studies suffered from methodological limitations, which lowers our confidence in their findings. Based on the findings of six studies, the injection safety devices were generally perceived as easy to use and as an improvement compared with conventional syringes. Some of these studies reported few technical problems while using the devices. In three studies assessing perceived safety, the majority of participants judged the devices as safe. Two studies reported positive perceptions of healthcare workers regarding patient tolerance of these injection safety devices. One study found that less than half the nurses felt comfortable using the insulin pens. Findings from four studies assessing preference and satisfaction were not consistent. CONCLUSIONS: This systematic review identified evidence that injection safety devices are generally perceived as easy to use, safe, and tolerated by patients. There were few reports of technical problems while using the devices and some discomfort by nurses using the insulin pens.

Safety engineered injection devices for intramuscular, subcutaneous and intradermal injections in healthcare delivery settings: a systematic review and meta-analysis.

BACKGROUND: Occupational sharps injuries are associated with transmission of bloodborne viruses to healthcare workers, including hepatitis B virus (HBV), hepatitis C virus (HCV), and human immunodeficiency virus (HIV). Similarly reuse of syringes in healthcare settings might transmit these infections between patients. The objective of this study was to systematically review the evidence about the effects of the use by health care workers of two types of safety engineered injection devices, when delivering intramuscular, subcutaneous, or intradermal injectable medications: sharps injury protection syringes and reuse prevention syringes. METHODS: We included both randomized and non-randomized studies comparing safety syringes to syringes without safety features. Outcomes of interest included needlestick injuries, and HIV, HBV and HCV infections amongst HCWs (for sharps injury prevention syringes) and patients (for reuse prevention syringes). When possible, we conducted meta-analyses using a random-effects model. We tested results for heterogeneity across studies using the I statistic. We assessed the quality of evidence by outcome using the GRADE methodology. RESULTS: We included nine eligible studies: six assessed devices that qualify as sharps injury prevention devices, and three assessed devices that qualify as both injury prevention devices and reuse prevention devices. Eight studies were observational while one was randomized. All studies assessed a single outcome: needle stick injuries among healthcare workers. For sharp injury prevention syringes, the meta-analysis of five studies resulted in a pooled relative risk of 0.54 [0.41, 0.71] for the effect on needlestick injuries per healthcare worker. The associated quality of evidence was rated as moderate. For reuse prevention syringes, data from one study provided a relative risk of 0.40 [0.27, 0.59] for the effect on needlestick injuries per healthcare worker. The associated quality of evidence was rated as moderate. We identified no studies reporting on the effect on the reuse of syringes. CONCLUSIONS: We identified moderate quality evidence that syringes with sharps injury prevention feature reduce the incidence of needlestick injuries per healthcare worker. We identified no studies reporting data for the remaining outcomes of interest for HCWs. Similarly we identified no studies reporting on the effect of syringes with a reuse prevention feature on the reuse of syringes or on the other outcomes of interest for patients.

Impact of missing participant data for dichotomous outcomes on pooled effect estimates in systematic reviews: a protocol for a methodological study.

BACKGROUND: There is no consensus on how authors conducting meta-analysis should deal with trial participants with missing outcome data. The objectives of this study are to assess in Cochrane and non-Cochrane systematic reviews: (1) which categories of trial participants the systematic review authors consider as having missing participant data (MPD), (2) how trialists reported on participants with missing outcome data in trials, (3) whether systematic reviewer authors actually dealt with MPD in their meta-analyses of dichotomous outcomes consistently with their reported methods, and (4) the impact of different methods of dealing with MPD on pooled effect estimates in meta-analyses of dichotomous outcomes. METHODS/DESIGN: We will conduct a methodological study of Cochrane and non-Cochrane systematic reviews. Eligible systematic reviews will include a group-level meta-analysis of a patient-important dichotomous efficacy outcome, with a statistically significant effect estimate. Teams of two reviewers will determine eligibility and subsequently extract information from each eligible systematic review in duplicate and independently, using standardized, pre-piloted forms. The teams will then use a similar process to extract information from the trials included in the meta-analyses of interest. We will assess first which categories of trial participants the systematic reviewers consider as having MPD. Second, we will assess how trialists reported on participants with missing outcome data in trials. Third, we will compare what systematic reviewers report having done, and what they actually did, in dealing with MPD in their meta-analysis. Fourth, we will conduct imputation studies to assess the effects of different methods of dealing with MPD on the pooled effect estimates of meta-analyses. We will specifically calculate for each method (1) the percentage of systematic reviews that lose statistical significance and (2) the mean change of effect estimates across systematic reviews. DISCUSSION: The impact of different methods of dealing with MPD on pooled effect estimates will help judge the associated risk of bias in systematic reviews. Our findings will inform recommendations regarding what assumptions for MPD should be used to test the robustness of meta-analytical results.

Consumption of energy drinks among lebanese youth: a pilot study on the prevalence and side effects.

BACKGROUND: The new millennium has been together with a variety of synthetic and caffeinated high-energy drinks targeting the youth market. Energy drinks raise the level of energy and their consumption has been increased significantly worldwide. OBJECTIVES: This research aimed to determine patterns of energy drink consumption and to assess the prevalence of adverse side effects among energy drink users. PATIENTS AND METHODS: A pilot cross-sectional study survey was undertaken on students aged between 13 and 30 years in private and public schools and universities in Lebanon over 5 months. A self-administered questionnaire was used inquiring about sociodemographic characteristics, consumption patterns, attitudes and beliefs about energy drinks. Bivariate and multivariate analyses were conducted. Data was analyzed using SPSS 17. RESULTS: We studied 1500 students (mean age: 18.92 ± 1.85; 51.3% were males). The overall prevalence of energy drinks consumption was 63.6% (60.5% were males), among which 50.5% used alcoholic energy drinks. Respondents indicated that most consumed energy drinks were "Red Bull" and "Boom Boom" (70.9% and 51.5% respectively). In total, 64.5% of participants believed the effect of these drinks in energizing the body, and 72.7% believed that they can stimulate intellectual capacities. In addition, 29.6% of consumers experienced at least one adverse effect, where tachycardia was reported in 21.1% of cases. On the other hand, desired effects felt after consumption were mostly pleasure (33.8%). Males had a 3-time more risk of consuming such drinks compared to females (OR: 0.381, P < 0.001; 95% CI: 0.300-0.484). Additionally, this analysis demonstrated a significant association between energy drinks consumption and regions outside Beirut (OR: 1.401, P: 0.006; 95% CI: 1.103-1.781), medical field of work (OR: 0.376, P: 0.010; 95% CI: 0.179-0.790) and higher personal income (OR: 1.317, P < 0.001; 95% CI: 1.117-1.553). CONCLUSIONS: This study showed a high prevalence of energy drinks consumption among youth. The current results highlight the importance of education to prevent the consumption of energy drinks in excessive quantities and modifying some wrong perceptions regarding the benefits of energy drinks in youth.