How the "Understanding Research Evidence" Web-Based Video Series From the National Collaborating Centre for Methods and Tools Contributes to Public Health Capacity to Practice Evidence-Informed Decision Making: Mixed-Methods Evaluation.

BACKGROUND: The National Collaborating Centre for Methods and Tools (NCCMT) offers workshops and webinars to build public health capacity for evidence-informed decision-making. Despite positive feedback for NCCMT workshops and resources, NCCMT users found key terms used in research papers difficult to understand. The Understanding Research Evidence (URE) videos use plain language, cartoon visuals, and public health examples to explain complex research concepts. The videos are posted on the NCCMT website and YouTube channel. OBJECTIVE: The first four videos in the URE web-based video series, which explained odds ratios (ORs), confidence intervals (CIs), clinical significance, and forest plots, were evaluated. The evaluation examined how the videos affected public health professionals' practice. A mixed-methods approach was used to examine the delivery mode and the content of the videos. Specifically, the evaluation explored (1) whether the videos were effective at increasing knowledge on the four video topics, (2) whether public health professionals were satisfied with the videos, and (3) how public health professionals applied the knowledge gained from the videos in their work. METHODS: A three-part evaluation was conducted to determine the effectiveness of the first four URE videos. The evaluation included a Web-based survey, telephone interviews, and pretest and posttests, which evaluated public health professionals' experience with the videos and how the videos affected their public health work. Participants were invited to participate in this evaluation through various open access, public health email lists, through informational flyers and posters at the Canadian Public Health Association (CPHA) conference, and through targeted recruitment to NCCMT's network. RESULTS: In the Web-based surveys (n=46), participants achieved higher scores on the knowledge assessment questions from watching the OR (P=.04), CI (P=.04), and clinical significance (P=.05) videos but not the forest plot (P=.12) video, as compared with participants who had not watched the videos. The pretest and posttest (n=124) demonstrated that participants had a better understanding of forest plots (P<.001) and CIs (P<.001) after watching the videos. Due to small sample size numbers, there were insufficient pretest and posttest data to conduct meaningful analyses on the clinical significance and OR videos. Telephone interview participants (n=18) thought the videos' use of animation, narration, and plain language was appropriate for people with different levels of understanding and learning styles. Participants felt that by increasing their understanding of research evidence, they could develop better interventions and design evaluations to measure the impact of public health initiatives. CONCLUSIONS: Overall, the results of the evaluation showed that watching the videos resulted in an increase in knowledge, and participants had an overall positive experience with the URE videos. With increased competence in using the best available evidence, professionals are empowered to contribute to decisions that can improve health outcomes of communities.

Promoting Awareness of Key Resources for Evidence-Informed Decision-making in Public Health: An Evaluation of a Webinar Series about Knowledge Translation Methods and Tools.

The National Collaborating Centre for Methods and Tools (NCCMT) has developed several resources to support evidence-informed decision-making - the process of distilling and disseminating best available evidence from research, context, and experience - and knowledge translation, applying best evidence in practice. One such resource, the Registry of Methods and Tools, is a free online database of 195 methods and tools to support knowledge translation. Building on the identification of webinars as a strategy to improve the dissemination of information, NCCMT launched the Spotlight on Knowledge Translation Methods and Tools webinar series in 2012 to promote awareness and use of the Registry. To inform continued implementation of this webinar series, NCCMT conducted an evaluation of the series' potential to improve awareness and use of the methods/tools within the Registry, as well as identify areas for improvement and "what worked." For this evaluation, the following data were analyzed: electronic follow-up surveys administered immediately following each webinar; an additional electronic survey administered 6 months after two webinars; and Google Analytics for each webinar. As of November 2015, there have been 22 webinars conducted, reaching 2048 people in multiple sectors across Canada and around the world. Evaluation results indicate that the webinars increase awareness about the Registry and stimulate use of the methods/tools. Although webinar attendees were significantly less likely to have used the methods/tools 6 months after webinars, this may be attributed to the lack of an identified opportunity in their work to use the method/tool. Despite technological challenges and requests for further examples of how the methods/tools have been used, there is overwhelming positive feedback that the format, presenters, content, and interaction across webinars "worked." This evaluation supports that webinars are a valuable strategy for increasing awareness and stimulating use of resources for evidence-informed decision-making and knowledge translation in public health practice.