Development of the Engage with Impact Toolkit: A comprehensive resource to support the evaluation of patient, family and caregiver engagement in health systems.

INTRODUCTION: Recent shifts in the patient, family and caregiver engagement field have focused greater attention on measurement and evaluation, including the impacts of engagement efforts. Current evaluation tools offer limited support to organizations seeking to reorient their efforts in this way. We addressed this gap through the development of an impact measurement framework and accompanying evaluation toolkit-the Engage with Impact Toolkit. METHODS: The measurement framework and toolkit were co-designed with the Evaluating Patient Engagement Working Group, a multidisciplinary group of patient, family and caregiver partners, engagement specialists, researchers and government personnel. Project activities occurred over four phases: (1) project scoping and literature review; (2) modified concept mapping; (3) working group deliberations and (4) toolkit web design. RESULTS: The project scope was to develop a measurement framework and an evaluation toolkit for patient engagement in health systems that were practical, accessible, menu-driven and aligned with current system priorities. Concept mapping yielded 237 impact statements that were sorted, discussed and combined into 81 unique items. A shorter list of 50 items (rated 8.0 or higher out of 10) was further consolidated to generate a final list of 35 items mapped across 8 conceptual domains of impact: (1) knowledge and skills; (2) confidence and trust; (3) equity and inclusivity; (4) priorities and decisions; (5) effectiveness and efficiency; (6) patient-centredness; (7) culture change and (8) patient outcomes and experience. Working Group members rated the final list for importance (1-5) and identified a core set of 33 items (one for each of the 8 domains and 25 supplementary items). Two domains (priorities and decisions; and culture change) yielded the highest overall importance ratings (4.8). A web-based toolkit (www.evaluateengagement.ca) hosts the measurement framework and related evaluation supports. CONCLUSION: The Engage with Impact Toolkit builds on existing engagement evaluation tools but brings a more explicit focus to supporting organizations to assess the impacts of their engagement work. PATIENT CONTRIBUTION: Patient, family and caregiver partners led the early conceptualization of this work and were involved at all stages and in all aspects of the work. As end-users of the toolkit, their perspectives, knowledge and opinions were critical.

Benefits of Digital Health Resources for Substance Use Concerns in Women: Scoping Review.

BACKGROUND: Digital health resources are being increasingly used to support women with substance use concerns. Although empirical research has demonstrated that these resources have promise, the available evidence for their benefit in women requires further investigation. Evidence supports the capacity of interventions that are sex-, gender-, and trauma-informed to improve treatment access and outcomes and to reduce health system challenges and disparities. Indeed, both sex- and gender-specific approaches are critical to improve health and gender equity. Violence and trauma are frequent among those with substance use concerns, but they disproportionately affect those who identify as female or women, further underscoring the need for trauma-informed care as well. OBJECTIVE: The objective of this investigation was to evaluate the evidence supporting the efficacy or effectiveness of online or mobile interventions for risky or harmful substance use in adults who identify as female or women, or who report a history of trauma. METHODS: This scoping review is based on an academic search in MEDLINE, APA PsycINFO, Embase, Cochrane Central, and CINAHL, as well as a grey literature search in US and Canadian government and funding agency websites. Of the 7807 records identified, 465 remained following title and abstract screening. Of these, 159 met all eligibility criteria and were reviewed and synthesized. RESULTS: The 159 records reflected 141 distinct studies and 125 distinct interventions. Investigations and the interventions evaluated predominantly focused on alcohol use or general substance use. Evaluated digital health resources included multisession and brief-session interventions, with a wide range of therapeutic elements. Multisession online and mobile interventions exhibited beneficial effects in 86.1% (105/122) of studies. Single-session interventions similarly demonstrated beneficial effects in 64.2% (43/67) of study conditions. Most investigations did not assess gender identity or conduct sex- or gender-based analyses. Only 13 investigations that included trauma were identified. CONCLUSIONS: Despite the overall promise of digital health interventions for substance use concerns, direct or quantitative evidence on the efficacy or effectiveness of interventions in females or women specifically is weak.