Exploring the impact of evaluation on learning and health innovation sustainability: protocol for a realist synthesis.

BACKGROUND: Within the Learning Health System (LHS) model, learning routines, including evaluation, allow for continuous incremental change to take place. Within these learning routines, evaluation assists in problem identification, data collection, and data transformation into contextualized information, which is then re-applied to the LHS environment. Evaluation that catalyzes learning and improvement may also contribute to health innovation sustainability. However, there is little consensus as to why certain evaluations seem to support learning and sustainability, while others impede it. This realist synthesis seeks to understand the contextual factors and underlying mechanisms or drivers that best support health systems learning and sustainable innovation. METHODS: This synthesis will be guided by Pawson and colleagues' 2005 and Emmel and colleagues' 2018 guidelines for conducting realist syntheses. The review process will encompass five steps: (1) scoping the review, (2) building theories, (3) identifying the evidence, (4) evidence selection and appraisal, and (5) data extraction and synthesis. An Expert Committee comprised of leaders in evaluation, innovation, sustainability, and realist methodology will guide this synthesis. Review findings will be reported using the RAMESES guidelines. DISCUSSION: The use of a realist review will allow for exploration and theorizing about the contextual factors and underlying mechanisms that make evaluations 'work' (or 'not work') to support learning and sustainability. Depending on results, we will attempt to synthesize findings into a series of recommendations for evaluations with the intention to support health systems learning and sustainability. Finalized results will be presented at national and international conferences, as well as disseminated via a peer-reviewed publication. SYSTEMATIC REVIEW REGISTRATION: This realist synthesis protocol has been registered with PROSPERO ( https://www.crd.york.ac.uk/prospero/  ID 382690).

Examining Use Behavior of a Goal-Supporting mHealth App in Primary Care Among Patients With Multiple Chronic Conditions: Qualitative Descriptive Study.

BACKGROUND: Although mobile health (mHealth) apps are increasingly being used to support patients with multiple chronic conditions (multimorbidity), most mHealth apps experience low interaction and eventual abandonment. To tackle this engagement issue, when developing an mHealth program, it is important to understand the social-behavioral factors that affect patients' use behavior. OBJECTIVE: The aim of this study was to explore the social and behavioral factors contributing to patients' use behavior of an mHealth app called the electronic Patient-Reported Outcome (ePRO). The ePRO app supports goal-oriented care delivery in interdisciplinary primary care models. METHODS: A descriptive qualitative study was used to analyze interview data collected for a larger mixed methods pragmatic trial. The original 15-month trial was conducted in 6 primary care teams across Ontario, Canada, between 2018 and 2019. The eligibility criteria for patients were being aged ≥60 years with ≥10 visits within the previous 12 months of study enrollment. For this analysis, patients were classified as long-term or short-term users based on their length of use of the ePRO app during the trial. The Social Cognitive Theory by Bandura was used to categorize social-behavioral factors that contributed to patients' decision to continue or discontinue using the app. RESULTS: The patient-provider relationship emerged as a key factor that shaped patients' experiences with the app and subsequent decision to continue using the app. Other factors that contributed to patients' decision to continue using the app were personal and social circumstances, perceived usefulness, patients' previous experience with goal-related behaviors, and confidence in one's capability. There was an overlap of experience between long- and short-term app users but, in general, long-term users perceived the app to be more useful and their goals to be more meaningful than short-term app users. This observation was complicated by the fact that patient health-related goals were dynamic and changed over time. CONCLUSIONS: Complex patients' use behavior of a goal-supporting mHealth app is shaped by an array of sociobehavioral factors that can evolve. To tackle this dynamism, there should be an emphasis on creating adaptable health technologies that are easily customizable by patients and able to respond to their changing contexts and needs. TRIAL REGISTRATION: ClinicalTrials.gov NCT02917954; https://clinicaltrials.gov/ct2/show/NCT02917954.

Examining health care providers' and middle-level managers' readiness for change: a qualitative study.

BACKGROUND: Readiness is a critical precursor of successful change; it denotes whether those involved in the change are motivated and empowered to participate in the change. Research on readiness tends to focus on frontline providers or individuals in non-managerial positions and offers limited attention to individuals in middle management positions who are expected to lead frontline providers in change implementation. Yet middle-level managers are also recipients of changes that are planned and decreed by those in higher positions. This study sought to examine both frontline provider and middle manager readiness for change in the context of primary care program integration. METHODS: Using a qualitative case study approach, we examined how frontline providers and middle managers experienced six readiness factors: discrepancy, appropriateness, valence, efficacy, fairness and trust in management. Data were collected through documents, meeting observation and semi-structured interviews with frontline providers and middle managers involved in the change. RESULTS: The findings highlighted similarities and differences in readiness experiences of frontline providers and middle managers. Across both types of participants, we found that the notion of valence should be expanded to consider individuals' evaluation of benefits to patients and the health system; efficacy applies to both content and process of change; fairness and trust in management findings require further exploration to determine their appropriateness to be incorporated in models of readiness for change; and trust in management (or lack of trust) has a cascading influence across the levels in the organization. CONCLUSIONS: Our study makes a contribution by nuancing and extending conceptualizations of individual readiness factors, and by highlighting the central role of middle manager readiness for change. Implications of the study include the need to consider readiness factors prior to the implementation of change and the importance of fostering readiness throughout all levels of the organization.