Development and usability testing of an online support tool to identify models and frameworks to inform implementation.

BACKGROUND: Theories, models and frameworks (TMFs) are useful when implementing, evaluating and sustaining healthcare evidence-based interventions. Yet it can be challenging to identify an appropriate TMF for an implementation project. We developed and tested the usability of an online tool to help individuals who are doing or supporting implementation practice activities to identify appropriate models and/or frameworks to inform their work. METHODS: We used methods guided by models and evidence on implementation science and user-centered design. Phases of tool development included applying findings from a scoping review of TMFs and interviews with 24 researchers/implementers on barriers and facilitators to identifying and selecting TMFs. Based on interview findings, we categorized the TMFs by aim, stage of implementation, and target level of change to inform the tool's algorithm. We then conducted interviews with 10 end-users to test the usability of the prototype tool and administered the System Usability Scale (SUS). Usability issues were addressed and incorporated into the tool. RESULTS: We developed Find TMF, an online tool consisting of 3-4 questions about the user's implementation project. The tool's algorithm matches key characteristics of the user's project (aim, stage, target change level) with characteristics of different TMFs and presents a list of candidate models/frameworks. Ten individuals from Canada or Australia participated in usability testing (mean SUS score 84.5, standard deviation 11.4). Overall, participants found the tool to be simple, easy to use and visually appealing with a useful output of candidate models/frameworks to consider for an implementation project. Users wanted additional instruction and guidance on what to expect from the tool and how to use the information in the output table. Tool improvements included incorporating an overview figure outlining the tool steps and output, displaying the tool questions on a single page, and clarifying the available functions of the results page, including adding direct links to the glossary and to complementary tools. CONCLUSIONS: Find TMF is an easy-to-use online tool that may benefit individuals who support implementation practice activities by making the vast number of models and frameworks more accessible, while also supporting a consistent approach to identifying and selecting relevant TMFs.

Identifying and selecting implementation theories, models and frameworks: a qualitative study to inform the development of a decision support tool.

BACKGROUND: Implementation theories, models and frameworks offer guidance when implementing and sustaining healthcare evidence-based interventions. However, selection can be challenging given the myriad of potential options. We propose to inform a decision support tool to facilitate the appropriate selection of an implementation theory, model or framework in practice. To inform tool development, this study aimed to explore barriers and facilitators to identifying and selecting implementation theories, models and frameworks in research and practice, as well as end-user preferences for features and functions of the proposed tool. METHODS: We used an interpretive descriptive approach to conduct semi-structured interviews with implementation researchers and practitioners in Canada, the United States and Australia. Audio recordings were transcribed verbatim. Data were inductively coded by a single investigator with a subset of 20% coded independently by a second investigator and analyzed using thematic analysis. RESULTS: Twenty-four individuals participated in the study. Categories of barriers/facilitators, to inform tool development, included characteristics of the individual or team conducting implementation and characteristics of the implementation theory, model or framework. Major barriers to selection included inconsistent terminology, poor fit with the implementation context and limited knowledge about and training in existing theories, models and frameworks. Major facilitators to selection included the importance of clear and concise language and evidence that the theory, model or framework was applied in a relevant health setting or context. Participants were enthusiastic about the development of a decision support tool that is user-friendly, accessible and practical. Preferences for tool features included key questions about the implementation intervention or project (e.g., purpose, stage of implementation, intended target for change) and a comprehensive list of relevant theories, models and frameworks to choose from along with a glossary of terms and the contexts in which they were applied. CONCLUSIONS: An easy to use decision support tool that addresses key barriers to selecting an implementation theory, model or framework in practice may be beneficial to individuals who facilitate implementation practice activities. Findings on end-user preferences for tool features and functions will inform tool development and design through a user-centered approach.

Quality indicators for ambulatory care for older adults with diabetes and comorbid conditions: A Delphi study.

BACKGROUND: An increasing number of people are living with multiple chronic conditions and it is unclear which quality indicators should be used to guide care for this population. OBJECTIVE: To critically appraise and select the most appropriate set of quality indicators for ambulatory care for older adults with five selected disease combinations. METHODS: A two-round web-based Delphi process was used to critically appraise and select quality of care indicators for older adults with diabetes and comorbidities. A fifteen-member Canadian expert panel with broad geographical and clinical representation participated in this study. The panel evaluated process indicators for meaningfulness, potential for improvements in clinical practice, and overall value of inclusion, while outcome indicators were evaluated for importance, modifiability and overall value of inclusion. A 70% agreement threshold was required for high consensus, and 60-69% for moderate consensus as measured on a 5-point Likert type scale. RESULTS: Twenty high-consensus and nineteen medium-consensus process and outcome indicators were selected for assessing care for older adults with selected disease combinations, including 1) concordant (conditions with a common management plan), 2) discordant (conditions with unrelated management plans), and 3) both types. Panelists reached rapid consensus on quality indicators for care for older adults with concordant comorbid conditions, but not for those with discordant conditions. All selected indicators assess clinical aspects of care. The feedback from the panelists emphasized the importance of developing indicators related to patient-centred aspects of care, including patient self-management, education, patient-physician relationships, and patient's preferences. CONCLUSIONS: The selected quality indicators are not intended to provide a comprehensive tool set for measuring quality of care for older adults with selected disease combinations. The recommended indicators address clinical aspects of care and can be used as a starting point for ambulatory care settings and development of additional quality indicators.

Quality indicators for care of osteoarthritis in primary care settings: a systematic literature review.

Background: Despite the high prevalence of osteoarthritis and the prominence of primary care in managing this condition, there is no systematic summary of quality indicators applicable for osteoarthritis care in primary care settings. Objectives: This systematic review aimed to identify evidence-based quality indicators for monitoring, evaluating and improving the quality of care for adults with osteoarthritis in primary care settings. Methods: Ovid MEDLINE and Ovid EMBASE databases and grey literature, including relevant organizational websites, were searched from 2000 to 2015. Two reviewers independently selected studies if (i) the study methodology combined a systematic literature search with assessment of quality indicators by an expert panel and (ii) quality indicators were applicable to assessment of care for adults with osteoarthritis in primary care settings. Included studies were appraised using the Appraisal of Indicators through Research and Evaluation (AIRE) instrument. A narrative synthesis was used to combine the indicators within themes. Applicable quality indicators were categorized according to Donabedian's 'structure-process-outcome' framework. Results: The search revealed 4526 studies, of which 32 studies were reviewed in detail and 4 studies met the inclusion criteria. According to the AIRE domains, all studies were clear on purpose and stakeholder involvement, while formal endorsement and use of indicators in practice were scarcely described. A total of 20 quality indicators were identified from the included studies, many of which overlapped conceptually or in content. Conclusions: The process of developing quality indicators was methodologically suboptimal in most cases. There is a need to develop specific process, structure and outcome measures for adults with osteoarthritis using appropriate methodology.

Quality indicators for care of depression in primary care settings: a systematic review.

BACKGROUND: Despite the growing interest in assessing the quality of care for depression, there is little evidence to support measurement of the quality of primary care for depression. This study identified evidence-based quality indicators for monitoring, evaluating and improving the quality of care for depression in primary care settings. METHODS: Ovid MEDLINE and Ovid PsycINFO databases, and grey literature, including relevant organizational websites, were searched from 2000 to 2015. Two reviewers independently selected studies if (1) the study methodology combined a systematic literature search with assessment of quality indicators by an expert panel and (2) quality indicators were applicable to assessment of care for adults with depression in primary care settings. Included studies were appraised using the Appraisal of Indicators through Research and Evaluation (AIRE) instrument, which contains four domains and 20 items. A narrative synthesis was used to combine the indicators within themes. Quality indicators applicable to care for adults with depression in primary care settings were extracted using a structured form. The extracted quality indicators were categorized according to Donabedian's 'structure-process-outcome' framework. RESULTS: The search revealed 3838 studies. Four additional publications were identified through grey literature searching. Thirty-nine articles were reviewed in detail and seven met the inclusion criteria. According to the AIRE domains, all studies were clear on purpose and stakeholder involvement, while formal endorsement and usage of indicators in practice were scarcely described. A total of 53 quality indicators were identified from the included studies, many of which overlap conceptually or in content: 15 structure, 33 process and four outcome indicators. This study identified quality indicators for evaluating primary care for depression among adult patients. CONCLUSIONS: The identified set of indicators address multiple dimensions of depression care and provide an excellent starting point for further development and use in primary care settings.