Systematic review of the use of process evaluations in knowledge translation research.

BACKGROUND: Experimental designs for evaluating knowledge translation (KT) interventions can provide strong estimates of effectiveness but offer limited insight into how the intervention worked. Consequently, process evaluations have been used to explore the causal mechanisms at work; however, there are limited standards to guide this work. This study synthesizes current evidence of KT process evaluations to provide future methodological recommendations. METHODS: Peer-reviewed search strategies were developed by a health research librarian. Studies had to be in English, published since 1996, and were not excluded based on design. Studies had to (1) be a process evaluation of a KT intervention study in primary health, (2) be a primary research study, and (3) include a licensed healthcare professional delivering or receiving the intervention. A two-step, two-person hybrid screening approach was used for study inclusion with inter-rater reliability ranging from 94 to 95%. Data on study design, data collection, theoretical influences, and approaches used to evaluate the KT intervention, analysis, and outcomes were extracted by two reviewers. Methodological quality was assessed with the Mixed Methods Appraisal Tool (MMAT). RESULTS: Of the 20,968 articles screened, 226 studies fit our inclusion criteria. The majority of process evaluations used qualitative forms of data collection (43.4%) and individual interviews as the predominant data collection method. 72.1% of studies evaluated barriers and/or facilitators to implementation. 59.7% of process evaluations were stand-alone evaluations. The timing of data collection varied widely with post-intervention data collection being the most frequent (46.0%). Only 38.1% of the studies were informed by theory. Furthermore, 38.9% of studies had MMAT scores of 50 or less indicating poor methodological quality. CONCLUSIONS: There is widespread acceptance that the generalizability of quantitative trials of KT interventions would be significantly enhanced through complementary process evaluations. However, this systematic review found that process evaluations are of mixed quality and lack theoretical guidance. Most process evaluation data collection occurred post-intervention undermining the ability to evaluate the process of implementation. Strong science and methodological guidance is needed to underpin and guide the design and execution of process evaluations in KT science. REGISTRATION: This study is not registered with PROSPERO.

Integrating community assembly and biodiversity to better understand ecosystem function: the Community Assembly and the Functioning of Ecosystems (CAFE) approach.

The research of a generation of ecologists was catalysed by the recognition that the number and identity of species in communities influences the functioning of ecosystems. The relationship between biodiversity and ecosystem functioning (BEF) is most often examined by controlling species richness and randomising community composition. In natural systems, biodiversity changes are often part of a bigger community assembly dynamic. Therefore, focusing on community assembly and the functioning of ecosystems (CAFE), by integrating both species richness and composition through species gains, losses and changes in abundance, will better reveal how community changes affect ecosystem function. We synthesise the BEF and CAFE perspectives using an ecological application of the Price equation, which partitions the contributions of richness and composition to function. Using empirical examples, we show how the CAFE approach reveals important contributions of composition to function. These examples show how changes in species richness and composition driven by environmental perturbations can work in concert or antagonistically to influence ecosystem function. Considering how communities change in an integrative fashion, rather than focusing on one axis of community structure at a time, will improve our ability to anticipate and predict changes in ecosystem function.

A protocol for a systematic review of the use of process evaluations in knowledge translation research.

BACKGROUND: Experimental designs for evaluating knowledge translation (KT) interventions for professional behavior change can provide strong estimates of intervention effectiveness but offer limited insight how the intervention worked or not. Furthermore, trials provide little insight into the ways through which interventions lead to behavior change and how they are moderated by different facilitators and barriers. As a result, the ability to generalize the findings from one study to a different context, organization, or clinical problem is severely compromised. Consequently, researchers have started to explore the causal mechanisms in complementary studies (process evaluations) alongside experimental designs for evaluating KT interventions. This study focuses on improving process evaluations by synthesizing current evidence on process evaluations conducted alongside experimental designs for evaluating KT interventions. METHODS/DESIGN: A medical research librarian will develop and implement search strategies designed to identify evidence that is relevant to process evaluations in health research. Studies will not be excluded based on design. Included studies must contain a process evaluation component aimed at understanding or evaluating a KT intervention targeting professional behavior change. Two reviewers will perform study selection, quality assessment, and data extraction using standard forms. Disagreements will be resolved through discussion or third party adjudication. Data to be collected include study design, details about data collection approaches and types, theoretical influences, approaches to evaluate intervention dose delivered, intervention dose received, intervention fidelity, intervention reach, data analysis, and study outcomes. This study is not registered with PROSPERO. DISCUSSION: There is widespread acceptance that the generalizability of quantitative trials of KT interventions would be significantly enhanced to other contexts, health professional groups, and clinical conditions through complementary process evaluations alongside trials. This systematic review will serve as a 'state of the science' on methodological approaches to process evaluations and will allow us to: 1) take stock of current research approaches and 2) develop concrete recommendations for knowledge users (e.g., quality consultants and health services researchers) designing future KT process evaluations.