Informing the research agenda for optimizing audit and feedback interventions: results of a prioritization exercise.

BACKGROUND: Audit and feedback (A&F) interventions are one of the most common approaches for implementing evidence-based practices. A key barrier to more effective A&F interventions is the lack of a theory-guided approach to the accumulation of evidence. Recent interviews with theory experts identified 313 theory-informed hypotheses, spread across 30 themes, about how to create more effective A&F interventions. In the current survey, we sought to elicit from stakeholders which hypotheses were most likely to advance the field if studied further. METHODS: From the list of 313, three members of the research team identified 216 that were clear and distinguishable enough for prioritization. A web-based survey was then sent to 211 A&F intervention stakeholders asking them to choose up to 50 'priority' hypotheses following the header "A&F interventions will be more effective if…". Analyses included frequencies of endorsement of the individual hypotheses and themes into which they were grouped. RESULTS: 68 of the 211 invited participants responded to the survey. Seven hypotheses were chosen by > 50% of respondents, including A&F interventions will be more effective… "if feedback is provided by a trusted source"; "if recipients are involved in the design/development of the feedback intervention"; "if recommendations related to the feedback are based on good quality evidence"; "if the behaviour is under the control of the recipient"; "if it addresses barriers and facilitators (drivers) to behaviour change"; "if it suggests clear action plans"; and "if target/goal/optimal rates are clear and explicit". The most endorsed theme was Recipient Priorities (four hypotheses were chosen 92 times as a 'priority' hypotheses). CONCLUSIONS: This work determined a set of hypotheses thought by respondents to be to be most likely to advance the field through future A&F intervention research. This work can inform a coordinated research agenda that may more efficiently lead to more effective A&F interventions.

Behaviour Change Domains Likely to Influence Occupational Therapist Use of the Canadian Occupational Performance Measure.

INTRODUCTION: Occupational therapists have shown low adoption rates for many evidence-based practices. One such practice is the limited uptake of standardized outcome measures such as the Canadian Occupational Performance Measure. Use of this measure has not consistently translated into practice despite decades of encouragement. Theory-based approaches to understanding healthcare provider behaviour change are needed if we are to realize the goal of attaining practice that is in keeping with evidence. This study utilized the Theoretical Domains Framework, a theory-based approach for understanding barriers to evidence-based practice, in order to increase our understanding of the limited uptake of the Canadian Occupational Performance Measure in occupational therapy practice. METHODS: Theoretical Domains Framework methods were followed. First, primary data was collected from occupational therapists through semistructured interviews that focused on key behaviour change domains as they related to the use of the Canadian Occupational Performance Measure. Two independent researchers coded interview data into domains, derived belief statements from the data, and used belief strength, conflict, and frequency to determine the more and less influential domains for using the Canadian Occupational Performance Measure. RESULTS: Interviews with 15 practicing occupational therapists across a range of practice areas yielded six key behaviour change domains for increasing the use of the Canadian Occupational Performance Measure. The more relevant domains were Social influences, Social professional role and identity, Beliefs about consequences, Beliefs about capabilities, Skills, and Behavioural regulation). The other eight domains were found to be less relevant. CONCLUSION: We identified important domains and beliefs that influence the use of the Canadian Occupational Performance Measure by occupational therapists. Results inform our understanding of the use of this measure in practice and identify potential targets for behaviour change interventions.

Scoping Review on Rehabilitation Scoping Reviews.

OBJECTIVE: To examine the extent, scope, and methodological quality of rehabilitation scoping reviews. DATA SOURCES: A comprehensive list of scoping reviews conducted in the broader health field (inception to July 2014), with a further update of that list (up to February 2017) using similar methods, including searching 9 electronic databases. STUDY SELECTION: Articles were included if they were scoping reviews within rehabilitation. Established review methods were used including (1) a PubMed filter detecting rehabilitation content and (2) title-and-abstract screening by 2 independent reviewers applied sequentially to articles from the existing list of scoping reviews and to the updated search results. Full-text articles were reviewed by 1 reviewer, with discrepancies resolved by another after pilot screening with > 80% agreement. Remaining discrepancies were resolved by external experts. DATA EXTRACTION: Two independent reviewers used piloted and standardized data extraction forms. DATA SYNTHESIS: We screened 1823 records, including 992 full texts, to identify 251 rehabilitation-related scoping reviews. Rehabilitation scoping reviews had an exponential yearly increase since 2008 (r2=0.89; P<.01). The literature addressed diverse topics (eg, spread over 43 condition groupings); 43% were published in Canada. Examples of methodological limitations included: 39% of reviews did not cite the use of a methodological framework, 96% did not include the appropriate flow diagram, 8% did not report eligibility criteria, and 57% did not report data extraction details. CONCLUSIONS: The increasing popularity of scoping reviews in rehabilitation has not been met by high standards in methodological quality. To increase the value of rehabilitation scoping reviews, rehabilitation stakeholders need to use existing methodological standards for the conduct, reporting, and appraisal of scoping reviews.

Clinical performance comparators in audit and feedback: a review of theory and evidence.

BACKGROUND: Audit and feedback (A&F) is a common quality improvement strategy with highly variable effects on patient care. It is unclear how A&F effectiveness can be maximised. Since the core mechanism of action of A&F depends on drawing attention to a discrepancy between actual and desired performance, we aimed to understand current and best practices in the choice of performance comparator. METHODS: We described current choices for performance comparators by conducting a secondary review of randomised trials of A&F interventions and identifying the associated mechanisms that might have implications for effective A&F by reviewing theories and empirical studies from a recent qualitative evidence synthesis. RESULTS: We found across 146 trials that feedback recipients' performance was most frequently compared against the performance of others (benchmarks; 60.3%). Other comparators included recipients' own performance over time (trends; 9.6%) and target standards (explicit targets; 11.0%), and 13% of trials used a combination of these options. In studies featuring benchmarks, 42% compared against mean performance. Eight (5.5%) trials provided a rationale for using a specific comparator. We distilled mechanisms of each comparator from 12 behavioural theories, 5 randomised trials, and 42 qualitative A&F studies. CONCLUSION: Clinical performance comparators in published literature were poorly informed by theory and did not explicitly account for mechanisms reported in qualitative studies. Based on our review, we argue that there is considerable opportunity to improve the design of performance comparators by (1) providing tailored comparisons rather than benchmarking everyone against the mean, (2) limiting the amount of comparators being displayed while providing more comparative information upon request to balance the feedback's credibility and actionability, (3) providing performance trends but not trends alone, and (4) encouraging feedback recipients to set personal, explicit targets guided by relevant information.

Publication trends of study protocols in rehabilitation.

INTRODUCTION: Growing evidence points for the need to publish study protocols in the health field. The aim of this paper was to observe whether the growing interest in publishing study protocols in the broader health field has been translated into increased publications of rehabilitation study protocols. EVIDENCE ACQUISITION: PubMed was searched with appropriate combinations of Medical Subject Headings up to December 2014. The effective presence of study protocols was manually screened. Regression models analyzed the yearly growth of publications. Two-sample Z-tests analyzed whether the proportion of systematic reviews (SRs) and randomized controlled trials (RCTs) among study protocols differed from that of the same designs for the broader rehabilitation research. EVIDENCE SYNTHESIS: Up to December 2014, 746 publications of rehabilitation study protocols were identified, with an exponential growth since 2005 (r2=0.981; P<0.001). RCT protocols were the most common among rehabilitation study protocols (83%), while RCTs were significantly more prevalent among study protocols than among the broader rehabilitation research (83% vs. 35.8%; P<0.001). For SRs, the picture was reversed: significantly less common among study protocols (2.8% vs. 9.3%; P<0.001). Funding was more often reported by rehabilitation study protocols than the broader rehabilitation research (90% vs. 53.1%; P<0.001). Rehabilitation journals published a significantly lower share of rehabilitation study protocols than they did for the broader rehabilitation research (1.8% vs.16.7%; P<0.001). CONCLUSIONS: Identifying the reasons for these discrepancies and reverting unwarranted disparities (e.g. low rate of publication for rehabilitation SR protocols) are likely new avenues for rehabilitation research and its publication. SRs, particularly those aggregating RCT results, are considered the best standard of evidence to guide rehabilitation clinical practice; however, that standard can be improved in rigor and/or transparency if the publications of rehabilitation SRs protocols become more common.

Advancing the literature on designing audit and feedback interventions: identifying theory-informed hypotheses.

BACKGROUND: Audit and feedback (A&F) is a common strategy for helping health providers to implement evidence into practice. Despite being extensively studied, health care A&F interventions remain variably effective, with overall effect sizes that have not improved since 2003. Contributing to this stagnation is the fact that most health care A&F interventions have largely been designed without being informed by theoretical understanding from the behavioral and social sciences. To determine if the trend can be improved, the objective of this study was to develop a list of testable, theory-informed hypotheses about how to design more effective A&F interventions. METHODS: Using purposive sampling, semi-structured 60-90-min telephone interviews were conducted with experts in theories related to A&F from a range of fields (e.g., cognitive, health and organizational psychology, medical decision-making, economics). Guided by detailed descriptions of A&F interventions from the health care literature, interviewees described how they would approach the problem of designing improved A&F interventions. Specific, theory-informed hypotheses about the conditions for effective design and delivery of A&F interventions were elicited from the interviews. The resulting hypotheses were assigned by three coders working independently into themes, and categories of themes, in an iterative process. RESULTS: We conducted 28 interviews and identified 313 theory-informed hypotheses, which were placed into 30 themes. The 30 themes included hypotheses related to the following five categories: A&F recipient (seven themes), content of the A&F (ten themes), process of delivery of the A&F (six themes), behavior that was the focus of the A&F (three themes), and other (four themes). CONCLUSIONS: We have identified a set of testable, theory-informed hypotheses from a broad range of behavioral and social science that suggest conditions for more effective A&F interventions. This work demonstrates the breadth of perspectives about A&F from non-healthcare-specific disciplines in a way that yields testable hypotheses for healthcare A&F interventions. These results will serve as the foundation for further work seeking to set research priorities among the A&F research community.

Methods for designing interventions to change healthcare professionals' behaviour: a systematic review.

BACKGROUND: Systematic reviews consistently indicate that interventions to change healthcare professional (HCP) behaviour are haphazardly designed and poorly specified. Clarity about methods for designing and specifying interventions is needed. The objective of this review was to identify published methods for designing interventions to change HCP behaviour. METHODS: A search of MEDLINE, Embase, and PsycINFO was conducted from 1996 to April 2015. Using inclusion/exclusion criteria, a broad screen of abstracts by one rater was followed by a strict screen of full text for all potentially relevant papers by three raters. An inductive approach was first applied to the included studies to identify commonalities and differences between the descriptions of methods across the papers. Based on this process and knowledge of related literatures, we developed a data extraction framework that included, e.g. level of change (e.g. individual versus organization); context of development; a brief description of the method; tasks included in the method (e.g. barrier identification, component selection, use of theory). RESULTS: 3966 titles and abstracts and 64 full-text papers were screened to yield 15 papers included in the review, each outlining one design method. All of the papers reported methods developed within a specific context. Thirteen papers included barrier identification and 13 included linking barriers to intervention components; although not the same 13 papers. Thirteen papers targeted individual HCPs with only one paper targeting change across individual, organization, and system levels. The use of theory and user engagement were included in 13/15 and 13/15 papers, respectively. CONCLUSIONS: There is an agreement across methods of four tasks that need to be completed when designing individual-level interventions: identifying barriers, selecting intervention components, using theory, and engaging end-users. Methods also consist of further additional tasks. Examples of methods for designing the organisation and system-level interventions were limited. Further analysis of design tasks could facilitate the development of detailed guidelines for designing interventions.

Study protocol for a scoping review on rehabilitation scoping reviews.

INTRODUCTION: Scoping reviews are increasingly popular in rehabilitation. However, significant variability in scoping review conduct and reporting currently exists, limiting potential for the methodology to advance rehabilitation research, practice and policy. Our aim is to conduct a scoping review of rehabilitation scoping reviews in order to examine the current volume, yearly distribution, proportion, scope and methodological practices involved in the conduct of scoping reviews in rehabilitation. Key areas of methodological improvement will be described. Methods and analysis: We will undertake the review using the Arksey and O'Malley scoping review methodology. Our search will involve two phases. The first will combine a previously conducted scoping review of scoping reviews (not distinct to rehabilitation, with data current to July 2014) together with a rehabilitation keyword search in PubMed. Articles found in the first phase search will undergo a full text review. The second phase will include an update of the previously conducted scoping review of scoping reviews (July 2014 to current). This update will include the search of nine electronic databases, followed by title and abstract screening as well as a full text review. All screening and extraction will be performed independently by two authors. Articles will be included if they are scoping reviews within the field of rehabilitation. A consultation exercise with key targets will inform plans to improve rehabilitation scoping reviews. Ethics and dissemination: Ethics will be required for the consultation phase of our scoping review. Dissemination will include peer-reviewed publication and conferences in rehabilitation-specific contexts.

A systematic review of interventions to increase the use of standardized outcome measures by rehabilitation professionals.

OBJECTIVE: To determine the types and effectiveness of interventions to increase the knowledge about, attitudes towards, and use of standardized outcome measures in rehabilitation professionals. DATA SOURCES: An electronic search using Medline, EMBASE, PsycINFO, CINAHL, Ergonomics Abstracts, Sports Discus. The search is current to February 2016. STUDY SELECTION: All study designs testing interventions were included as were all provider and patient types. Two reviewers independently conducted a title and abstract review, followed by a full-text review. DATA EXTRACTION: Two reviewers independently extracted a priori variables and used consensus for disagreements. Quality assessment was conducted using the Assessment of Quantitative Studies published by the Effective Public Health Practice Group. DATA SYNTHESIS: We identified 11 studies involving at least 1200 providers. Nine of the studies showed improvements in outcome measure use rates but only three of these studies used an experimental or quasi-experimental design. Eight of the studies used an educational approach in the intervention and three used audit and feedback. Poor intervention description and quality of studies limited recommendations. CONCLUSIONS: Increased attention to testing interventions focused on known barriers, matched to behavior change techniques, and with stronger designs is warranted.

Evaluation of a training program for medicines-oriented policymakers to use a database of systematic reviews.

BACKGROUND: Suboptimal prescribing and medications use is a problem for health systems globally. Systematic reviews are a comprehensive resource that can help guide evidence-informed decision-making and implementation of interventions addressing such issues; however, a barrier to the use of systematic reviews is their inaccessibility (due to both dispersion across journals and inaccessibility of content). Publicly available databases, such as Rx for Change, provide quick access to summaries of appraised systematic reviews of professional and consumer-oriented interventions to improve prescribing behaviour and appropriate medication use, and may help maximise the use of evidence to inform decisions. The present study aims to evaluate a training program to improve attitudes towards, confidence in skills, intentions to use, and use of systematic review evidence contained within Rx for Change. METHODS: Guided by the Knowledge to Action framework, a training program with content customised to local provider and consumer contexts was developed with knowledge user input. The training program consisted of a 6 minute information video, a 1 hour workshop with hands-on, interactive and didactic components, and two post-training reminders. Forty-nine people from five medicines-focused organisations in Canada and Australia attended one of six workshops. Participants were surveyed immediately pre and post and 3 months after training to evaluate their attitudes towards, confidence in skills, intentions to use, and use of Rx for Change, and attitudes towards and confidence in skills for using evidence for decision-making. Analyses for differences for each of the outcomes at three time points (pre, post and 3 months after training) was performed using a random effects model. RESULTS: Immediately post-training, there were higher respondent attitudes towards Rx for Change (mean increase = 0.54 out of 5, 95% CI, 0.18-0.83, P < 0.005); intention to use Rx for Change (0.53, 95% CI, 0.21-0.86, P < 0.005); confidence in skills for using Rx for Change (2.08, 95% CI, 1.74-2.42, P < 0.005); and confidence in skills for using evidence in policy decision-making (0.50, 95% CI, 0.22-0.77, P < .005) compared to pre-training. Confidence in skills for using both Rx for Change and evidence were maintained 3 months after training (both P < 0.005). CONCLUSIONS: Participants of this training program reported sustained improvements in their confidence in skills for using evidence in policy decision-making. This may have important implications for uptake of systematic review evidence promoting improved prescribing and medication use.

Practice Feedback Interventions: 15 Suggestions for Optimizing Effectiveness.

Electronic practice data are increasingly being used to provide feedback to encourage practice improvement. However, evidence suggests that despite decades of experience, the effects of such interventions vary greatly and are not improving over time. Guidance on providing more effective feedback does exist, but it is distributed across a wide range of disciplines and theoretical perspectives. Through expert interviews; systematic reviews; and experience with providing, evaluating, and receiving practice feedback, 15 suggestions that are believed to be associated with effective feedback interventions have been identified. These suggestions are intended to provide practical guidance to quality improvement professionals, information technology developers, educators, administrators, and practitioners who receive such interventions. Designing interventions with these suggestions in mind should improve their effect, and studying the mechanisms underlying these suggestions will advance a stagnant literature.

Scoping reviews: time for clarity in definition, methods, and reporting.

OBJECTIVES: The scoping review has become increasingly popular as a form of knowledge synthesis. However, a lack of consensus on scoping review terminology, definition, methodology, and reporting limits the potential of this form of synthesis. In this article, we propose recommendations to further advance the field of scoping review methodology. STUDY DESIGN AND SETTING: We summarize current understanding of scoping review publication rates, terms, definitions, and methods. We propose three recommendations for clarity in term, definition and methodology. RESULTS: We recommend adopting the terms "scoping review" or "scoping study" and the use of a proposed definition. Until such time as further guidance is developed, we recommend the use of the methodological steps outlined in the Arksey and O'Malley framework and further enhanced by Levac et al. The development of reporting guidance for the conduct and reporting of scoping reviews is underway. CONCLUSION: Consistency in the proposed domains and methodologies of scoping reviews, along with the development of reporting guidance, will facilitate methodological advancement, reduce confusion, facilitate collaboration and improve knowledge translation of scoping review findings.

A systematic review of the use of theory in randomized controlled trials of audit and feedback.

BACKGROUND: Audit and feedback is one of the most widely used and promising interventions in implementation research, yet also one of the most variably effective. Understanding this variability has been limited in part by lack of attention to the theoretical and conceptual basis underlying audit and feedback. Examining the extent of theory use in studies of audit and feedback will yield better understanding of the causal pathways of audit and feedback effectiveness and inform efforts to optimize this important intervention. METHODS: A total of 140 studies in the 2012 Cochrane update on audit and feedback interventions were independently reviewed by two investigators. Variables were extracted related to theory use in the study design, measurement, implementation or interpretation. Theory name, associated reference, and the location of theory use as reported in the study were extracted. Theories were organized by type (e.g., education, diffusion, organization, psychology), and theory utilization was classified into seven categories (justification, intervention design, pilot testing, evaluation, predictions, post hoc, other). RESULTS: A total of 20 studies (14%) reported use of theory in any aspect of the study design, measurement, implementation or interpretation. In only 13 studies (9%) was a theory reportedly used to inform development of the intervention. A total of 18 different theories across educational, psychological, organizational and diffusion of innovation perspectives were identified. Rogers' Diffusion of Innovations and Bandura's Social Cognitive Theory were the most widely used (3.6% and 3%, respectively). CONCLUSIONS: The explicit use of theory in studies of audit and feedback was rare. A range of theories was found, but not consistency of theory use. Advancing our understanding of audit and feedback will require more attention to theoretically informed studies and intervention design.

Administration of the Canadian Occupational Performance Measure: effect on practice.

BACKGROUND: The Canadian Occupational Performance Measure (COPM) is recommended as a systematic approach to identify issues and determine client progress in occupational therapy, yet little empirical evidence is available that supports this practice. PURPOSE: To determine if COPM administration was associated with changes in eight dimensions of occupational therapy practice. METHODS: Twenty-four occupational therapists on eight geriatric rehabilitation sites completed a before-and-after study with a repeated baseline. The eight practice dimensions were assessed after three months of usual care (no COPM use) and after three months of intervention (COPM use) using chart stimulated recall (CSR) interviews and chart audit. FINDINGS: Mean practice scores for CSR interviews indicated a statistically significant practice improvement (p < .0001) across the eight dimensions, including knowledge of client perspective, clinical decision making, clinician ability to articulate outcomes, and documentation. Chart audit indicated that COPM use resulted in identifying more occupation-focused issues. IMPLICATIONS: COPM administration could improve occupational therapy practice.

Creating a knowledge translation trainee collaborative: from conceptualization to lessons learned in the first year.

Trainees (e.g., graduate students, residents, fellows) are increasingly identifying knowledge translation as their research discipline. In Canada, a group of trainees have created a trainee-initiated and trainee-led national collaborative to provide a vehicle for trainees to examine the diversity of knowledge translation research and practice, and to link trainees from diverse geographical areas and disciplines. The aim of this paper is to describe our experience and lessons learned in creating the Knowledge Translation Trainee Collaborative. In this meeting report, we outline the process, challenges, and opportunities in planning and experiencing the collaborative's inaugural meeting as participant organizers, and present outcomes and learnings to date.

A scoping review of the use of theory in studies of knowledge translation.

BACKGROUND: Advancing the science of knowledge translation (KT) in occupational therapy is critical. Explicit application of theory can advance this science; yet, how theory is applied and the degree to which it can guide research remain poorly defined. PURPOSE: To understand how theory is applied within KT research. METHODS: A scoping review was conducted to examine and summarize the extent, range, and nature of the application of three specific KT theories: Diffusion of Innovations, Promoting Action on Research Implementation in Health Services framework, and Theory of Planned Behaviour. FINDINGS: Theory use was seen most frequently in medicine and nursing. Only 3 of 90 articles were in rehabilitation. Five approaches to theory application were found, the most common being the use of to predict success of KT (57/90). IMPLICATIONS: In-depth study of the importance and methods of theory application in KT research is needed, in particular in occupational therapy.