Multimethods study to develop tools for competency-based assessments of implementation research training programmes in low and middle-income countries.

INTRODUCTION: Methods and validated tools for evaluating the performance of competency-based implementation research (IR) training programmes in low-middle-income countries (LMICs) are lacking. In this study, we developed tools for assessing the performance of IR training programmes based on a framework of IR core competencies in LMICs. METHODS: We developed self-assessment and objective-assessment tools drawing on the IR competency framework. We used exploratory factor analyses and a one-parameter logistic model to establish construct validity and internal consistency of the tools drawing on a survey conducted in 2020 with 166 trainees before and after an index IR course across five universities and LMICs under the Special Program for Research and Training in Tropical Diseases postgraduate IR training scheme. We conducted key informant interviews (KII) with 20 trainees and five trainers to reflect on the usefulness of the tools and framework for guiding IR training in LMICs. RESULTS: Two 16-item tools for self-assessment of IR knowledge and self-efficacy and a 40-item objective assessment tool were developed. The factor loadings of items in the self-assessment tools were 0.65-0.87 with Cronbach's alpha (α) of 0.97, and 0.77-0.91 with α of 0.98 for the IR knowledge and self-efficacy tools, respectively. The distribution of item difficulty in the objective-assessment tool was consistent before and after the index IR course. Pearson correlation coefficient (r) between the self-assessed and objectively assessed IR knowledge before the index IR course was low, r=0.27 (p value: <0.01), with slight improvements after the index IR course, r=0.43 (p value: <0.01). All KII respondents reported the assessment tools and framework were valid for assessing IR competencies. CONCLUSION: The IR competency framework and tools developed for assessing IR competencies and performance of IR training programmes in LMICs are reliable and valid. Self-assessment methods alone may not suffice to yield a valid assessment of performance in these settings.

Developing a framework of core competencies in implementation research for low/middle-income countries.

The field of implementation research (IR) is growing. However, there are no recognised IR core competencies in low/middle-income countries (LMICs), nor consistent curriculum across IR training programs globally. The goal of this effort is to develop a framework of IR core competencies for training programs in LMICs. The framework was developed using a mixed-methods approach consisting of two online surveys with IR training coordinators (n = 16) and academics (n = 89) affiliated with seven LMIC institutions, and a modified-Delphi process to evaluate the domains, competencies and proficiency levels included in the framework. The final framework comprised of 11 domains, 59 competencies and 52 sub-competencies, and emphasised competencies for modifying contexts, strengthening health systems, addressing ethical concerns, engaging stakeholders and communication especially for LMIC settings, in addition to competencies on IR theories, methods and designs. The framework highlights the interconnectedness of domains and competencies for IR and practice, and training in IR following the outlined competencies is not a linear process but circular and iterative, and starting points for training may vary widely by the project, institution and challenge being addressed. The framework established the need for a theory-based approach to identifying proficiency levels for IR competencies (ie, to determine proficiency levels for IR based on generalisable educational theories for competency-based education), and the relevance of various IR competencies for LMICs compared with high-income settings. This framework is useful for identifying and evaluating competencies and trainings, and providing direction and support for professional development in IR.