Balancing the effectiveness and cost of online education: A preliminary realist economic evaluation.

PURPOSE: While online learning for faculty development has grown substantially over recent decades, it has been further accelerated in the face of the worldwide pandemic. The effectiveness of online learning has been repeatedly established through systematic reviews and meta-analyses, yet questions remain about its cost-effectiveness. This study evaluates how synchronous online supervision training workshops and their cost-effectiveness might work, and in what contexts. METHODS: We conducted preliminary realist economic evaluation including qualitative (13 realist interviews), and quantitative approaches (cost Ingredients method). We developed a cost-optimised model based on identified costs and cost-sensitive mechanisms. RESULTS: We identified 14 recurring patterns (so-called demi-regularities) illustrating multiple online workshop outcomes (e.g. satisfaction, engagement, knowledge), generated by various mechanisms (e.g. online technology, mixed pedagogies involving didactic and active/experiential learning, peer learning), and triggered by two contexts (supervisor experience levels, and workplace location). Each workshop cost $302.92 per learner, but the optimised model including senior facilitators cost $305.70. CONCLUSIONS: Our initial realist program theories were largely supported and refined. Although findings were largely concordant with previous literature, we illustrate how online workshop costs compare favourably with face-to-face alternatives. We encourage program developers to consider synchronous online learning for faculty development especially for remote learners, and in resource-constrained environments.

Unpacking economic programme theory for supervision training: Preliminary steps towards realist economic evaluation.

INTRODUCTION: Cost studies are increasingly popular given resource constraints. While scholars stress the context-dependent nature of cost, and the importance of theory, cost studies remain context-blind and atheoretical. However, realist economic evaluation (REE) privileges context and the testing/refinement of economic programme theory. This preliminary REE serves to test and refine economic programme theory for supervision training programmes of different durations to better inform future programme design/implementation. METHODS: Our preliminary REE unpacked how short (half-day) and extended (12 week) supervision training programmes in Victoria, Australia, produced costs and outcomes. We employed mixed methods: qualitative realist and quantitative cost methods. Economically optimised programme models were developed guided by identified cost-sensitive mechanisms and contexts. RESULTS: As part of identified context-mechanism-outcome configurations (CMOCs) for both training programmes, we found a wider diversity of positive outcomes but greater costs for the extended programme (11 outcomes; AU$3069/learner) compared with the short programme (7 outcomes; $385/learner). We identified four shared cost-sensitive mechanisms for both programmes (training duration, learner protected time, learner engagement, and facilitator competence) and one shared cost-sensitive context (learners' supervisory experience). We developed two economically optimised programme models: (1) short programme for experienced supervisors facilitated by senior educators ($406/learner); and (2) extended programme for inexperienced supervisors facilitated by junior educators ($2104/learner). DISCUSSION: Our initial economic programme theory was somewhat supported, refuted and refined. Results were partly consistent with previous research, but also extended it through unpacking cost-sensitive mechanisms and contexts. Although our preliminary REE fills a pressing gap in the methodology literature, conducting REE was challenging given our desire to integrate economic and realist analyses fully, and remain faithful to realist principles. Attention to training duration and experience levels of the facilitator-learner dyad may help to balance the cost and outcomes of training programmes.

Supervision training in healthcare: a realist synthesis.

Supervision matters: it serves educational, supportive and management functions. Despite a plethora of evidence on the effectiveness of supervision, scant evidence for the impact of supervision training exists. While three previous literature reviews have begun to examine the effectiveness of supervision training, they fail to explore the extent to which supervision training works, for whom, and why. We adopted a realist approach to answer the question: to what extent do supervision training interventions work (or not), for whom and in what circumstances, and why? We conducted a team-based realist synthesis of the supervision training literature focusing on Pawson's five stages: (1) clarifying the scope; (2) determining the search strategy; (3) study selection; (4) data extraction; and (5) data synthesis. We extracted contexts (C), mechanisms (M) and outcomes (O) and CMO configurations from 29 outputs including short (n = 19) and extended-duration (n = 10) supervision training interventions. Irrespective of duration, interventions including mixed pedagogies involving active and/or experiential learning, social learning and protected time served as mechanisms triggering multiple positive supervisor outcomes. Short-duration interventions also led to positive outcomes through mechanisms such as supervisor characteristics, whereas facilitator characteristics was a key mechanism triggering positive and negative outcomes for extended-duration interventions. Disciplinary and organisational contexts were not especially influential. While our realist synthesis builds on previous non-realist literature reviews, our findings extend previous work considerably. Our realist synthesis presents a broader array of outcomes and mechanisms than have been previously identified, and provides novel insights into the causal pathways in which short and extended-duration supervision training interventions produce their effects. Future realist evaluation should explore further any differences between short and extended-duration interventions. Educators are encouraged to prioritize mixed pedagogies, social learning and protected time to maximize the positive supervisor outcomes from training.

Supervision training interventions in the health and human services: realist synthesis protocol.

INTRODUCTION: Supervision training aims to develop workplace supervisory competencies. Despite extensive supervision literature, including literature reviews, the processes through which supervision training interventions produce their effects, for whom and under what circumstances is not clearly delineated. The purpose of this study is to explain the effect of contextual factors on the underpinning mechanisms of supervision training outcomes. METHODS AND ANALYSIS: We propose to examine supervision training interventions across the health and human services workforce using realist methods. Pawson's five stages for undertaking a realist synthesis will be followed: (1) clarifying the scope of the review; (2) determining the search strategy; (3) study selection; (4) extracting data and (5) synthesising the evidence and drawing conclusions. Extracted data will include study characteristics, characteristics of participant cohort, intervention type, contextual factors, underlying mechanisms and supervision training outcomes. Patterns in context-mechanism-outcome configurations will be identified. Initial programme theories will be developed based on a comprehensive search of the literature, which will include key terms relating to supervision and training. The search strategy will involve: (1) electronic database searching using Medline, Cumulative Index to Nursing and Allied Health Literature, Social Services Abstracts, Educational Resources Information Center, PsycINFO and Australian Public Affairs Information Service and (2) hand and citation searching. We will also contact authors where necessary and discuss identified literature among the project team with extensive expertise in supervision training. ETHICS AND DISSEMINATION: The realist synthesis will propose an evidence-informed theory of supervision training interventions (ie, what interventions work for whom and why). The findings will be disseminated in peer-reviewed journals and presentations and through discussions with relevant organisations and stakeholders. The research will be used by educators to develop evidenced-based supervision training interventions. It will also help workplace supervisors to better understand what types of supervision training might work most optimally for them and their colleagues. Other researchers could use the synthesis findings to guide future supervision research. PROSPERO REGISTRATION NUMBER: CRD42018094186.

What should we teach the teachers? Identifying the learning priorities of clinical supervisors.

Clinicians who teach are essential for the health workforce but require faculty development to improve their educational skills. Curricula for faculty development programs are often based on expert frameworks without consideration of the learning priorities as defined by clinical supervisors themselves. We sought to inform these curricula by highlighting clinical supervisors own requirements through answering the research question: what do clinical supervisors identify as relative strengths and areas for improvement in their teaching practice? This mixed methods study employed a modified version of the Maastricht Clinical Teaching Questionnaire (mMCTQ) which included free-text reflections. Descriptive statistics were calculated and content analysis was conducted on textual comments. 481 (49%) of 978 clinical supervisors submitted their mMCTQs and associated reflections for the research study. Clinical supervisors self-identified relatively strong capability with interpersonal skills or attributes and indicated least capability with assisting learners to explore strengths, weaknesses and learning goals. The qualitative category 'establishing relationships' was the most reported strength with 224 responses. The qualitative category 'feedback' was the most reported area for improvement, with 151 responses. Key areas for curricular focus include: improving feedback practices; stimulating reflective and agentic learning; and managing the logistics of a clinical education environment. Clinical supervisors' self-identified needs provide a foundation for designing engaging and relevant faculty development programs.

Clinical supervision training across contexts.

BACKGROUND: Clinicians require specific skills to teach or supervise students in the workplace; however, there are barriers to accessing faculty member development, such as time, cost and suitability. The Clinical Supervision Support Across Contexts (ClinSSAC) programme was designed to provide accessible interprofessional educator training to clinical supervisors across a wide range of clinical settings. CONTEXT: In Australia there are increasing numbers of health care students, creating pressure on existing placements. Students are now increasingly learning in community settings, where clinicians have traditionally had less access to faculty member development. INNOVATION: An interprofessional team collaborated in the development and implementation of ClinSSAC. A total of 978 clinicians participated in a face-to-face, interactive, introductory module to clinical supervision; 672 people accessed the equivalent online core module, with 23 per cent completing all activities. Additional profession-and discipline-specific modules were also developed. IMPLICATIONS: Formal project evaluation found that most participants rated the workshops as helpful or very helpful for their roles as clinical supervisors. Interdisciplinary learning from the workshops was reported to enable cross-discipline supervision. Large participant numbers and favourable ratings indicate a continuing need for basic training in education. Key factors to workshop success included expert facilitators, the interprofessional context and interactive model. The online modules were an important adjunct, and provided context-specific resources, but the low online completion rate suggests protected face-to-face time for faculty member development is still required. Programmes such as ClinSSAC have the capacity to promote interprofessional education and practice. There are barriers to accessing faculty member development, such as time, cost and suitability.