Factors influencing clinician-educators' assessment practice in varied Southern contexts: a health behaviour theory perspective.

In many contexts, responsibility for exit-level assessment design and implementation in undergraduate medical programmes lies with individuals who convene clinical clerkships. Their assessment practice has significant consequences for students' learning and the patients and communities that graduates will serve. Interventions to enhance assessment must involve these assessors, yet little is known about factors influencing their assessment practice. The purpose of this study was to explore factors that influence assessment practice of clerkship convenors in three varied low-and-middle income contexts in the global South. Taking assessment practice as a behaviour, Health Behaviour Theory (HBT) was deployed as a theoretical framework to explore, describe and explain assessor behaviour. Thirty-one clinician-educators responsible for designing and implementing high-stakes clerkship assessment were interviewed in South Africa and Mexico. Interacting personal and contextual factors influencing clinician-educator assessment intention and action were identified. These included attitude, influenced by impact and response appraisal, and perceived self-efficacy; along with interpersonal, physical and organisational, and distal contextual factors. Personal competencies and conducive environments supported intention to action transition. While previous research has typically explored factors in isolation, the HBT framing enabled a systematic and coherent account of assessor behaviour. These findings add a particular contextual perspective to understanding assessment practice, yet also resonate with and extend existing work that predominantly emanates from high-income contexts in the global North. These findings provide a foundation for the planning of assessment change initiatives, such as targeted, multi-factorial faculty development.

Developing a programmatic approach to faculty development and scholarship using the ASPIRE criteria: AMEE Guide No. 165.

Faculty Development (FD) has become essential in shaping design, delivery and quality assurance of health professions education. The growth of FD worldwide has led to a heightened expectation for quality and organizational integrity in the delivery of FD programmes. To address this, AMEE, An International Association for Health Professions Education, developed quality standards for FD through the development of the AMEE ASPIRE to Excellence criteria. This guide uses the ASPIRE criteria as a framework for health professions educators who wish to establish or expand approaches to FD delivery and scholarship within their institutions.

What happens to misunderstandings of biomedical concepts across a medical curriculum?

Research on the extent and nature of commonly misunderstood fundamental biomedical concepts across a medical curriculum is scarce. These misunderstandings could point toward robust misconceptions. We examined first whether common misunderstandings persist throughout a medical curriculum, followed by a fine-grained analysis to identify their nature. We designed and administered a 2-tier test to 987 medical students across our curriculum, with 8 questions covering the respiratory and cardiovascular systems, cell division, and homeostatic processes. Proportions of incorrect responses were computed. Four questions where misunderstandings persisted were further qualitatively analyzed. A one-way ANOVA showed the proportion of incorrect responses decreased significantly by students' academic year [F(6, 986) = 96.05, P < 0.001]. While novices and end-of -first-year students showed similar proportion of incorrect responses (P > 0.05), incorrect responses decreased significantly between first years and second years (P < 0.001). Thereafter, the proportion of incorrect responses remained stable from second to final year (P > 0.05), with ∼35% of incorrect responses. Five questions showed no decrease of incorrect responses between second and final years, with two questions where final year students performed marginally better than novices. A Chi-square analysis, with Bonferroni post hoc test, showed certain misunderstandings appeared frequently across the curriculum. The qualitative analysis of the open-ended questions yielded 15 categories of common misunderstandings of fundamental biomedical concepts in all years of training. If educators become aware of commonly misunderstood biomedical concepts, preventative measures could be taken to prevent robust misconceptions.

The impact of a training programme incorporating the conceptual framework of the International Classification of Functioning (ICF) on knowledge and attitudes regarding interprofessional practice in Rwandan health professionals: a cluster randomized control trial.

BACKGROUND: The first step in improving interprofessional teamwork entails training health professionals (HP) to acknowledge the role and value the contribution of each member of the team. The International Classification of Functioning, Disability and Health (ICF) has been developed by WHO to provide a common language to facilitate communication between HPs. OBJECTIVE: To determine whether ICF training programme would result in improved knowledge and attitudes regarding interprofessional practice within Rwandan district hospitals. DESIGN, SETTING AND PARTICIPANTS: A cluster randomised, single blinded, control trial design was used to select four district hospitals. Participants included physicians, social workers, physiotherapists, nutritionists, clinical psychologists/mental health nurses. INTERVENTION: Health professionals either received one day's training in interprofessional practice (IPP) based on the ICF (experimental group) as a collaborative framework or a short talk on the topic (control group). OUTCOME MEASURES: Validated questionnaires were used to explore changes in knowledge and attitudes. Ethical approval was obtained from the relevant authorities. RESULTS: There were 103 participants in the experimental and 100 in the control group. There was no significant difference between Knowledge and Attitude scales at baseline. Post-intervention the experimental group (mean = 41.3, SD = 9.5) scored significantly higher on the knowledge scale than the control group (mean = 17.7, SD = 4.7 (t = 22.5; p < .001)). The median scores on the Attitude Scale improved in the Experimental group from 77.8 to 91.1%, whereas the median scores of the control remained approximately 80% (Adjusted Z = 10.72p < .001). CONCLUSION: The ICF proved to be a useful framework for structuring the training of all HPs in IPP and the training resulted in a significant improvement in knowledge and attitudes regarding IPP. As suggested by the HPs, more training and refresher courses were needed for sustainability and the training should be extended to other hospitals in Rwanda. It is thus recommended that the framework can be used in interprofessional education and practice in Rwanda and possibly in other similar countries. TRIAL REGISTRATION: Name of the registry: Pan African Clinical Trial Registry. TRIAL REGISTRATION NUMBER: PACTR201604001185358 . Date of registration: 22/04/2016. URL of trial registry record: www.pactr.org.

Performance assessment: Consensus statement and recommendations from the 2020 Ottawa Conference.

INTRODUCTION: In 2011 the Consensus Statement on Performance Assessment was published in Medical Teacher. That paper was commissioned by AMEE (Association for Medical Education in Europe) as part of the series of Consensus Statements following the 2010 Ottawa Conference. In 2019, it was recommended that a working group be reconvened to review and consider developments in performance assessment since the 2011 publication. METHODS: Following review of the original recommendations in the 2011 paper and shifts in the field across the past 10 years, the group identified areas of consensus and yet to be resolved issues for performance assessment. RESULTS AND DISCUSSION: This paper addresses developments in performance assessment since 2011, reiterates relevant aspects of the 2011 paper, and summarises contemporary best practice recommendations for OSCEs and WBAs, fit-for-purpose methods for performance assessment in the health professions.

The impact of a training programme incorporating the conceptual framework of the International Classification of Functioning (ICF) on behaviour regarding interprofessional practice in Rwandan health professionals: A cluster randomized control trial.

BACKGROUND: Appropriate collaboration between health professionals (HP) can reduce medical errors, enhance the spread of critical information, and assist in interpretation of health information resulting in improved patient care. The International Classification of Functioning, Disability and Health (ICF) may provide a useful conceptual framework to facilitate better interprofessional practice. PURPOSE: To determine whether a training programme based on the ICF framework resulted in improved interprofessional behaviour among HPs in Rwanda. METHODOLOGY: A cluster randomised control trial was used. Four district hospitals were randomly allocated to receive either a day's training in interprofessional practice based on the ICF framework (experimental) or a short talk and a booklet on the topic (control). A total of 203 participants included medical doctors, nurses, and other HPs took part in this study. Simple random sampling was used to select the hospital records of 200 patients discharged from relevant wards at both the experimental and control hospitals at baseline and at two, four and six months after training (800 patients' records from each group). A self-designed checklist has undergone some validation and was based on the ICF conceptual model was used to audit the quality of information included in the patients' records. Ethical approval was obtained from the relevant authorities. RESULTS: The demographic and medical profile of the patients in the two sets was equivalent. An ANOVA and post-hoc Tukey test indicated the mean number of items correctly filled in was not significant at baseline (p = 0.424) but the difference was significant (p < .001) for the post-intervention scores at two, four and six months. The control group scores did not improve over time. The improved behavior was still evident at six months although it had begun to decay. CONCLUSION: Behaviour change as evidenced by more comprehensive recording of patient management can result from a well-structured training programme. The ICF appeared to provide a common language and facilitate HPs interaction and patient management plans. IMPLICATION: The ICF provided an effective conceptual framework to structure the content of the training and the audit tool. It is recommended that the framework be used to facilitate interprofessional education and practice in Rwanda and that the training approach may be applicable to other health care contexts.

The learning environment in remediation: a review.

BACKGROUND: The focus of this concise article is how best to support students to achieve success at medical school. Our aim is not to provide a guide to remediating under-performance in medical students. This, in our view, implies an approach that fundamentally is about quick fixes for addressing individual student deficits, such as intensive coaching of clinical skills to help a student scrape through a resit examination. Instead, we believe that student success is not solely the result of individual factors but rather relies on a complex range of factors, including the provision of a supportive environment. METHODS: We drew on our knowledge of a wide range of literature related to remediation and other medical education structures and functions. Our aim was to take a different perspective on the different dimensions of 'remediation' - the structural, curricular, ideological and individual - to consider how best to provide a supportive environment for all learners to progress towards the required outcomes. CONCLUSION: Medical students are becoming increasingly diverse and medical curricula must create learning environments that support all students to thrive. Effective remediation should not be about intensive 'teaching to test' after examination failure. Rather, both the context and the individual have a role to play in ensuring the selection, teaching, assessment and feedback practices support the learning journeys of individuals. We provide guidance for faculty member development and engaging with students to help achieve this goal. Effective remediation should not be about intensive 'teaching to test' after examination failure.

The supervisor's toolkit: A framework for doctoral supervision in health professions education: AMEE Guide No. 104.

Doctoral studies represent a complex undertaking for students and supervisors. Some research describes the experience of students while there are volumes of advice for students considering a doctorate. Yet the terrain for supervisors is less well-trodden and the concept of a pedagogy of supervision is only really starting to emerge. Texts on the doctoral journey from the supervisor's perspective are uncommon and less yet has been written in the context of health professions education. The aim of this Guide, therefore, is to provide guidance for the supervisor's journey, drawing on our collective experience and such literature as there is. We explore the doctoral journey of students and their supervisors, highlighting what the implications are for supervisory practice. Recognising the doctorate as much more than merely conducting a research project, and seeing it as a shared educational endeavour is fundamental to understanding the doctoral journey - a journey that is complex and mutable, constantly shifting as the candidate moves from novice to expert, from dependence to growing autonomy. Our intention is to present this Guide as a toolkit for both the novice and the experienced supervisor as it, on the one hand, seeks to make the practice of supervision more transparent while on the other, challenges the reader to critically reflect on the supervisory space in which they currently reside. Our hope is that the Guide opens up opportunities for generative conversations about the practice of doctoral supervision in health professions education.

Equipping family physician trainees as teachers: a qualitative evaluation of a twelve-week module on teaching and learning.

BACKGROUND: There is a dire need to expand the capacity of institutions in Africa to educate health care professionals. Family physicians, as skilled all-rounders at district level, are potentially well placed to contribute to an extended training platform in this context. To play this role, they need to both have an understanding of their specialist role that incorporates teaching and be equipped for their role as trainers of current and future health workers and specialists. A teaching and learning capacity-building module was introduced into a new master's programme in family medicine at Stellenbosch University, South Africa. We report on the influence of this module on graduates after the first six years. METHODS: A qualitative study was undertaken, interviewing thirteen graduates of the programme. Thematic analysis of data was done by a team comprising tutors and graduates of the programme and an independent researcher. Ethical clearance was obtained. RESULTS: The module influenced knowledge, skills and attitudes of respondents. Perceptions and evidence of changes in behaviour, changes in practice beyond the individual respondent and benefits to students and patients were apparent. Factors underlying these changes included the role of context and the role of personal factors. Contextual factors included clinical workload and opportunity pressure i.e., the pressure and responsibility to undertake teaching. Personal factors comprised self-confidence, modified attitudes and perceptions towards the roles of a family physician and towards learning and teaching, in addition to the acquisition of knowledge and skills in teaching and learning. The interaction between opportunity pressure and self-confidence influenced the application of what was learned about teaching. CONCLUSIONS: A module on teaching and learning influenced graduates' perceptions of, and self-reported behaviour relating to, teaching as practicing family physicians. This has important implications for educating family physicians in and for Africa and indirectly on expanding capacity to educate health care professionals in Africa.

Modelling the pre-assessment learning effects of assessment: evidence in the validity chain.

OBJECTIVES: We previously developed a model of the pre-assessment learning effects of consequential assessment and started to validate it. The model comprises assessment factors, mechanism factors and learning effects. The purpose of this study was to continue the validation process. For stringency, we focused on a subset of assessment factor-learning effect associations that featured least commonly in a baseline qualitative study. Our aims were to determine whether these uncommon associations were operational in a broader but similar population to that in which the model was initially derived. METHODS: A cross-sectional survey of 361 senior medical students at one medical school was undertaken using a purpose-made questionnaire based on a grounded theory and comprising pairs of written situational tests. In each pair, the manifestation of an assessment factor was varied. The frequencies at which learning effects were selected were compared for each item pair, using an adjusted alpha to assign significance. The frequencies at which mechanism factors were selected were calculated. RESULTS: There were significant differences in the learning effect selected between the two scenarios of an item pair for 13 of this subset of 21 uncommon associations, even when a p-value of < 0.00625 was considered to indicate significance. Three mechanism factors were operational in most scenarios: agency; response efficacy, and response value. CONCLUSIONS: For a subset of uncommon associations in the model, the role of most assessment factor-learning effect associations and the mechanism factors involved were supported in a broader but similar population to that in which the model was derived. Although model validation is an ongoing process, these results move the model one step closer to the stage of usefully informing interventions. Results illustrate how factors not typically included in studies of the learning effects of assessment could confound the results of interventions aimed at using assessment to influence learning.

A model of the pre-assessment learning effects of assessment is operational in an undergraduate clinical context.

BACKGROUND: No validated model exists to explain the learning effects of assessment, a problem when designing and researching assessment for learning. We recently developed a model explaining the pre-assessment learning effects of summative assessment in a theory teaching context. The challenge now is to validate this model. The purpose of this study was to explore whether the model was operational in a clinical context as a first step in this process. METHODS: Given the complexity of the model, we adopted a qualitative approach. Data from in-depth interviews with eighteen medical students were subject to content analysis. We utilised a code book developed previously using grounded theory. During analysis, we remained alert to data that might not conform to the coding framework and open to the possibility of deploying inductive coding. Ethical clearance and informed consent were obtained. RESULTS: The three components of the model i.e., assessment factors, mechanism factors and learning effects were all evident in the clinical context. Associations between these components could all be explained by the model. Interaction with preceptors was identified as a new subcomponent of assessment factors. The model could explain the interrelationships of the three facets of this subcomponent i.e., regular accountability, personal consequences and emotional valence of the learning environment, with previously described components of the model. CONCLUSIONS: The model could be utilized to analyse and explain observations in an assessment context different to that from which it was derived. In the clinical setting, the (negative) influence of preceptors on student learning was particularly prominent. In this setting, learning effects resulted not only from the high-stakes nature of summative assessment but also from personal stakes, e.g. for esteem and agency. The results suggest that to influence student learning, consequences should accrue from assessment that are immediate, concrete and substantial. The model could have utility as a planning or diagnostic tool in practice and research settings.

A model of the pre-assessment learning effects of summative assessment in medical education.

It has become axiomatic that assessment impacts powerfully on student learning. However, surprisingly little research has been published emanating from authentic higher education settings about the nature and mechanism of the pre-assessment learning effects of summative assessment. Less still emanates from health sciences education settings. This study explored the pre-assessment learning effects of summative assessment in theoretical modules by exploring the variables at play in a multifaceted assessment system and the relationships between them. Using a grounded theory strategy, in-depth interviews were conducted with individual medical students and analyzed qualitatively. Respondents' learning was influenced by task demands and system design. Assessment impacted on respondents' cognitive processing activities and metacognitive regulation activities. Individually, our findings confirm findings from other studies in disparate non-medical settings and identify some new factors at play in this setting. Taken together, findings from this study provide, for the first time, some insight into how a whole assessment system influences student learning over time in a medical education setting. The findings from this authentic and complex setting paint a nuanced picture of how intricate and multifaceted interactions between various factors in an assessment system interact to influence student learning. A model linking the sources, mechanism and consequences of the pre-assessment learning effects of summative assessment is proposed that could help enhance the use of summative assessment as a tool to augment learning.

Alienation and engagement in postgraduate training at a South African medical school.

BACKGROUND: The importance of contextual factors, such as the learning environment and sociocultural characteristics of the student, are becoming increasingly evident. Mann [2001. Alternative perspectives on the student experience: Alienation and engagement. Stud High Educ 26(1):7-19.] proposed that all learning experiences can be viewed as either alienating or engaging and Case expanded on this work. AIM: The purpose of this study was to explore perceptions of alienation or engagement as experienced by residents in anatomical pathology at one South African university. METHOD: A cross-sectional case study, with 16 semi-structured interviews was conducted. Residents were categorised as either alienated or engaged, based mainly on workplace experiences. RESULTS: Four relevant dimensions were identified; individual, home, workplace and institution. The personal attributes, strategies for coping and reasons for choosing pathology of alienated residents differed from those with engaged experiences. Poor socioeconomic background and schooling did not lead to predominantly alienating experiences, but this group still lacked some generic skills. In the workplace, two main factors resulting in alienated experiences were the interaction between residents and consultants and residents' comprehension of workplace-based learning. CONCLUSIONS: We present a simple model which may be used to identify factors that engage and alienate students in the learning experience in the workplace-based setting. Addressing these factors can contribute towards a more engaging experience for all residents.

The mechanism of impact of summative assessment on medical students' learning.

It has become axiomatic that assessment impacts powerfully on student learning, but there is a surprising dearth of research on how. This study explored the mechanism of impact of summative assessment on the process of learning of theory in higher education. Individual, in-depth interviews were conducted with medical students and analyzed qualitatively. The impact of assessment on learning was mediated through various determinants of action. Respondents' learning behaviour was influenced by: appraising the impact of assessment; appraising their learning response; their perceptions of agency; and contextual factors. This study adds to scant extant evidence and proposes a mechanism to explain this impact. It should help enhance the use of assessment as a tool to augment learning.

Faculty development: yesterday, today and tomorrow.

Medical education has evolved to become a discipline in its own right. With demands on medical faculties to be socially responsible and accountable, there is now increasing pressure for the professionalisation of teaching practice. Developing a cadre of professional and competent teachers, educators, researchers and leaders for their new roles and responsibilities in medical education requires faculty development. Faculty development is, however, not an easy task. It requires supportive institutional leadership, appropriate resource allocation and recognition for teaching excellence. This guide is designed to assist those charged with preparing faculty for their many new roles in teaching and education in both medical and allied health science education. It provides a historical perspective of faculty development and draws on the medical, health science and higher education literature to provide a number of frameworks that may be useful for designing tailored faculty development programmes. These frameworks can be used by faculty developers to systematically plan, implement and evaluate their staff development programmes. This guide concludes with some of the major trends and driving forces in medical education that we believe will shape future faculty development.

Initiating highly active antiretroviral therapy in sub-Saharan Africa: an assessment of the revised World Health Organization scaling-up guidelines.

OBJECTIVES: To assess the utility of the 2003 revised World Health Organization (WHO) criteria [initiating highly active antiretroviral therapy (HAART) in stage IV, in stage III plus CD4 cell count < 350 x 10(6) cells/l, or in stage I or II plus CD4 cell count < 200 x 10 cells/l] relative to other scenarios of HAART initiation. METHODS: Progression to AIDS and death in 292 patients taking HAART and 974 not taking HAART in a South African institution in 1992-2001, stratifying patients by baseline CD4 cell count and WHO stage. RESULTS: HAART was associated with decreased AIDS [adjusted rate ratio [ARR], 0.16; 95% confidence interval (CI), 0.08-0.31) and death (ARR, 0.10; 95% CI, 0.06-0.18). Benefit of HAART was significant across all WHO stages plus CD4 cell counts. The greatest number of deaths averted was in stages IV [74.0/100 patient-years (PY); 95% CI, 50.2-84.5) and III (32.8/100 PY; 95% CI, 22.4-40.9). AIDS cases averted in stage III (22.0/100 PY; 95% CI, 6.1-26.9) were higher than in stage I and II with CD4 cell count < 200 x 10(6) cells/l (8.9/100 PY 95% CI, 5.6-13.3). Treatment initiation for symptomatic disease resulted in greater benefits than using any CD4 cell thresholds. Application of WHO criteria increased the treatment-eligible proportion from 44.5% to 56.7% (P < 0.05) but did not prevent more death (P > 0.05) than treating symptomatic disease. CONCLUSION: Implementation of the revised WHO guidelines in sub-Saharan Africa may result in a significantly increased number of individuals eligible for treatment but would not be as effective a strategy for preventing death as treating symptomatic disease.