Measurement of student engagement in health professions education: a review of literature.

Student engagement is a complex multidimensional construct that has attained great interest in health professions education (HPE). Definition and conceptualization of student engagement is an important step that should drive the development of the instruments for its measurement. We have recently proposed a comprehensive framework for student engagement in HPE with a definition of engagement as student investment of time and energy in academic and non-academic experiences that include learning, teaching, research, governance, and community activities. The dimensions of student engagement in this framework included the cognitive, affective, behavioral, agentic, and socio-cultural. Guided by the student engagement framework, this non-systematic review aims to identify, critically appraise, and summarize the existing methods for measuring student engagement in HPE. Extrapolating from higher education literature, we attempted to link the theoretical perspectives of student engagement with the published methods of its measurement in HPE context. In addition, we have described the different methods of measuring student engagement including self-report surveys, real time measures, direct observation, interviews/focus groups, and the use of multiple instruments. The span of engagement dimensions measured by self-report surveys ranges from one to five dimensions. However, measurement of agentic and sociocultural dimensions of engagement in HPE is still limited and further research is required. We have also reflected on the existing methods of measuring engagement of students as active partners in HPE. The review also describes the advantages, limitations, and psychometric properties of each method for measuring student engagement. We ended the review with a guiding conclusion on how to develop and select an instrument for measuring student engagement in HPE. Finally, we addressed the gaps in the literature about measuring engagement of HPE students and future research plans.

Self-assessment and peer-assessment of the psychomotor skills in preclinical prosthodontics: A prospective study.

INTRODUCTION: Students' self-assessment and peer assessments are essential to understand the rubrics criteria and improve their psychomotor skills. The purpose of this study was to assess the impact of self-assessment score (SAS), peer-assessment score (PAS) and teacher assessment score (TAS) on the final scores of the psychomotor skills of dental students in a preclinical fixed prosthodontics course. MATERIALS AND METHODS: A prospective study included D2 students in a preclinical fixed prosthodontics course for two consecutive academic years from September 2017 to May 2019. Students' performance of various procedures over 14 sessions was assessed by SAS, PAS and TAS. Repeated-measures ANOVA was used to assess differences between the three scores and control charts showed their change over time. Multivariable linear regression was used to assess the association between two outcome variables: TAS and the total preclinical score and the explanatory variables: SAS, PAS, TAS, gender and GPA. The significance level was set at 5%. RESULTS: SAS had a significantly higher mean (80.21) than TAS (79.32) and PAS (78.58). SAS reached higher levels earlier than TAS and PAS. PAS had a stronger association with TAS than SAS (partial eta squared, η2 = 0.17 and 0.13), whereas SAS significantly predicted total preclinical score (p < .0001). CONCLUSIONS: PAS helped develop the skills of dental students during training, while SAS had an impact on their total preclinical score. Different student-centred activities are needed to support students' preclinical training in fixed prosthodontics courses.

Does professionalism change with different sociodemographic variables? A survey of Arab medical residents.

BACKGROUND: Medical professionalism reflects the commitment of physicians to their patients, society, themselves, and the profession. The study examined residents' attitudes towards professionalism and how these attitudes vary among the different demographic groups, namely gender, specialty, and year of residency. METHODS: A proportionate random sampling strategy was used to select the study sample. Medical residents from six specialties at a large tertiary care teaching facility were invited to participate in an online survey. The survey used the modified Learners Attitude of Medical Professionalism Scale (LAMPS), which consists of five domains: respect, excellence, altruism, duty/accountability, and integrity. Chi-square, Student t-test, one-way ANOVA, factorial ANOVA, and post hoc analysis were used to examine the attitudinal differences towards professionalism among the different demographic factors. RESULTS: The overall response rate was 82.7%. Overall, the residents' self-reported attitudes towards professionalism was positive. The highest score was for the "respect" domain (4.61), and the lowest was for "altruism" (3.67). No significant association was found between the mean scores and the three studied variables, namely, gender, specialty (surgical/nonsurgical), and level (senior/junior). CONCLUSIONS: No significant differences were observed in the overall attitude towards professionalism among the residents regarding their year of residency, gender, and specialty. The low altruism score and absence of improvement of the total score regarding the residents' increasing experience in the profession are concerns that need remedial action. Therefore, we suggest that future research look for possible explanations by using multi-institutional surveys that explore not only the residents' attitudes, but also the trainers' attitudes and practice, work situations, the hidden curriculum, and culture. Key messagesAttitudes towards professionalism among different demographic groups of residents do not show similar variations as has been reported in the literature, albeit in different sociocultural contexts.The low altruism score and absence of improvement of the total score as the residents gained more experience in the profession are concerns that need remedial action.A longitudinal study involving more than one institution for both residents and their faculty members to compare faculty scores with those of residents, while controlling for specialty and gender, may help elucidate the factors affecting attitudes towards professionalism and suggest possible means of addressing unfavourable attitudes.

Development and validation of teacher and student questionnaires measuring inhibitors of curriculum viability.

BACKGROUND: Curriculum viability is determined by the degree to which quality standards have or have not been met, and by the inhibitors that affect attainment of those standards. The literature reports many ways to evaluate whether a curriculum reaches its quality standards, but less attention is paid to the identification of viability inhibitors in different areas of the curriculum that hamper the attainment of quality. The purpose of this study is to develop and establish the reliability and validity of questionnaires that measure the presence of inhibitors in an undergraduate medical curriculum. METHODS: Teacher and student questionnaires developed by the authors were sent to medical educationalists for qualitative expert validation and to establish their content validity. To establish the response process validity, cognitive interviews were held with teachers and students to clarify any confusion about the meaning of items in the questionnaires. Reliability and construct validity of the questionnaires were established by responses from 575 teachers and 247 final-year medical students. RESULTS: Qualitative expert validation was provided by 21 experts. The initial teacher and student questionnaires containing respectively 62 items to measure 12 theoretical constructs, and 28 items to measure 7 constructs, were modified to improve their clarity and relevance. The overall scale validity index for the questionnaires was, in order, .95 and .94. Following the cognitive interviews, the resultant teacher and student questionnaires were reduced to respectively 52 and 23 items. Furthermore, after the confirmatory analysis, the final version of the teacher questionnaire was reduced to 25 items to measure 6 constructs and the student questionnaire was reduced to 14 items to measure 3 constructs. Good-for-fit indices were established for the final model and Cronbach alphas of, in order, .89 and .81 were found for the teacher and student questionnaire. CONCLUSION: The valid and reliable curriculum viability inhibitor questionnaires for teachers and students developed in this study can be used by medical schools to identify inhibitors to achieve standards in different areas of the curriculum.

A narrative study on work place based conflicts in Obstetrics and Gynaecology Department.

OBJECTIVE: To explore the patterns in research and underlying factors of conflicts in obstetrics and gynaecology and its effect on restricting the quality of education and training of residents. METHODS: The narrative study was conducted at the Obstetrics and Gynaecology Department, Lady Aitchison Hospital, King Edward Medical University, Lahore, Pakistan, from October 2018 to January 2019, and comprised narrative essays by residents associated with the department. Data was subjected to thematic analysis. RESULTS: Of the 27 residents, 26(96.3%) were females and 1(3.7%) was male. Overall, 19(70.3%) were aged 25-30 years and 8(29.6%) were aged 30-35 years. Three levels of conflict were identified: organisational, interpersonal and individual. Causes of organisational conflict included inadequate facilities, poor security and unclear duty appointments. Interpersonal factors included lack of communication, lack of patient autonomy, non-cooperative co-workers, illiteracy of attendants and unprofessional behaviour. Individual factors were overburdening duty hours and duty negligence. CONCLUSIONS: There was found to be a need to design education programmes, like workshops, that may enable post-graduate residents in obstetrics and gynaecology to handle conflicts at workplace.

Entrustable Professional Activities for Small-Group Facilitation: A Validation Study Using Modified Delphi Technique.

:: Entrustable professional activities (EPAs) provide a novel approach to support teachers' structured professionalization and to assess improvement in teaching competence thereafter. Despite their novelty, it is important to assess EPAs as a construct to ensure that they accurately reflect the work of the targeted profession. BACKGROUND:: The co-creation of an EPA framework for training and entrustment of small-group facilitators has been discussed in the literature. Although a rigorous design process was used to develop the framework, its content validity has not been established yet.Approach: A modified Delphi technique was used. Three survey rounds were conducted from December 2019 to April 2020. Expert health professions educationalists were recruited using purposive sampling and snowball techniques. In Round 1, a rubric consisting of seven items was used to assess the quality of nine pre-designed EPAs. In Round 2, competencies required to perform the agreed-upon EPAs were selected from 12 competencies provided. In Round 3, consensus was sought on sub-activities recommended for agreed-upon EPAs. Quantitative data were analyzed using multiple statistical analyses, including item-wise and rubric-wise content validity indices, asymmetric confidence interval, mean, standard deviation, and response frequencies. Qualitative data were thematically analyzed using content analysis. FINDINGS:: Three of the nine proposed EPAs achieved statistical consensus for retention. These EPAs were: (1) preparing an activity, (2) facilitating a small-group session, and (3) reflecting upon self and the session. Nine of the 12 pre-determined competencies achieved consensus and were then mapped against each agreed-upon EPA based on their relevance. Finally, results indicated consensus on five, six, and four sub-activities for EPA 1, EPA 2, and EPA 3, respectively. CONCLUSIONS:: The final framework delineates three EPAs for small-group facilitation and their associated sub-activities. The full description of each EPA provided in this article includes the title, context, task specification, required competencies, and entrustment resources. Program developers, administrative bodies, and teaching staff may find this EPA framework useful to structure faculty development, to entrust teachers, and to support personal development.

Development of an entrustable professional activities (EPAs) framework for small group facilitators through a participatory design approach.

Background: Recent reports suggest that faculty development (FD) programs need a structured framework to design training and assess improvement in teaching performance of participants. Entrustable professional activities (EPAs) can serve as a novel framework to plan and conduct structured FD programs, and to assess the proficiency of small group facilitators after training. Objective: The researchers aimed to develop an EPAs framework for small group facilitators. Design: In March 2019, three workshops were organized to develop the EPAs framework by using a participatory action design approach. An orientation workshop was conducted to train the participating students and teachers. Then, a design workshop was conducted to develop the EPA framework, where data were collected from three sources: scribe notes, audio recordings, and field charts. Thematic analysis was performed, and consensus was sought from participants on the extracted professional tasks and competencies in the consensus workshop. In the third workshop, the participants also mapped professional tasks with relevant competencies. Results: A total of 15 teachers and 15 studentsf participated in the co-design process. Through a robust thematic analysis of multisource data, 57 professional tasks and 52 competencies emerged, which were converged into 11 tasks and 17 competencies after removing duplicating and non-qualifying professional tasks and competencies. Finally, a consensus was achieved on nine tasks and 12 competencies. Conclusions: The proposed EPAs framework can serve as a road map for longitudinal training and entrustment of small group facilitators. It can also guide small group facilitators in their continuous professional development and in building their teaching portfolios.

High enthusiasm about long lasting mentoring relationships and older mentors.

BACKGROUND: Mentoring plays a pivotal role in workplace-based learning, especially in the medical realm. Organising a formal mentoring programme can be labor and time intensive and generally impractical in resource constrained medical schools with limited numbers of mentors. Hence, informal mentoring offers a valuable alternative, but will be more likely to be effective when mentors and protégés share similar views. It is therefore important to gain more insight into factors influencing perceptions of informal mentoring. This study aims to explore mentors and protégés' perceptions of informal mentoring and how these vary (or not) with gender, age and the duration of the relationship. METHOD: We administered an Informal Mentor Role Instrument (IMRI) to medical practitioners and academics from Egypt, Pakistan and Saudi Arabia. The questionnaire was developed for the study from other validated instruments. It contained 39 items grouped into 7 domains: acceptance, counselling, friendship, parenting, psychological support, role modelling and sociability. RESULTS: A total of 103 mentors and 91 protégés completed the IMRI. Mentors had a better appreciation for the interpersonal aspects of informal mentoring than protégés, especially regarding acceptance, counselling and friendship. Moreover, being older and engaged in a longer mentoring relationship contributed to more positive perceptions of interpersonal aspects of mentoring, regardless of one's role (mentor or protégé). CONCLUSION: It can be concluded that the expectations of mentors and protégés differed regarding the content and aim of the interpersonal characteristics of their mentoring relationship. We recommend mentors and protégés to more explicitly exchange their expectations of the informal mentoring relationship, as typically practiced in formal mentoring. Additionally, in our study, seniority and lasting relationships seem crucial for good informal mentoring. It appears beneficial to foster lasting informal mentoring relationships and to give more guidance to younger mentors.

Collaborative use of virtual patients after a lecture enhances learning with minimal investment of cognitive load.

BACKGROUND: The use of virtual patients (VPs), due to their high complexity and/or inappropriate sequencing with other instructional methods, might cause a high cognitive load, which hampers learning. AIM: To investigate the efficiency of instructional methods that involved three different applications of VPs combined with lectures. METHOD: From two consecutive batches, 171 out of 183 students have participated in lecture and VPs sessions. One group received a lecture session followed by a collaborative VPs learning activity (collaborative deductive). The other two groups received a lecture session and an independent VP learning activity, which either followed the lecture session (independent deductive) or preceded it (independent inductive). All groups were administrated written knowledge acquisition and retention tests as well as transfer tests using two new VPs. All participants completed a cognitive load questionnaire, which measured intrinsic, extraneous and germane load. Mixed effect analysis of cognitive load and efficiency using the R statistical program was performed. RESULTS: The highest intrinsic and extraneous load was found in the independent inductive group, while the lowest intrinsic and extraneous load was seen in the collaborative deductive group. Furthermore, comparisons showed a significantly higher efficiency, that is, higher performance in combination with lower cognitive load, for the collaborative deductive group than for the other two groups. CONCLUSION: Collaborative use of VPs after a lecture is the most efficient instructional method, of those tested, as it leads to better learning and transfer combined with lower cognitive load, when compared with independent use of VPs, either before or after the lecture.

Using entrustable professional activities (EPAs) to assess teaching competence and transfer of training: A personal view.

Faculty development (FD) activities aim to improve teaching competencies. Success of these activities is conditioned with the ability of participants to transfer the learned competencies into their teaching practices. Unfortunately, evaluation of the effectiveness of FD rely mostly upon self-reported or verbal feedback, without valid evaluation of their progress in teaching performance. This shortcoming may be attributed to the unavailability of a systematic assessment system to evaluate participants' performance in the workplace. In this article, we advocate how the concept and principles of entrustable professional activities (EPAs) can be used in assessment of teaching competencies and ensuring the transfer of training to the workplace.

Patient safety awareness among Undergraduate Medical Students in Pakistani Medical School.

OBJECTIVE: To measure the level of awareness of patient safety among undergraduate medical students in Pakistani Medical School and to find the difference with respect to gender and prior experience with medical error. METHODS: This cross-sectional study was conducted at the University of Lahore (UOL), Pakistan from January to March 2017, and comprised final year medical students. Data was collected using a questionnaire 'APSQ- III' on 7 point Likert scale. Eight questions were reverse coded. Survey was anonymous. SPSS package 20 was used for statistical analysis. RESULTS: Questionnaire was filled by 122 students, with 81% response rate. The best score 6.17 was given for the 'team functioning', followed by 6.04 for 'long working hours as a cause of medical error'. The domains regarding involvement of patient, confidence to report medical errors and role of training and learning on patient safety scored high in the agreed range of >5. Reverse coded questions about 'professional incompetence as an error cause' and 'disclosure of errors' showed negative perception. No significant differences of perceptions were found with respect to gender and prior experience with medical error (p= >0.05). CONCLUSION: Undergraduate medical students at UOL had a positive attitude towards patient safety. However, there were misconceptions about causes of medical errors and error disclosure among students and patient safety education needs to be incorporated in medical curriculum of Pakistan.

The effectiveness of sequencing virtual patients with lectures in a deductive or inductive learning approach.

BACKGROUND: Virtual patients (VPs) can be sequenced with other instructional methods in different ways. AIM: To investigate the effect of sequencing VPs with lectures in a deductive approach, in comparison with an inductive approach, on students' knowledge acquisition, retention, and transfer. METHODS: For two different topics, 84 out of 87 students have participated in the lecture and VP sessions. Students from female and male campuses have been randomly assigned to one of the two learning approaches (deductive and inductive), yielding four experimental groups. Each group received a lecture session and an independent VP learning activity, which either followed the lecture session in the deductive group or preceded it in the inductive group. Students were administrated knowledge acquisition and retention written tests as well as transfer tests using two new VPs. RESULTS: There was no significant effect for the learning approach on knowledge acquisition or retention, while for knowledge transfer, males have benefited from the inductive approach in topic 1 while in the more complex topic 2, they have benefited from the deductive approach. On the other hand, females seem to be largely unaffected by learning approach. CONCLUSIONS: Sequencing VPs in inductive and deductive learning approaches leads to no significant differences on students' performance when full guidance is offered in the inductive approach.

Becoming a Doctor in Different Cultures: Toward a Cross-Cultural Approach to Supporting Professional Identity Formation in Medicine.

Becoming a doctor is fundamentally about developing a new, professional identity as a physician, which in and of itself may evoke many emotions. Additionally, medical trainees are increasingly moving from one cultural context to another and are challenged with navigating the resulting shifts in their professional identify. In this Article, the authors aim to address medical professional identity formation from a polyvocal, multidisciplinary, cross-cultural perspective. They delineate the cultural approaches to medical professionalism, reflect on professional identity formation in different cultures and on different theories of identity development, and advocate for a context-specific approach to professional identity formation. In doing so, the authors aim to broaden the developing professional identity formation discourse to include non-Western approaches and notions.

A fresh look at Miller's pyramid: assessment at the 'Is' and 'Do' levels.

In its silver jubilee, we celebrate the ground-breaking pyramid of George Miller by submitting a fresh look at it. We discuss two questions. (i) Does the classical pyramidal structure perfectly portray the relationships of the four levels that were described by Miller? (ii) Can the model of Miller fulfill the unmet needs of assessors to measure evolving essential constructs and accommodate the increasingly sophisticated practice of assessment of health professionals? In response to the first question, Miller's pyramid is revisited in view of two assumptions for pyramidal structures, namely: hierarchy and tapering. Then we suggest different configurations for the same classical four levels and indicate when to use each one. With regard to the second question, we provide a rationale for amending the pyramid with two further dimensions to assess personal qualities of students at the 'Is' level and their performance in teams at the 'Do' (together) level. At the end of the article, we yearn to think outside the pyramid and suggest the Assessment Orbits framework to assess students as individuals and in teams. The five Assessment Orbits alert educators to assess the emerging cognitive and non-cognitive constructs, without implying features such as hierarchy or tapering that are ingrained in pyramidal structures. The 'Is' orbit attends to the personal qualities of graduates 'who' we may (or may not) trust to be our physicians. Assessment of teams at the 'Do' level (together) offers a paradigm shift in assessment from competitive ranking (storming) among students toward norming and performing as teams.

Faculty development for learning and teaching of medical professionalism.

INTRODUCTION: Professionalism must be explicitly taught, but teaching professionalism is challenging, because medical teachers are not prepared to teach this content area. AIM: This study aims at designing and evaluating a faculty development programme on learning and teaching professionalism in the Arabian context. Programme development: The study used a participatory design, where four authors and 28 teachers shared the responsibility in programme design in three steps: orientation workshop for teachers, vignette development, and teaching professionalism to students. The workshop provided the cognitive base on the salient attributes of professionalism in the Arabian context. After the workshop, authors helped teachers to develop a total of 32 vignettes in various clinical aspects, portraying a blend of professionalism dilemmas. A battery of seven questions/triggers was suggested to guide students' reflection. PROGRAMME EVALUATION: The programme was evaluated with regard to its "construct" and its "outcomes". The programme has fulfilled the guiding principles for its design and it has emerged from a genuine professionalism framework from local scholarly studies in the Arabian context. Programme outcomes were evaluated at the four levels of Kirkpatrick's model; reaction, learning, behaviour, and results. DISCUSSION: The study communicates a number of context-specific issues that should be considered when teaching professionalism in Arabian culture with respect to teachers and students. Three lessons were learned from developing vignettes, as reported by the authors. This study advocates the significance of transforming faculty development from the training discourse of stand-alone interventions to mentorship paradigm of the communities of learning. CONCLUSION: A three-step approach (orientation workshop, vignettes development, and teaching professionalism) proved effective for faculty development for learning and teaching of professionalism. Professionalism can be taught using vignettes that demonstrate professionalism dilemmas in a particular context.

Twelve Tips for teaching medical professionalism at all levels of medical education.

Review of studies published in medical education journals over the last decade reveals that teaching medical professionalism is essential, yet challenging. According to a recent Best Evidence in Medical Education (BEME) guide, there is no consensus on a theoretical or practical model to integrate the teaching of professionalism into medical education. The aim of this article is to outline a practical manual for teaching professionalism at all levels of medical education. Drawing from research literature and author's experience, Twelve Tips are listed and organised in four clusters with relevance to (1) the context, (2) the teachers, (3) the curriculum, and (4) the networking. With a better understanding of the guiding educational principles for teaching medical professionalism, medical educators will be able to teach one of the most challenging constructs in medical education.