Measuring sociocultural engagement of health professions education students: a psychometric study.

BACKGROUND: Sociocultural engagement of students refers to broadening viewpoints and providing awareness of, and respect for, diverse backgrounds and perspectives. However, there are no existing validated instruments in the literature for measuring sociocultural engagement of health professions education (HPE) students. Therefore, the aim of this study is to develop and validate a questionnaire designed to assess sociocultural engagement among HPE students. METHODS: The study included undergraduate HPE students (n = 683) at Gulf Medical University. The initial version of the sociocultural engagement of students' questionnaire (SESQ) was developed after extensive literature review and guided by the Global Learning Qualifications Framework. We then tested the content validity of the questionnaire by using focus group discussion with subject experts (n = 16) and pilot testing with students (n = 20). We distributed the content-validated version of the SESQ (16 items) to undergraduate students in six HPE colleges. To examine the construct validity and construct reliability of the questionnaire, we conducted exploratory factor analysis, followed by confirmatory factor analysis. RESULTS: Confirmatory factor analysis supported the two-factor structure which consists of 13 items with good fitness indices (χ2 = 214.35, df = 61, χ 2/df = 3.51, CFI = 0.98, RMSEA = 0.06, SRMR = 0.025, and AIC = 208.00). The two factors were sociocultural interactions (8 items) and sociocultural adaptation (5 items). The construct reliability of the total questionnaire is 0.97 and the two factors were 0.93 and 0.92 for sociocultural interactions and sociocultural adaptation, respectively. In addition, there were significant weak correlations between both factors of sociocultural engagement scores and student satisfaction with the university experience (r = .19 for each, P = .01). CONCLUSIONS: The sociocultural engagement of students' questionnaire exhibits good evidence of construct validity and reliability. Further studies will be required to test the validity of this questionnaire in other contexts.

Influence of tutor interventions and group process on medical students' engagement in problem-based learning.

INTRODUCTION: Student engagement is influenced by several variables, among which are the teaching styles employed by faculty. In problem-based learning (PBL), the role of faculty is to facilitate the learning of the tutorial group as they work through clinical problems. However, the influence of tutor intervention styles and group process on engagement of students in PBL tutorials is unclear. METHODS: The study was conducted with year 2 and 3 medical students (n = 176) in PBL tutorial groups at the end of an integrated PBL course. Students evaluated their behavioural, cognitive and emotional engagement in PBL tutorials using a pre-validated 11-item questionnaire. Students also filled in a modified version of a previously published tutor intervention profile (TIP) questionnaire. The modified TIP questionnaire represents three constructs (1) steering the learning process (6 items), (2) stimulating student autonomy (4 items) and (3) establishing relatedness with students (3 items). In addition, PBL group process was evaluated using a 5-item nominal scale: (1) tutorial atmosphere, (2) listening and information sharing, (3) group performance, (4) decision making and (5) reaction to leadership. RESULTS: Establishing sense of relatedness in the group by PBL tutors was the most important predictor of emotional engagement (F = 41.213, ΔR2 = 0.191, ß = 0.438, P = 0.000). On the other hand, steering the learning process was a significant predictor of behavioural engagement (F = 19.0, ΔR2=0.098, ß = 0.314, P = 0.000). However, stimulating student autonomy was not a significant predictor of student engagement in PBL tutorials. On the other hand, enhancing the group process in PBL tutorials significantly predicts student engagement with strong impact on emotional and cognitive engagement of students. CONCLUSIONS: Establishing the sense of relatedness in the group and steering the learning process by PBL tutors as well as improving PBL group process are significant predictors of student engagement in PBL tutorials with emotional and cognitive engagement being the most sensitive variables affected.

Construct validity of a questionnaire for measuring student engagement in problem-based learning tutorials.

BACKGROUND: Student engagement is student investment of time and energy in academic and non-academic experiences that include learning, teaching, research, governance, and community activities. Although previous studies provided some evidence of measuring student engagement in PBL tutorials, there are no existing quantitative studies in which cognitive, behavioral, and emotional engagement of students in PBL tutorials is measured. Therefore, this study aims to develop and examine the construct validity of a questionnaire for measuring cognitive, behavioral, and emotional engagement of students in PBL tutorials. METHODS: A 15-item questionnaire was developed guided by a previously published conceptual framework of student engagement. Focus group discussion (n = 12) with medical education experts was then conducted and the questionnaire was piloted with medical students. The questionnaire was then distributed to year 2 and 3 medical students (n = 176) in problem-based tutorial groups at the end of an integrated course, where PBL is the main strategy of learning. The validity of the internal structure of the questionnaire was tested by confirmatory factor analysis using structural equation modeling assuming five different models. Predictive validity evidence of the questionnaire was studied by examining the correlations between students' engagement and academic achievement. RESULTS: Confirmatory factor analysis indicates a good fit between the measurement and structural model of an 11-item questionnaire composed of a three-factor structure: behavioral engagement (3 items), emotional engagement (4 items), and cognitive engagement (4 items). Models in which the three latent factors were considered semi-independent provided the best fit. The construct reliabilities of behavioral, cognitive, and emotional factors were 0.82, 0.82, and 0.76, respectively. We failed however to find significant relationships between academic achievement and engagement. CONCLUSIONS: We found a strong evidence to support the construct validity of a three-factor structure of student engagement in PBL tutorial questionnaire. Further studies are required to test the validity of this instrument in other educational settings. The predictive validity is another area needing further scrutiny.

Measurement of student engagement in health professions education: a review of literature.

Student engagement is a complex multidimensional construct that has attained great interest in health professions education (HPE). Definition and conceptualization of student engagement is an important step that should drive the development of the instruments for its measurement. We have recently proposed a comprehensive framework for student engagement in HPE with a definition of engagement as student investment of time and energy in academic and non-academic experiences that include learning, teaching, research, governance, and community activities. The dimensions of student engagement in this framework included the cognitive, affective, behavioral, agentic, and socio-cultural. Guided by the student engagement framework, this non-systematic review aims to identify, critically appraise, and summarize the existing methods for measuring student engagement in HPE. Extrapolating from higher education literature, we attempted to link the theoretical perspectives of student engagement with the published methods of its measurement in HPE context. In addition, we have described the different methods of measuring student engagement including self-report surveys, real time measures, direct observation, interviews/focus groups, and the use of multiple instruments. The span of engagement dimensions measured by self-report surveys ranges from one to five dimensions. However, measurement of agentic and sociocultural dimensions of engagement in HPE is still limited and further research is required. We have also reflected on the existing methods of measuring engagement of students as active partners in HPE. The review also describes the advantages, limitations, and psychometric properties of each method for measuring student engagement. We ended the review with a guiding conclusion on how to develop and select an instrument for measuring student engagement in HPE. Finally, we addressed the gaps in the literature about measuring engagement of HPE students and future research plans.

Student engagement in health professions education: AMEE Guide No. 152.

This guide aims to support our colleagues to have comprehensive understanding of student engagement in health professions education. Despite the universal agreement about the significance of student engagement, there is lack of uniformity in conceptualizing and operationalizing this emerging construct. We review the theoretical basis explaining student engagement from three main perspectives: behavioral, psychological, and socio-cultural. In addition, we propose a contemporary and comprehensive framework for the student engagement in higher education, which is applicable to health professions education contexts. Drawing from this framework, we explain the conceptualization of the construct and its preceding factors, mediators, dimensions, spheres, and outcomes of student engagement. The proposed framework introduces student 'engagement through partnerships' as a novel component compared with the existing models of student engagement in higher education. This way, we are proposing a mixed model that not only considers the student as a 'customer' but also as a 'partner' in education. Engagement of students through partnerships include four areas: (1) provision of the education program, (2) scholarly research, (3) governance and quality assurance, and (4) community activities. This guide will provide practical applications on how to improve student engagement in health professions education. Finally, we highlight the current gaps in areas of research in the student engagement literature and suggested plans for future directions.[Box: see text].

The effect of access to electronic resources during examination on medical and dental students scores in summative assessment: quasi experimental study.

BACKGROUND: Access to electronic (E) resources has become an indispensable requirement in medical education and practice. OBJECTIVE: Our objective was to assess the effect of E-resources access during examination on end-course-exam scores of medical and dental students. METHODS: A quasi-experimental study which included two cohorts of medical (n = 106 & 85) and three cohorts of dental students (n = 66, 64 and 69) who took end-course- exams. Each exam was composed of two- parts (Part I and II), that encompassed equal number of questions and duration. Access to E-resources was allowed in part-II only. Items Difficulty Index (DI), Discrimination Index, (DisI), Point Biserial, (PBS) and cognitive level were determined. RESULTS: The study included 390 students. The proportion of items at various levels of DI, DisI, and PBS and the average values for item DI, DisI in both parts of each exam were comparable. The average scores in part-II were significantly higher than part-I (P < 0.001, < 0.001 and 0.04) and lower-order cognitive-level items scores were higher in three exams (P < 0.0001, 0.0001, 0.0001). Higher- order cognitive level items scores were comparable between part I and II in all courses. The significant factor for change in marks were questions cognitive level and type of the course. CONCLUSION: Access to E-resources during examination does not make a significant difference in scores of higher-order cognitive level items. Question cognitive level and course type were the significant factors for the change in exam scores when accessing E-resources. Time-restricted E-resources accessed tests that examine higher cognitive level item had no significant academic integrity drawback.

Implications of introducing case based radiological images in anatomy on teaching, learning and assessment of medical students: a mixed-methods study.

BACKGROUND: Introducing radiological anatomy in the preclinical curriculum can increase the understanding of Anatomy. Regardless of the integration when teaching anatomy, it is essential to maintain oversight as to what and how much is being taught. In addition, the knowledge requirements for preclinical students should be considered. The purpose of this kind of integration is that the student should be able to apply the knowledge which can help them better understand anatomy and not to make the course more challenging. This study aimed to understand whether adding radiological images would increase the difficulty level of the questions. METHODS: We introduced radiological images, including X Rays, CT scans and MRIs, when teaching anatomy in the preclinical curriculum. A class of 99 students were tested using A-type MCQs (n = 84). All 84 questions were categorized on whether they were case-based with or without a radiological image. The item analysis of both groups of test questions was then compared based on their difficulty and discrimination index. A qualitative student perception regarding the inclusion of radiological images in anatomy was also measured using a questionnaire with a 5-point Likert scale. RESULTS: The results showed that the performance level of the students was similar when comparing the test questions in both groups. The item analysis of the MCQs in the two groups revealed that by integrating radiological images when teaching anatomy, the various parameters in both groups of test questions were in the same range. More than 80% of the students felt that radiological images facilitate the achievement of learning outcomes and help to apply their knowledge in clinical contexts. The study's findings reported that the rate of satisfaction by including radiological images when teaching anatomy is high. CONCLUSION: Recognition and interpretation of images are essential in an undergraduate medical program. Students found it helpful when radiological images were introduced to them when teaching anatomy. Since the students' performance in summative exams in both groups of questions was in the same range, the findings also point out that adding radiological images when teaching anatomy does not increase the difficulty of the subject.

A guide to best practice in faculty development for health professions schools: a qualitative analysis.

BACKGROUND: This is a practice guide for the evaluation tool specifically created to objectively evaluate longitudinal faculty development programs (FDP) using the "5×2 -D backward planning faculty development model". It was necessary to create this tool as existing evaluation methods are designed to evaluate linear faculty development models with a specific endpoint. This backward planning approach is a cyclical model without an endpoint, consisting of 5 dynamic steps that are flexible and interchangeable, therefore can be a base for an evaluation tool that is objective and takes into account all the domains of the FDP in contrast to the existing, traditional, linear evaluation tools which focus on individual aspects of the program. The developed tool will target evaluation of longitudinal faculty development programs regardless of how they were planned. METHODOLOGY: Deductive qualitative grounded theory approach was used. Evaluation questions were generated and tailored based on the 5 × 2-D model followed by 2 Delphi rounds to finalize them. Based on the finalized evaluation questions from the results of the Delphi rounds, two online focus group discussions (FGDs) were conducted to deduce the indicators, data sources and data collection method. RESULTS: Based on the suggested additions, the authors added 1 new question to domains B, with a total of 42 modifications, such as wording changes or discarding or merging questions. Some domains received no comments, therefore, were not included in round 2. For each evaluation question, authors generated indicators, data sources and data collection methods during the FGD. CONCLUSION: The methodology used to develop this tool takes into account expert opinions. Comprehensiveness of this tool makes it an ideal evaluation tool during self-evaluation or external quality assurance for longitudinal FDP. After its validation and testing, this practice guide can be used worldwide, along with the provided indicators which can be quantified and used to suit the local context.

Student engagement in undergraduate medical education: A scoping review.

INTRODUCTION: Student engagement is defined as behavioural, cognitive and emotional aspects of students' academic experience in teaching, learning and research through interacting with other students, faculty and community. Despite the growing interest in the field of student engagement, medical education research in this area is still fragmented. This scoping review aims to contribute to the understanding of measurements, drivers and outcomes of medical student engagement. METHODS: The authors searched MEDLINE, PubMed, ProQuest, SCOPUS, ERIC, Science Direct and EBESCO for English articles published from 1990 until October 2021. In addition, we hand-searched key medical education journals and references in recently published articles. Using specific selection criteria, two authors independently reviewed the articles for eligibility, followed by data extraction using both quantitative and qualitative analysis. RESULTS: Of the 2136 retrieved articles, 51 studies were selected for the review, and 94% of them were published in the past 8 years. The prevailing measures of student engagement failed to cover the multidimensionality of the construct with more focus on the behavioural dimension. Quantitative methods represented two thirds of the studies with a higher frequency of cross-sectional designs and using self-reports. The identified drivers of medical students' engagement are directed mainly to modifying the context of the learning environment. These factors increased student engagement by fostering relevance of learning, positive student relationships with peers and faculty, and enhancing student agency, and sense of competence. Cognitive engagement appears to be a positive predictor of academic achievement, but the relationship with other dimensions of engagement is controversial. CONCLUSIONS: Student engagement in undergraduate medical education is an important, yet under-researched construct. The research that does exist suggests engagement to be malleable, something that can be modified by different types of interventions taking into consideration the context of education and practice. Further research is required, however, to address the gaps identified in this review.

Polarity Based Model for Guiding Medical School Strategy During Crisis - A Cross Sectional Qualitative Study.

BACKGROUND: Crises in academia can best be dealt with as a polarity that needs to be leveraged rather than a problem that needs to be solved. This work aimed at utilizing the Polarity Approach for Continuity and Transformation (PACT)™ to establish a guide for medical schools during times of crisis to minimize the effect of crisis-driven decisions on strategic growth. SUBJECTS AND METHODS: A qualitative study following the 5-Steps of the PACT process was conducted. A virtual mapping session was held with 108 medical educators from 22 countries to determine the upsides and downsides of strategic orientation and crisis management subsequently. RESULTS: Four polarity maps were generated identifying four tension areas; University reputation, mission, teams, and individuals followed by a 72-item assessment and another mapping session to map the warning signs and action steps. A comparison between private school scores and the whole cohort of respondents showed that private schools had the least problems in team-oriented work. CONCLUSION: This study highlighted the importance of taking measures to communicate the mission and supporting team functions inside universities either by enhancing resources or utilizing time and effort-saving strategies.

Psychometric Properties of Visual Indicators of Teaching and Learning Success "VITALS" Instrument for Evaluation of Clinical Teachers.

PURPOSE: We have previously developed an instrument for students' evaluation of clinical teachers that we called Visual Indicators of Clinical Teaching and Learning Success (VITALS). This study measures the reliability of VITALS as an instrument for student evaluation of clinical tutors. Additionally, the study explores the minimum number of student raters necessary for an acceptable reliability, and provides evidence of construct validity of the evaluation scores. MATERIALS AND METHODS: This retrospective study included 1825 evaluation forms completed by medical students evaluating clinical tutors using the VITALS instrument. Reliability was measured by applying generalizability theory (G-theory) analysis using a two-facet design (raters and items). A D-study was used to determine the minimum number of raters required to achieve a reliability ≥0.80. Face validity was tested by measuring tutors' degree of agreement with the items of the study instrument. RESULTS: The overall G-coefficient was 0.89. The subject of measurement (clinical tutors' scores) represented 15.8% of the variance across all tutors and items. The variance due to the interaction between raters (students) and tutors contributed to 43.5%, while the variance due to items was negligible. The remaining 40% of the variance was due to unexplained sources of error. The D-study demonstrated that a minimum of 12 raters (students) are required to achieve a reliability of 0.80. Finally, most of the clinical tutors agreed that all items in the instrument were appropriate. CONCLUSION: We demonstrate that VITALS exhibits good psychometric properties. There should be at least 12 students rating each clinical tutor to have an acceptable level of reliability for the study instrument. Face validity of the study instrument was evidenced by its high level of approval among clinical tutors.

Virtual Clinical Encounter Examination (VICEE): A novel approach for assessing medical students' non-psychomotor clinical competency.

INTRODUCTION: The Corona Virus Disease-19 (COVID-19) pandemic disrupted medical education across the world. Online teaching has grown rapidly under lockdown. Yet the online approach for assessment presents a number of challenges, particularly when evaluating clinical competencies. The aim of this study was to investigate the feasibility, acceptability, reliability and validity of an online Virtual Clinical Encounter Examination (VICEE) to assess non-psychomotor competencies (non-procedure or manual skills) of medical students. METHOD: Sixty-one final year medical students took the VICEE as part of the final summative examination. A panel of faculty experts developed the exam cases and competencies. They administered the test online via real-time interaction with artificial intelligence (AI) based virtual patients, along with faculty and IT support. RESULTS: Student and faculty surveys demonstrated satisfaction with the experience. Confirmatory factor analysis supported convergent validity of VICEE with Direct Observation Clinical Encounter Examination (DOCEE), a previously validated clinical examination. The observed sensitivity was 81.8%, specificity 64.1% and likelihood ratio 12.6, supporting the ability of VICEE to diagnose 'clinical incompetence' among students. CONCLUSION: Our results suggest that online AI-based virtual patient high fidelity simulation may be used as an alternative tool to assess some aspects of non-psychometric competencies.

Medical education in the United Arab Emirates: Challenges and opportunities.

Medical education (ME) in the United Arab Emirates (UAE) has a relatively short history that begins with the inception of the UAE almost 50 years ago. The UAE has made great strides in widening access to ME through the rapid implementation of national agendas aimed at advancing healthcare and expanding higher education, in addition to the presence of a strong infrastructure for privatization and business development. While progress is being made at all levels of ME, complex challenges for both undergraduate and postgraduate ME remain. Going forward, issues of standardization, quality, sustainability of academic and healthcare workforces, and research must continue to be addressed.

Transformation to learning from a distance.

This article was migrated. The article was marked as recommended. COVID-19 is a strong disruptive force that has not only influenced our global health and economy but also has changed the way we teach, learn and communicate with our students. It has disturbed the regular education pattern and the standard practices that we adapted over many years. The challenge is beyond changing the mode of delivering instructions from face to face to online. The real challenge is in creating a culture that supports the adoption of innovative practices, which require different skills and competences from the teacher, student, mentor and administrator, and at the same time maintaining the quality of the products. In other words, changing what was exceptional to be the norm over a short period of time. This article describes our approach "Open Learning" in managing such change. Our over-riding philosophy is about ensuring that students have high quality resources, and the enthusiasm and learning skills to benefit from them. At the same time we want to optimise the use of the available online applications and learning management system so that their use is within the capability of our faculty. This paper describes the evolution of our approach and the principles upon which it has been based. Our experiences over the past few months will transform the educational experience of our students over the years to come.

Measuring medical students' professional competencies in a problem-based curriculum: a reliability study.

BACKGROUND: Identification and assessment of professional competencies for medical students is challenging. We have recently developed an instrument for assessing the essential professional competencies for medical students in Problem-Based Learning (PBL) programs by PBL tutors. This study aims to evaluate the reliability and validity of professional competency scores of medical students using this instrument in PBL tutorials. METHODS: Each group of seven to eight students in PBL tutorials (Year 2, n = 46) were assessed independently by two faculty members. Each tutor assessed students in his/her group every five weeks on four occasions. The instrument consists of ten items, which measure three main competency domains: interpersonal, cognitive and professional behavior. Each item is scored using a five-point Likert scale (1 = poor, 5 = exceptional). Reliability of professional competencies scores was calculated using G-theory with raters nested in occasions. Furthermore, criterion-related validity was measured by testing the correlations with students' scores in written examination. RESULTS: The overall generalizability coefficient (G) of the professional competency scores was 0.80. Students' professional competencies scores (universe scores) accounted for 27% of the total variance across all score comparisons. The variance due to occasions accounted for 10%, while the student-occasion interaction was zero. The variance due to raters to occasions represented 8% of the total variance, and the remaining 55% of the variance was due to unexplained sources of error. The highest reliability measured was the interpersonal domain (G = 0.84) and the lowest reliability was the professional behavior domain (G = 0.76). Results from the decision (D) study suggested that an adequate dependability (G = 0.71) can be achieved by using one rater for five occasions. Furthermore, there was a positive correlation between the written examination scores and cognitive competencies scores (r = 0.46, P < 0.01), but not with the other two competency domains (interpersonal and professionalism). CONCLUSIONS: This study demonstrates that professional competency assessment scores of medical students in PBL tutorials have an acceptable reliability. Further studies for validating the instrument are required before using it for summative evaluation of students by PBL tutors.

Medical College of the Future: from Informative to Transformative.

The complexity of medical education and healthcare systems is a 'wicked problem'. Change will be a continuous iteration between evaluation and revaluation. Medicine is a social science. It is about people, societies and human interaction and communication. Medical College of the future should be developed in the light of social constructivism theories. Students from year one, day one, should be embedded in the work environment. Academic Healthcare Systems will be the norm not the exceptional. The training of students will be in all healthcare related facilities in the community. Public-private partnership in education and research will spread and become more regulated and encouraged. The students who want to be the future physicians should be selected differently. The curriculum will be more context related. Entrustable professional activities will be measured more frequently at different points of the students learning trajectory. Research and innovation will be integral to the students' learning experience. They should be exposed to how researchers think and behave and be embedded in a research environment. The medical college of the future will be using advanced technology which will be disruptive and transform existing educational models. E-Learning materials will be shared by consortia of collaborating medical colleges from all around the world. Artificial intelligence and machine learning will influence how students learn. National and international accreditation systems should ensure quality, but not stifle innovation. Physical learning spaces should reflect and express the underlying assumptions about "what is learning and teaching?". This should be reflected and expressed through the design of its buildings, infrastructure, technology, furniture and out of class learning environment. Predicting the future is difficult in a rapidly changing world. Next generation physicians should be competent in treating the next generation of patients while maintaining the health of the population.

The Problem and Goals Are Global, the Solutions Are Local: Revisiting Quality Measurements and the Role of the Private Sector in Global Health Professions Education.

The shortage of a competent health workforce is a global challenge. However, its manifestations and proposed solutions are very much context related (i.e., local). In addition to the shortage of health professionals, the quality of health professions education programs, institutions, and graduates, and how to measure quality, are also problematic. Commonly used metrics like the Credit Hours System and the European Credit Transfer and Accumulation System have limitations (e.g., being more focused on quantity than quality).In this Invited Commentary, the author discusses the need to revisit quality measurements in health professions education and the issue of whether the private sector has a role to play in narrowing the ever-increasing gap between the demand for health care professionals and the health care workforce shortage.

Creating a Framework for Medical Professionalism: An Initial Consensus Statement From an Arab Nation.

Background Medical professionalism has received increased worldwide attention, yet there is limited information on the applicability and utility of established Western professionalism frameworks in non-Western nations. Objective We developed a locally derived consensus definition of medical professionalism for the United Arab Emirates (UAE), which reflects the cultural and social constructs of the UAE and the Middle East. Methods We used a purposive sample of 14 physicians working in the UAE as clinical and education leaders. This expert panel used qualitative methods, including the world café, nominal group technique, the Delphi method, and an interpretive thematic analysis to develop the consensus statement. Results The expert panel defined 9 attributes of medical professionalism. There was considerable overlap with accepted Western definitions, along with important differences in 3 aspects: (1) the primacy of social justice and societal rights; (2) the role of the physician's personal faith and spirituality in guiding professional practices; and (3) societal expectations for professional attributes of physicians that extend beyond the practice of medicine. Conclusions Professionalism is a social construct influenced by cultural and religious contexts. It is imperative that definitions of professionalism used in the education of physicians in training and in the assessment of practicing physicians be formulated locally and encompass specific competencies relevant to the local, social, and cultural context for medical practice. Our goal was to develop a secular consensus statement that encompasses culture and values relevant to professionalism for the UAE and the Arab region.