What works in radiology education for medical students: a systematic review and meta-analysis.

BACKGROUND: Medical imaging related knowledge and skills are widely used in clinical practice. However, radiology teaching methods and resultant knowledge among medical students and junior doctors is variable. A systematic review and meta-analysis was performed to compare the impact of different components of radiology teaching methods (active versus passive teaching, eLearning versus traditional face-to-face teaching) on radiology knowledge / skills of medical students. METHODS: PubMed and Scopus databases were searched for articles published in English over a 15-year period ending in June 2021 quantitatively comparing the effectiveness of undergraduate medical radiology education programs regarding acquisition of knowledge and/or skills. Study quality was appraised by the Medical Education Research Study Quality Instrument (MERSQI) scoring and analyses performed to assess for risk of bias. A random effects meta-analysis was performed to pool weighted effect sizes across studies and I2 statistics quantified heterogeneity. A meta-regression analysis was performed to assess for sources of heterogeneity. RESULTS: From 3,052 articles, 40 articles involving 6,242 medical students met inclusion criteria. Median MERSQI score of the included articles was 13 out of 18 possible with moderate degree of heterogeneity (I2 = 93.42%). Thematic analysis suggests trends toward synergisms between radiology and anatomy teaching, active learning producing superior knowledge gains compared with passive learning and eLearning producing equivalent learning gains to face-to-face teaching. No significant differences were detected in the effectiveness of methods of radiology education. However, when considered with the thematic analysis, eLearning is at least equivalent to traditional face-to-face teaching and could be synergistic. CONCLUSIONS: Studies of educational interventions are inherently heterogeneous and contextual, typically tailored to specific groups of students. Thus, we could not draw definitive conclusion about effectiveness of the various radiology education interventions based on the currently available data. Better standardisation in the design and implementation of radiology educational interventions and design of radiology education research are needed to understand aspects of educational design and delivery that are optimal for learning. TRIAL REGISTRATION: Prospero registration number CRD42022298607.

What impacts students' satisfaction the most from Medicine Student Experience Questionnaire in Australia: a validity study.

PURPOSE: This study evaluated the validity of student feedback derived from Medicine Student Experience Questionnaire (MedSEQ), as well as the predictors of students' satisfaction in the Medicine program. METHODS: Data from MedSEQ applying to the University of New South Wales Medicine program in 2017, 2019, and 2021 were analyzed. Confirmatory factor analysis (CFA) and Cronbach's α were used to assess the construct validity and reliability of MedSEQ respectively. Hierarchical multiple linear regressions were used to identify the factors that most impact students' overall satisfaction with the program. RESULTS: A total of 1,719 students (34.50%) responded to MedSEQ. CFA showed good fit indices (root mean square error of approximation=0.051; comparative fit index=0.939; chi-square/degrees of freedom=6.429). All factors yielded good (α>0.7) or very good (α>0.8) levels of reliability, except the "online resources" factor, which had acceptable reliability (α=0.687). A multiple linear regression model with only demographic characteristics explained 3.8% of the variance in students' overall satisfaction, whereas the model adding 8 domains from MedSEQ explained 40%, indicating that 36.2% of the variance was attributable to students' experience across the 8 domains. Three domains had the strongest impact on overall satisfaction: "being cared for," "satisfaction with teaching," and "satisfaction with assessment" (ß=0.327, 0.148, 0.148, respectively; all with P<0.001). CONCLUSION: MedSEQ has good construct validity and high reliability, reflecting students' satisfaction with the Medicine program. Key factors impacting students' satisfaction are the perception of being cared for, quality teaching irrespective of the mode of delivery and fair assessment tasks which enhance learning.

Medical student selection process enhanced by improving selection algorithms and changing the focus of interviews in Australia: a descriptive study

PURPOSE: The study investigates the efficacy of new features introduced to the selection process for medical school at the University of New South Wales, Australia: (1) considering the relative ranks rather than scores of the Undergraduate Medicine and Health Sciences Admission Test and Australian Tertiary Admission Rank; (2) structured interview focusing on interpersonal interaction and concerns should the applicants become students; and (3) embracing interviewers' diverse perspectives. METHODS: Data from 5 cohorts of students were analyzed, comparing outcomes of the second year in the medicine program of 4 cohorts of the old selection process and 1 of the new process. The main analysis comprised multiple linear regression models for predicting academic, clinical, and professional outcomes, by section tools and demographic variables. RESULTS: Selection interview marks from the new interview (512 applicants, 2 interviewers each) were analyzed for inter-rater reliability, which identified a high level of agreement (kappa=0.639). No such analysis was possible for the old interview since it required interviewers to reach a consensus. Multivariate linear regression models utilizing outcomes for 5 cohorts (N=905) revealed that the new selection process was much more effective in predicting academic and clinical achievement in the program (R2=9.4%­17.8% vs. R2=1.5%­8.4%). CONCLUSION: The results suggest that the medical student selection process can be significantly enhanced by employing a non-compensatory selection algorithm; and using a structured interview focusing on interpersonal interaction and concerns should the applicants become students; as well as embracing interviewers' diverse perspectives.

Medical teachers' experience of emergency remote teaching during the COVID-19 pandemic: a cross-institutional study.

BACKGROUND: The COVID-19 pandemic and the consequent social distancing measures caused unprecedented disruption for medical and healthcare education. This study examined medical teachers' experience with emergency remote teaching during the pandemic and their acceptance of online teaching after the pandemic. METHODS: In this sequential mixed methods study, online surveys were disseminated to teachers (n = 139) at two Asia-Pacific medical schools to evaluate their experience with emergency remote teaching during the pandemic. Subsequently, in-depth interviews were conducted with teachers from both institutions (n = 13). Each interviewee was classified into an adopter category based on Rogers' Diffusion of Innovations Theory. Interview transcripts were analyzed thematically, and the descriptive themes were mapped to broader themes partly based on the Technology Acceptance Model and these included: (i) perceived usefulness of online teaching, (ii) perceived ease of delivering online teaching, (iii) experience with institutional support and (iv) acceptance of online teaching after the pandemic. RESULTS: Our participants described accounts of successes with their emergency remote teaching and difficulties they experienced. In general, most participants found it difficult to deliver clinical skills teaching remotely and manage large groups of students in synchronous online classes. With regards to institutional support, teachers with lower technological literacy required just-in-time technical support, while teachers who were innovative in their online teaching practices found that IT support alone could not fully address their needs. It was also found that teachers' acceptance of online teaching after the pandemic was influenced by their belief about the usefulness of online teaching. CONCLUSIONS: This study demonstrated that our participants managed to adapt to emergency remote teaching during this pandemic, and it also identified a myriad of drivers and blockers to online teaching adoption for medical teachers. It highlights the need for institutes to better support their teaching staff with diverse needs in their online teaching.

Quality use of diagnostic imaging in trauma, and the impact on emergency medical practice-a retrospective clinical audit.

PURPOSE: The current study aimed to evaluate the appropriateness of the decisions made to order diagnostic imaging for emergency trauma patients. METHODS: A retrospective audit of 642 clinical decisions for 370 non-consecutive trauma cases was conducted at a level 1 adult trauma centre. Radiographs and computed tomography investigations were compared for compliance with the American College of Radiology/European Society of Radiology (ACR/ESR) imaging guidelines. The non-compliant imaging orders were evaluated for the following medical officer ranks: Junior Medical Officer (JMO), Senior Resident Medical Officer (SRMO), Emergency Medicine Registrar (EMR) and Consultants/Career Medical Officer (CMO). The time of imaging order (day/evening or night shift), whether the imaging led to conservative or surgical patient management, and whether the imaging order decisions led to a change in the clinical management of the patient were also considered. RESULTS: Non-compliance with the ACR/ESR guidelines was low. At least half of non-compliant decisions made by each level of medical officer resulted in a change in patient management. In total, 11 (65%), 25 (53%), 12 (63%) and 11 (52%) of non-compliant imaging orders placed by JMOs, SRMOs, EMRs and Consultants, respectively, resulted in change of patient management. JMOs and SRMOs ordered a significantly higher proportion of non-compliant imaging studies for conservatively managed patients in comparison to surgically managed patients. CONCLUSION: This study highlighted that most non-compliant imaging orders benefited the patient.

Adaptive tutorials versus web-based resources in radiology: a mixed methods analysis in junior doctors of efficacy and engagement.

BACKGROUND: Radiology education is limited in undergraduate Medicine programs. Junior doctors might not have the necessary background to effectively order and interpret diagnostic imaging investigations. Furthermore, junior doctors are often time-poor, balancing clinical commitments with ongoing learning, leadership and teaching responsibilities. Previous studies have demonstrated the efficacy of radiology-themed online adaptive tutorials for senior medical students. Such adaptive tutorials might therefore be an efficient and effective form of radiology education for junior doctors. METHODS: A randomised controlled crossover trial was performed to evaluate the impact of adaptive tutorials on learning the indications for, and interpretation of, basic imaging studies, compared with peer-reviewed web-based resources. Ninety-one volunteer junior doctors, comprising 53 postgraduate year 1 (PGY 1) and 38 postgraduate year 2 (PGY 2), were randomly allocated into two groups. In the first phase of the trial, focusing on head CT, one group accessed adaptive tutorials while the other received web-based resources. In the second phase of the trial, focusing on chest CT, the groups crossed over. Following each phase of the trial, participants completed exam-style online assessments. At the conclusion of the study, participants also completed an online questionnaire regarding perceived engagement and efficacy of each type of educational resource. RESULTS: Junior doctors completed the adaptive tutorials significantly faster than the relevant web-based resources for both head CT and chest CT (p = 0.03 and < 0.01 respectively). Mean quiz scores were higher in the groups receiving adaptive tutorials on head CT and chest CT (86.4% vs 83.5 and 77.7% vs 75% respectively). However, in contrast to previous studies in senior medical students, these differences were not statistically significant. Participants reported higher engagement and perceived value of adaptive tutorials, compared with web-based resources. CONCLUSIONS: Adaptive tutorials are more time-efficient than existing web-based resources for learning radiology by junior doctors, while both types of resources were equally effective for learning in this cohort. Junior doctors found the adaptive tutorials more engaging and were more likely to recommend these resources to their colleagues.

Selecting top candidates for medical school selection interviews- a non-compensatory approach.

BACKGROUND: Medical schools apply a range of selection methods to ensure that admitted students succeed in the program. In Australia, selection tools typically include measures of academic achievement (e.g. the Australian Tertiary Admission Rank - ATAR) and aptitude tests (e.g. the Undergraduate Medicine and Health Sciences Admissions Test - UMAT). These are most commonly used to determine which applicants are invited for additional selection processes, such as interviews. However, no previous study has examined the efficacy of the first part of the selection process. In particular, are compensatory or non-compensatory approaches more effective in evaluating the outcomes of cognitive and aptitude tests, and do they affect the demographics of students selected for interview? METHODS: This study utilised data from consecutive cohorts of mainstream domestic students who applied to enter the UNSW Medicine program between 2013 to 2018. A compensatory ranked selection model was compared with a non-compensatory ranked model. Initially, ATAR marks and UMAT scores for each applicant were ranked within each cohort. In the compensatory model, the mean of the ATAR and UMAT ranks were used to determine the outcome. In the non-compensatory model, the lowest rank of ATAR and UMAT determined the outcome for each applicant. The impact of each model on the gender and socioeconomic status of applicants selected to interview was evaluated across all cohorts. RESULTS: The non-compensatory ranked selection model resulted in substantially higher ATAR and UMAT thresholds for invitation to interview, with no significant effect on the socioeconomic status of the selected applicants. CONCLUSIONS: These results are important, demonstrating that it is possible to raise the academic threshold for selection to medicine without having any negative impact on applicants from low socioeconomic backgrounds. Overall, the evidence gathered in this study suggests that a non-compensatory model is preferable for selecting applicants for medical student selection interview.

The collaborative learning development exercise (CLeD-EX): an educational instrument to promote key collaborative learning behaviours in medical students.

BACKGROUND: Modern clinical practice increasingly relies on collaborative and team-based approaches to care. Regulatory bodies in medical education emphasise the need to develop collaboration and teamwork competencies and highlight the need to do so from an early stage of medical training. In undergraduate medical education, the focus is usually on collaborative learning, associated with feedback and reflection on this learning This article describes a novel educational instrument, the Collaborative Learning Development Exercise (CLeD-EX), which aims to foster the development of key collaborative learning competencies in medical students. In this article we report on the effectiveness, feasibility and educational impact of the CLeD-EX. METHODS: In this study, the "educational design research" framework was used to develop, implement and evaluate the CLeD-EX. This involved adopting a systematic approach towards designing a creative and innovative instrument which would help solve a real-world challenge in developing collaborative learning skills. The systematic approach involved a qualitative exploration of key collaborative learning behaviours which are influential in effective collaborative learning contexts. The identified competencies were employed in the design of the CLeD-EX. The design of the CLeD-EX included features to facilitate structured feedback by tutors to students, complemented by self-evaluation and reflection. The CLeD-EX was field-tested with volunteer junior medical students, using a controlled pre-test post-test design. Analysis of the completed CLeD-EX forms, self-perception surveys (i.e. pre-test and post-test surveys) and analyses of reflective reports were used to explore the educational impact of CLeD-EX, as well as its utility and practicality. RESULTS: After using the CLeD-EX, students showed a significant improvement in critical thinking and group process as measured by a previously validated instrument. Both students and tutors recognised CLeD-EX as an effective instrument, especially as a structured basis for giving and receiving feedback and for completing the feedback loop. CLeD-EX was also found to be feasible, practical and focused, while promoting learning and effective interactions in small group learning. CONCLUSION: The findings of this study support the introduction of an effective and feasible educational instrument such as the CLeD-EX, to facilitate the development of students' skills in collaborative learning.

Insights into student assessment outcomes in rural clinical campuses.

BACKGROUND: There is an ongoing debate about the impact of studying medicine in rural vs. metropolitan campuses on student assessment outcomes. The UNSW Medicine Rural Clinical School has five main campuses; Albury-Wodonga, Coffs Harbour, Griffith, Port Macquarie and Wagga Wagga. Historical data of student assessment outcomes at these campuses raised concerns regarding potential biases in assessment undertaken, as well as the availability and quality of learning resources. The current study aims to identify the extent to which the location of examination (rural versus metropolitan) has an impact on student marks in OSCEs. METHODS: Assessment data was employed for this study from 275 medical students who sat their final examinations in Years 3 and 6 of the undergraduate Medicine program at UNSW in 2018. The data consists of matched student assessment results from the Year 3 (Y3) MCQ examination and OSCE, and from the Year 6 (Y6) MCQ, OSCE and management viva examinations. The analysis used Univariate Analysis of Variance and linear regression models to identify the impact of site of learning and site of examination on assessment outcomes. RESULTS: The results demonstrate that neither site of learning nor site of examination had any significant impact on OSCE or Management Viva assessment outcomes while potential confounders are controlled. CONCLUSION: It is suggested that some of the supposed disadvantages inherent at rural campuses are effectively mitigated by perceived advantages; more intensive interaction with patients, the general and medical communities at those sites, as well as effective e-learning resources and moderation of assessment grades.

Adaptive Tutorials Versus Web-Based Resources in Radiology: A Mixed Methods Analysis of Efficacy and Engagement in Senior Medical Students.

RATIONALE AND OBJECTIVES: Radiology education is suited to delivery via e-learning which may be used to fill gaps in knowledge and help prepare medical students for internship. There is limited evidence of effectiveness of adaptive tutorials, a form of e-learning in a senior medical student cohort. MATERIALS AND METHODS: A randomized mixed methods crossover trial was performed to assess effectiveness of adaptive tutorials on engagement and understanding of appropriate use and interpretation of basic imaging studies. Eighty-one volunteer medical students from years 5 and 6 of a 6-year program were randomly allocated to one of two groups. In the first phase of the trial on head CT, one group received access to adaptive tutorials and the other to peer-reviewed web-based resources. A cross over was performed and the second phase of the trial addressing chest CT commenced. Examination style assessments were completed at the end of each phase. At the trial's conclusion, an online questionnaire was provided to evaluate student perceptions of engagement and efficacy of each educational resource. RESULTS: Adaptive tutorial groups in both phases achieved higher mean scores than controls which were statistically significant in the first phase only. Students reported higher engagement and overall perceived value of the adaptive tutorials than controls. CONCLUSION: Adaptive tutorials are overwhelmingly supported by senior medical students. Questionnaire responses suggest the engaging nature of the tutorials efficiently aids participation and knowledge retention which is in principle supported by test results.

Using Narratives to Teach Students Enrolled in Science and Medical Science Bachelor's Degree Programs.

INTRODUCTION: Narratives (as opposed to stories) can assess multiple facets of the same problem through the viewpoints of different characters. METHODS: Narratives related to three cancer patients, from diagnosis to cure or death, were used to teach seven cancer-related themes in a Cancer Pathology course offered to third-year medical science and science (college) undergraduates. RESULTS: The majority of students preferred narrative-based learning compared with traditional learning methods because they felt that it improved their learning experience and retention of information. CONCLUSION: Narrative-based learning may improve the learning experience of students by contextualizing complex concepts and highlighting real-world applications of knowledge.

Knowledge Maps: an Online Tool for Knowledge Mapping with Automated Feedback.

Concept and knowledge maps have been shown to improve students' learning by emphasising meaningful relationships between phenomena. A user-friendly online tool that enables assessment of students' maps with automated feedback might therefore have significant benefits for learning. For that purpose, we developed an online software platform known as Knowledge Maps. Two pilot studies were performed to evaluate the usability and efficacy of Knowledge Maps. Study A demonstrated significantly improved perceptions of learning after using Knowledge Maps to learn pathology. Study B showed significant improvement between pre-test and post-test scores in an anatomy course. These preliminary studies indicate that this software is readily accepted and may have potential benefits for learning.

Knowledge maps: a tool for online assessment with automated feedback.

In higher education, most assessments or examinations comprise either multiple-choice items or open-ended questions such as modified essay questions (MEQs). Online concept and knowledge maps are potential tools for assessment, which might emphasize meaningful, integrated understanding of phenomena. We developed an online knowledge-mapping assessment tool, which provides automated feedback on student-submitted maps. We conducted a pilot study to investigate the potential utility of online knowledge mapping as a tool for automated assessment by comparing the scores generated by the software with manual grading of a MEQ on the same topic for a cohort of first-year medical students. In addition, an online questionnaire was used to gather students' perceptions of the tool. Map items were highly discriminating between students of differing knowledge of the topic overall. Regression analysis showed a significant correlation between map scores and MEQ scores, and responses to the questionnaire regarding use of knowledge maps for assessment were overwhelmingly positive. These results suggest that knowledge maps provide a similar indication of students' understanding of a topic as a MEQ, with the advantage of instant, consistent computer grading and time savings for educators. Online concept and knowledge maps could be a useful addition to the assessment repertoire in higher education.

Enhancing the defensibility of examiners' marks in high stake OSCEs.

BACKGROUND: Most assessments in health professions education consist of knowledge-based examinations as well as practical and clinical examinations. Among the most challenging aspects of clinical assessments is decision making related to borderline grades assigned by examiners. Borderline grades are commonly used by examiners when they do not have sufficient information to make clear pass/fail decisions. The interpretation of these borderline grades is rarely discussed in the literature. This study reports the application of the Objective Borderline Method (version 2, henceforth: OBM2) to a high stakes Objective Structured Clinical Examination undertaken at the end of the final year of a Medicine program in Australia. METHODS: The OBM2 uses all examination data to reclassify borderline grades as either pass or fail. Factor analysis was used to estimate the suitability of data for application of OBM2. Student's t-tests, utilising bootstrapping, were used to compare the OBM2 with 'traditional' results. Interclass correlations were used to estimate the association between the grade reclassification and all other grades in this examination. RESULTS: The correlations between scores for each station and pass/fail outcomes increased significantly after the mark reclassification, yet the reclassification did not significantly impact on students' total scores. Examiners, students and program leaders expressed high levels of satisfaction and the Faculty's Curriculum Development Committee has decided that the OBM2 will be used for all future clinical examinations. Implications of the OBM2 are discussed. CONCLUSIONS: The OBM2 provides a feasible, defensible and acceptable solution for classification of borderline grades as either pass or fail.

A model for the use of blended learning in large group teaching sessions.

BACKGROUND: Although blended learning has the potential to enhance the student experience, both in terms of engagement and flexibility, it can be difficult to effectively restructure existing courses. To achieve these goals for an introductory Pathology course, offered to more than 250 undergraduate students at UNSW Sydney, we devised a novel approach. METHODS: For each topic presented over 2-3 weeks, a single face-to-face overview lecture was retained. The remaining content that had previously been delivered as conventional lectures was converted into short (12-18 min) online modules. These were based on lecture slides with added animations/highlights, plus narration using edited excerpts of previous lecture recordings. The modules also incorporated interactive questions and review quizzes with feedback which used various question types. Modules were developed in PowerPoint and iSpring and uploaded to Moodle as SCORM packages. Each topic concluded with an interactive large-group session focussing on integration of the content, with in-class questions to which students could respond via the Echo360 Active Learning Platform (ALP). Overall, more than 50% of face-to-face lecture time was replaced by online modules and interactive large-group sessions. Quantitative evaluation data included usage statistics from 264 students and feedback via online survey responses from 41 students. Qualitative evaluation data consisted of reflective commentaries from 160 student ePortfolios, which were analysed to identify factors affecting learning benefits and user acceptability. RESULTS: All of the modules were completed by 74% of students and on average, 83.1% of students eventually passed the optional review quizzes. Notably, 88.4% of students responded to in-class questions during the integration and feedback sessions via the ALP. Student reflections emphasised that the modules promoted understanding, which was reinforced through active learning. The modules were described as enjoyable, motivating and were appreciated for their flexibility, which enabled students to work at their own pace. CONCLUSIONS: In transforming this introductory Pathology course, we have demonstrated a model for the use of blended learning in large group teaching sessions, which achieved high levels of completion, satisfaction and value for learning.

Learning pathology using collaborative vs. individual annotation of whole slide images: a mixed methods trial.

BACKGROUND: Students in biomedical disciplines require understanding of normal and abnormal microscopic appearances of human tissues (histology and histopathology). For this purpose, practical classes in these disciplines typically use virtual microscopy, viewing digitised whole slide images in web browsers. To enhance engagement, tools have been developed to enable individual or collaborative annotation of whole slide images within web browsers. To date, there have been no studies that have critically compared the impact on learning of individual and collaborative annotations on whole slide images. METHODS: Junior and senior students engaged in Pathology practical classes within Medical Science and Medicine programs participated in cross-over trials of individual and collaborative annotation activities. Students' understanding of microscopic morphology was compared using timed online quizzes, while students' perceptions of learning were evaluated using an online questionnaire. RESULTS: For senior medical students, collaborative annotation of whole slide images was superior for understanding key microscopic features when compared to individual annotation; whilst being at least equivalent to individual annotation for junior medical science students. Across cohorts, students agreed that the annotation activities provided a user-friendly learning environment that met their flexible learning needs, improved efficiency, provided useful feedback, and helped them to set learning priorities. Importantly, these activities were also perceived to enhance motivation and improve understanding. CONCLUSION: Collaborative annotation improves understanding of microscopic morphology for students with sufficient background understanding of the discipline. These findings have implications for the deployment of annotation activities in biomedical curricula, and potentially for postgraduate training in Anatomical Pathology.

Differential impact of student behaviours on group interaction and collaborative learning: medical students' and tutors' perspectives.

BACKGROUND: Collaboration is of increasing importance in medical education and medical practice. Students' and tutors' perceptions about small group learning are valuable to inform the development of strategies to promote group dynamics and collaborative learning. This study investigated medical students' and tutors' views on competencies and behaviours which promote effective learning and interaction in small group settings. METHODS: This study was conducted at UNSW Australia. Five focus group discussions were conducted with first and second year medical students and eight small group tutors were interviewed. Data were transcribed verbatim and thematic analysis was conducted. RESULTS: Students and tutors identified a range of behaviours that influenced collaborative learning. The main themes that emerged included: respectfulness; dominance, strong opinions and openness; constructiveness of feedback; active listening and contribution; goal orientation; acceptance of roles and responsibilities; engagement and enthusiasm; preparedness; self- awareness and positive personal attributes. An important finding was that some of these student behaviours were found to have a differential impact on group interaction compared with collaborative learning. This information could be used to promote higher quality learning in small groups. CONCLUSION: This study has identified medical students' and tutors' perceptions regarding interactional behaviours in small groups, as well as behaviours which lead to more effective learning in those settings. This information could be used to promote learning in small groups.