ABSTRACT
An Objective Structured Practical Examination (OSPE) is an effective and robust, but resource-intensive, means of evaluating anatomical knowledge. Since most OSPEs employ short answer or fill-in-the-blank style questions, the format requires many people familiar with the content to mark the examinations. However, the increasing prevalence of online delivery for anatomy and physiology courses could result in students losing the OSPE practice that they would receive in face-to-face learning sessions. The purpose of this study was to test the accuracy of Decision Trees (DTs) in marking OSPE questions as a first step to creating an intelligent, online OSPE tutoring system. The study used the results of the winter 2020 semester final OSPE from McMaster University's anatomy and physiology course in the Faculty of Health Sciences (HTHSCI 2FF3/2LL3/1D06) as the data set. Ninety percent of the data set was used in a 10-fold validation algorithm to train a DT for each of the 54 questions. Each DT was comprised of unique words that appeared in correct, student-written answers. The remaining 10% of the data set was marked by the generated DTs. When the answers marked by the DT were compared to the answers marked by staff and faculty, the DT achieved an average accuracy of 94.49% across all 54 questions. This suggests that machine learning algorithms such as DTs are a highly effective option for OSPE grading and are suitable for the development of an intelligent, online OSPE tutoring system.
ABSTRACT
The COVID-19 pandemic forced many universities and colleges to rapidly adopt online course delivery. As with any new foray, realizing the optimal aspects of a course to change became incredibly important for course instructors. In this study, we used a particularly sensitive method, i.e. Q-methodology, to evaluate changes based on students' perceptions from fall 2020 to winter 2021. Q-methodology is commonly used to uncover shared values, opinions, and preferences. Using Q-methodology, students participating in both semesters of an undergraduate anatomy and physiology course were surveyed in fall 2020 and winter 2021. The Q-sample included 44 statements. Data from fall 2020 were treated as the baseline and changes in students' perceptions from 2020 to 2021 were assessed. In total, 31 students completed both fall 2020 and winter 2021 course evaluations. Three salient factors emerged from the fall 2020 evaluation: Overtaxed students, Solo Achievers, and In-Person Learners. At the baseline, students were concerned mostly about the delivery of the course, then the winter 2021 evaluation showed how they were adjusting to online learning. The longitudinal Q-study proved to be robust in identifying changes in perceptions. These granular findings indicate how students might differ in viewing and evaluating online courses. This methodology can be used in redesigning and restructuring different components of an online course in higher education settings.
Subject(s)
COVID-19 , Students, Medical , Humans , Pandemics , COVID-19/epidemiology , Surveys and Questionnaires , AttitudeABSTRACT
As post-secondary education migrates online, developing and evaluating new avenues for assessment in anatomy is paramount. Three-dimensional (3D) visualization technology is one area with the potential to augment or even replace resource-intensive cadaver use in anatomical education. This manuscript details the development of a smartphone application, entitled "Virtual Reality Bell-Ringer (VRBR)," capable of displaying monoscopic two-dimensional (2D) or stereoscopic 3D images with the use of an inexpensive cardboard headset for use in spot examinations. Cadaveric image use, creation, and pinning processes are explained, and the source code is provided. To validate this tool, this paper compares traditional laboratory-based spot examination assessment stations against those administered using the VRBR application to test anatomical knowledge. Participants (undergraduate, n = 38; graduate, n = 13) completed three spot examinations specific to their level of study, one in each of the modalities (2D, 3D, laboratory) as well as a mental rotation test (MRT), Stereo Fly stereotest, and cybersickness survey. Repeated measures ANCOVA suggested participants performed significantly better on laboratory and 3D stations compared to 2D stations. Moderate to severe cybersickness symptoms were reported by 63% of participants in at least one category while using the VRBR application. Highest reported symptoms included: eye strain, general discomfort, difficulty focusing, and difficulty concentrating. Overall, the VRBR application is a promising tool for its portability, affordability, and accessibility. Due to reported cybersickness and other technical limitations, the use of VRBR as an alternative to cadaveric specimens presents several challenges when testing anatomy knowledge that must be addressed before widespread adoption.
