Formative peer evaluation instrument for a team-based learning course: Content and construct validity.

PURPOSE: Team-based learning (TBL) is an evidence-based approach to promote teamwork. Peer evaluation (PE) is an essential component to shape future team engagement and promote reflection. As PEs vary in use, implementation, and assessment, this study establishes the content and construct validity of a formative PE tool for a TBL course. METHODS: A ten-item instrument was developed based on a comprehensive review of PE literature and was critically edited by a team of experienced educators. Each student in a graduate histology course rated peers at two timepoints on a scale from Never to Always (0-3). The instrument's factor structure was analyzed by dividing the response set (D1 and D2); with D1 utilized for exploratory factor analysis (EFA) and D2 for confirmatory factor analysis (CFA). Cronbach's alpha assessed internal consistency. RESULTS: Data from 158 students across four cohorts were included in the analyses (D1, D2 = 972). A three-factor solution had good overall internal consistency (alpha = 0.82), and within the subscales ranged from 0.67 to 0.81. The factor structures were resonant with existing literature on (1) preparation, (2) providing feedback, and (3) feedback receptivity and attitude. CONCLUSION: This study establishes evidence of content and construct validity of a formative PE instrument for a TBL course.

Use of Mini-CEX as Formative Assessment Tool in the Training of Undergraduate Medical Students in ENT Situation Analysis and the Way Forward.

Background: Mini-CEX assesses clinical competency and is mainly used as a formative assessment tool. Its use in postgraduate training is well documented. However, Mini-CEX would play a significant role in training undergraduate medical students, especially with the commencement of competency-based medical education in India. This work reports the situational analysis of the Mini-CEX implementation in the department of ENT. Methods and Material: The Department of ENT is using Mini-CEX for formative assessment of students' clinical competence since 2017. Each student had to complete a minimum of five Mini-CEX encounters before the summative assessment. We reviewed the Mini-CEX assessment records of 149 undergraduate medical students who appeared for the summative exam in 2018. Results: We analysed the records of 874 Mini-CEX encounters. Each Mini-CEX encounter took 11 min on average. Each student completed five such assessments, which accounted for 55 min of one-to-one teacher-student interaction focused on clinical skills learning. The feedback time varied from 1 to 30 min. Feedback was focused on the cognitive (46%) and psychomotor (42%) domains. However, the majority of students reflected that they learned psychomotor skills during the Mini-CEX. Students selected only a few skills for the Mini-CEX, ignoring many must-know skills. Conclusions: Mini-CEX is feasible as a formative assessment tool for medical undergraduates' ENT training. It improves the assessor-student interaction, provides effective feedback, and develops the practice of reflection among students. However, regular review and training of the assessors and students are needed as a quality assurance measure.

Implementing Formative Assessment in Human Anatomy Practical Sessions: Medical Students' Perception and Effect on Final Exam Performance.

Background: Formative assessment with feedback is part of the assessment program in medical education to improve students' learning. Limited research has focused on its application and impact on practical anatomy education. Methods: This study aimed to examine medical students' perceptions of formative assessment in practical anatomy sessions of body systems-based educational units and explore its influence on final practical exam performance. A descriptive, cross-sectional study was conducted. Data was collected from 173 Year 2 medical students through a survey that addressed their perception of process and importance of formative assessment and feedback. The survey employed a 5-point Likert scale. Two open-ended questions were appended at the end of the survey. Students' performance in Unit 3 (where formative assessment was conducted) was compared to their performance in Unit 2 (where no formative assessment was conducted) and with the performance of the previous academic year's students in Unit 3 (where no formative assessment was conducted). Descriptive statistics were used. The level of statistical significance was set at p-value < 0.05. Responses to open-ended questions (qualitative data) were counted, categorized as themes, and presented as frequencies and percentages. Results: The survey showed high internal consistency, and its validity was established through exploratory factor analysis. Results showed that the mean mark for the unit with formative assessment and feedback was significantly higher than for the units without formative assessment and feedback. Students showed positive perception of formative assessment and feedback conducted after practical anatomy sessions. They reported useful insights regarding the benefits they gained from formative assessment and feedback as well as constructive suggestions for future improvements. Conclusion: The study indicates that students positively perceived formative assessment and feedback sessions after practical anatomy sessions. Findings also refer to a positive effect of formative assessment on students' performance in summative practical assessment in anatomy.

Early practice makes histology masters: The use of a formative assessment quiz to prepare histology learners for a high-stakes final examination.

Assessment of learners in the anatomical sciences is a complex task as it not only tests students' fact knowledge, but also the analysis of visual information. Sometimes, novice histology learners must acquire image recognition skills of microscopic structures in a short time frame. This paper describes a strategy of offering first year dental students at the University of Michigan a short, non-grade contributing, formative assessment quiz in order to better prepare them for a high-stakes, final summative histology examination. Data collected over 7 years indicate that students, who made use of this formative assessment opportunity, performed significantly better in their summative examination than students, who did not. Especially early practice quiz users profited most. Students, who used the practice quiz until they achieved a perfect score, also had statistically higher final examination scores. Students, who did not use the practice quiz, had a significantly lower cumulative D1-year Grade Point Average (D1-GPA) than students, who did, indicating that academically weaker students often underuse supporting learning resources. In general, scores from the Dental Admission Test (DAT) were weak predictors of learning success in the DENT 510 histology course. In contrast, the D1-GPA values had a medium strength positive correlation with final histology examination scores. The major problem that was encountered with this practice quiz strategy was that many students waited until close to the day of the final examination before taking advantage of this feedback opportunity, thereby reducing its potential benefit for improving their overall learning strategy for histology.

Serious Illness Communication Tool for Palliative Care Fellows: Development, Implementation, and Lessons Learned.

Introduction: Learning expert communication skills is a core educational goal within palliative care training, yet there are few communication-based educational tools specifically designed for specialty-trained palliative care providers. Objective: To develop and implement a tool to facilitate effective learning of serious illness communication for hospice and palliative medicine fellows. Methods: A novel formative assessment tool was developed within the UCLA Palliative Care Fellowship program, and utilized throughout the academic year on a weekly basis. Focus groups were held for fellows and faculty separately at the end of the academic year in order to gain insight into the experience and effectiveness of the tool. Focus group transcripts were analyzed through thematic analysis. Results: There was a 47% participation rate in the focus groups (n = 7). Qualitative analysis demonstrated positive impact of the tool in identifying learning goals, improving quality of feedback, and in standardizing language around advanced communication skills, with most value in the first half of the fellowship year. Some aspects of the tool were found to increase feedback anxiety, including the competency scoring component, frequency of use, and utilization with individual patient encounters. Conclusion: This novel communication tool provides an important addition to serious illness communication training of specialist palliative care providers, in creating a shared mental model for naming and organizing skills, as well as generating specific high quality learning goals and feedback. Preliminary feedback from our pilot implementation phase provided important information about how to refine the tool and remove components that unnecessarily added to feedback anxiety.

Experiences of medical students and faculty regarding the use of long case as a formative assessment method at a tertiary care teaching hospital in a low resource setting: a qualitative study.

INTRODUCTION: The long case is used to assess medical students' proficiency in performing clinical tasks. As a formative assessment, the purpose is to offer feedback on performance, aiming to enhance and expedite clinical learning. The long case stands out as one of the primary formative assessment methods for clinical clerkship in low-resource settings but has received little attention in the literature. OBJECTIVE: To explore the experiences of medical students and faculty regarding the use of the Long Case Study as a formative assessment method at a tertiary care teaching hospital in a low-resource setting. METHODOLOGY: A qualitative study design was used. The study was conducted at Makerere University, a low-resource setting. The study participants were third- and fifth-year medical students as well as lecturers. Purposive sampling was utilized to recruit participants. Data collection comprised six Focus Group Discussions with students and five Key Informant Interviews with lecturers. The qualitative data were analyzed by inductive thematic analysis. RESULTS: Three themes emerged from the study: ward placement, case presentation, and case assessment and feedback. The findings revealed that students conduct their long cases at patients' bedside within specific wards/units assigned for the entire clerkship. Effective supervision, feedback, and marks were highlighted as crucial practices that positively impact the learning process. However, challenges such as insufficient orientation to the long case, the super-specialization of the hospital wards, pressure to hunt for marks, and inadequate feedback practices were identified. CONCLUSION: The long case offers students exposure to real patients in a clinical setting. However, in tertiary care teaching hospitals, it's crucial to ensure proper design and implementation of this practice to enable students' exposure to a variety of cases. Adequate and effective supervision and feedback create valuable opportunities for each learner to present cases and receive corrections.

Assessment in Undergraduate Competency-Based Medical Education: A Systematic Review.

BACKGROUND: Studies that have methodically compiled the body of research on the competency-based medical education (CBME) assessment procedure and pinpointed knowledge gaps about the structure of the assessment process are few. Thus, the goals of the study were to create a model assessment framework for competency-based medical education that would be applicable in the Indian setting as well as to thoroughly examine the competency-based medical education assessment framework. METHODS: PubMed, MEDLINE (Ovid), EMBASE (Ovid), Scopus, Web of Science, and Google Scholar were the databases that were searched. The search parameters were restricted to English language publications about competency-based education and assessment methods, which were published between January 2006 and December 2020. A descriptive overview of the included research (in tabular form) served as the foundation for the data synthesis. RESULTS: Databases provided 732 records; out of which 36 fulfilled the inclusion and exclusion criteria. Thirty-six studies comprised a mix of randomized controlled trials, focus group interviews, and questionnaire studies, including cross-sectional studies, qualitative studies (03), mixed-method studies, etc. The papers were published in 10 different journals. The greatest number was published in BMC Medical Education (18). The average quality score for included studies was 62.53% (range: 35.71-83.33%). Most authors are from the UK (07), followed by the USA (05). The included studies were grouped into seven categories based on their dominant focus: moving away from a behavioristic approach to a constructive approach of assessment (01 studies), formative assessment (FA) and feedback (10 studies), the hurdles in the implementation of feedback (04 studies), utilization of computer or online based formative test with automated feedback (05 studies), video feedback (02 studies), e-learning platforms for formative assessment (04 studies), studies related to workplace-based assessment (WBA)/mini-clinical evaluation exercise (mini-CEX)/direct observation of procedural skills (DOPS) (10 studies). CONCLUSIONS: Various constructivist techniques, such as concept maps, portfolios, and rubrics, can be used for assessments. Self-regulated learning, peer feedback, online formative assessment, an online computer-based formative test with automated feedback, the use of a computerized web-based objective structured clinical examination (OSCE) evaluation system, and the use of narrative feedback instead of numerical scores in mini-CEX are all ways to increase student involvement in the design and implementation of the formative assessment.

Efficacy of Google Form-based MCQ tests for formative assessment in medical biochemistry education.

BACKGROUND: Biochemistry, being a vast and complex subject, can be challenging for Phase I MBBS students to comprehend and retain. Embracing rapidly evolving technology can facilitate a more accessible learning experience. In this study, we investigated the effectiveness of using Google Form-based multiple-choice question (MCQ) tests as a formative assessment tool after each biochemistry lecture series. The aim was to assess the improvement and gather feedback of Phase I MBBS students on the utility of this assessment tool. MATERIALS AND METHODS: This educational prospective longitudinal study was conducted by the Department of Biochemistry at a university-affiliated medical college and tertiary care hospital. The study included 150 Phase I MBBS students as participants. Google Form-based MCQ tests were implemented as educational interventions after each lecture series during the study period. The study compared the internal assessment (IA) MCQ marks of students before and after the implementation of the intervention. In addition, feedback questionnaires were collected from the students. RESULTS: There was a significant improvement in students' scores between the first IA (mean ± standard deviation [SD], 8.16 ± 3.08) and second IA (mean ± SD, 17.64 ± 2.02) (P < 0.0001). According to students' feedback, 149 out of 150 (99.3%) students found the use of Google Form-based MCQ tests as a formative assessment tool in the teaching-learning process to be highly beneficial and motivated them to engage in their biochemistry studies. CONCLUSION: With the shift toward competency-based medical education (CBME) in India, it is crucial for educators to embrace novel teaching-learning and evaluation approaches. Our study highlighted the efficacy of employing Google Form-based MCQ tests in enhancing students' comprehension of the biochemistry subject, evaluating their scores and improving the overall quality of learning. Through this mode of assessment, teachers were able to provide targeted feedback on areas that required improvement, thereby enhancing the learning experience.

Formative assessment in K-12 English as a foreign language education: A systematic review.

The surge of globalisation and the recognition of English as a lingua franca have propelled the worldwide advancement of English as a foreign language (EFL) education. Formative assessment can support the English language teaching and learning in classroom settings. This systematic review critically analyses recent research on formative assessment within K-12 EFL education, underscoring its substantial role in enhancing teaching efficacy and student outcomes. Specifically, it addresses three key aspects: the beliefs, perceptions and literacy of teachers and students regarding formative assessment, the effectiveness of different assessment practices on learners' performance and the contextual challenges in implementing these strategies. A comprehensive analysis of qualitative and quantitative studies reveals that the attitudes of teachers and students considerably influence the effectiveness of formative assessment. Innovative methods, such as technology-assisted and peer assessments, show promise in enhancing learner outcomes. However, the education system and cultural contexts present persistent challenges in implementing formative assessment in EFL classes. The findings highlight the importance of ensuring professional training for educators and raising student awareness regarding the benefits of formative assessment. This study emphasises the need for aligning adaptable assessment practices with diverse educational environments and calls for a closer integration of theoretical and practical approaches in EFL education. Moreover, this review provides valuable insights into formative assessment strategies in EFL learning, offering guidance for educators, policymakers and stakeholders in language education.

Optimising the diagnostic accuracy of First post-contrAst SubtracTed breast MRI (FAST MRI) through interpretation-training: a multicentre e-learning study, mapping the learning curve of NHS Breast Screening Programme (NHSBSP) mammogram readers using an enriched dataset.

BACKGROUND: Abbreviated breast MRI (FAST MRI) is being introduced into clinical practice to screen women with mammographically dense breasts or with a personal history of breast cancer. This study aimed to optimise diagnostic accuracy through the adaptation of interpretation-training. METHODS: A FAST MRI interpretation-training programme (short presentations and guided hands-on workstation teaching) was adapted to provide additional training during the assessment task (interpretation of an enriched dataset of 125 FAST MRI scans) by giving readers feedback about the true outcome of each scan immediately after each scan was interpreted (formative assessment). Reader interaction with the FAST MRI scans used developed software (RiViewer) that recorded reader opinions and reading times for each scan. The training programme was additionally adapted for remote e-learning delivery. STUDY DESIGN: Prospective, blinded interpretation of an enriched dataset by multiple readers. RESULTS: 43 mammogram readers completed the training, 22 who interpreted breast MRI in their clinical role (Group 1) and 21 who did not (Group 2). Overall sensitivity was 83% (95%CI 81-84%; 1994/2408), specificity 94% (95%CI 93-94%; 7806/8338), readers' agreement with the true outcome kappa = 0.75 (95%CI 0.74-0.77) and diagnostic odds ratio = 70.67 (95%CI 61.59-81.09). Group 1 readers showed similar sensitivity (84%) to Group 2 (82% p = 0.14), but slightly higher specificity (94% v. 93%, p = 0.001). Concordance with the ground truth increased significantly with the number of FAST MRI scans read through the formative assessment task (p = 0.002) but by differing amounts depending on whether or not a reader had previously attended FAST MRI training (interaction p = 0.02). Concordance with the ground truth was significantly associated with reading batch size (p = 0.02), tending to worsen when more than 50 scans were read per batch. Group 1 took a median of 56 seconds (range 8-47,466) to interpret each FAST MRI scan compared with 78 (14-22,830, p < 0.0001) for Group 2. CONCLUSIONS: Provision of immediate feedback to mammogram readers during the assessment test set reading task increased specificity for FAST MRI interpretation and achieved high diagnostic accuracy. Optimal reading-batch size for FAST MRI was 50 reads per batch. Trial registration (25/09/2019): ISRCTN16624917.

Transforming histology students from passive lecture listeners into active lecture learners.

Traditional academic lectures have long been criticized as providing a passive learning environment to students. Often, they do not capture the audience's attention, resulting in learners being distracted or bored and thereby reducing their learning efficacy. Consequently, they are being abandoned by many schools and universities as an educational modus or modified into more learner-centered experiences. This descriptive article outlines a strategy of inserting active learning PowerPoint slides into traditional histology lectures. Suggestions and advice for adding or improving existing active learning slides are being offered to readers, who are planning to modify their own lecture presentations. The author's experience of using this formative assessment strategy with different types of histology learners is also discussed.

Test-Only Learning via Virtual Patients to Improve Surgical Illness Scripts of Preclinical Medical Students as a Preparation for Clinical Clerkships: An Experimental Study.

OBJECTIVE: The aim was to determine whether preclinical medical students can develop their illness scripts to a level comparable to that of clerkship students through test-only learning using repeated formative online testlets. METHODS: In this experimental study, participants were 52 preclinical and 53 clinical medical students. The intervention group consisted of preclinical medical students, and the control group consisted of clinical medical students. The intervention group responded to online testlets containing feedback, an innovative formative assessment method called ContExtended Questions, on general surgery for 8 days by spending no more than 30 minutes each day. The control group completed the general surgery clerkship. The performances were assessed using 20 Key-Feature Question items. The intervention group was assessed twice: immediately after the intervention (the immediate test), and again 1 month later (the delayed test). The control group was assessed once, immediately after the clerkship. All performance tests were identical. RESULTS: The preclinical students had a significantly higher mean score on the immediate test (83.1 ± 9.6) compared to the clinical students (75.4 ± 8.9), P < .001. The effect size (Cohen's d) was .83. However, the mean score in the delayed test (76.9 ± 13.6) was not significantly different from clinical students' mean score (75.4 ± 8.9), P > .05. CONCLUSIONS: Test-only learning as a spaced repetition of online formative testlets is effective in preparing preclinical medical students to the clinical clerkship. Through using this approach in preclinical period, they can prepare themselves for the clinical environment to optimize the benefits derived from clerkships.

Improving assessment of procedural skills in health sciences education: a validation study of a rubrics system in neurophysiotherapy.

BACKGROUND: The development of procedural skills is essential in health sciences education. Rubrics can be useful for learning and assessing these skills. To this end, a set of rubrics were developed in case of neurophysiotherapy maneuvers for undergraduates. Although students found the rubrics to be valid and useful in previous courses, the analysis of the practical exam results showed the need to change them in order to improve their validity and reliability, especially when used for summative purposes. After reviewing the rubrics, this paper analyzes their validity and reliability for promoting the learning of neurophysiotherapy maneuvers and assessing the acquisition of the procedural skills they involve. METHODS: In this cross-sectional and psychometric study, six experts and 142 undergraduate students of a neurophysiotherapy subject from a Spanish university participated. The rubrics' validity (content and structural) and reliability (inter-rater and internal consistency) were analyzed. The students' scores in the subject practical exam derived from the application of the rubrics, as well as the rubrics' criteria difficulty and discrimination indices were also determined. RESULTS: The rubrics´ content validity was found to be adequate (Content Validity Index > 0.90). These showed a unidimensional structure, and an acceptable internal consistency (α = 0.71) and inter-rater reliability (Fleiss' ƙ=0.44, ICC = 0.94). The scores of the subject practical exam practically covered the entire range of possible theoretical scores, showing all the criterion medium-low to medium difficulty indices - except for the one related to the physical therapist position-. All the criterion exhibited adequate discrimination indices (rpbis > 0.39), as did the rubric as a whole (Ferguson's δ = 0.86). Students highlighted the rubrics´ usefulness for learning the maneuvers, as well as their validity and reliability for formative and summative assessment. CONCLUSIONS: The changed rubrics constitute a valid and reliable instrument for evaluating the execution quality of neurophysiotherapy maneuvers from a summative evaluation viewpoint. This study facilitates the development of rubrics aimed at promoting different practical skills in health-science education.

Does 'summative' count? The influence of the awarding of study credits on feedback use and test-taking motivation in medical progress testing.

Despite the increasing implementation of formative assessment in medical education, its' effect on learning behaviour remains questionable. This effect may depend on how students value formative, and summative assessments differently. Informed by Expectancy Value Theory, we compared test preparation, feedback use, and test-taking motivation of medical students who either took a purely formative progress test (formative PT-group) or a progress test that yielded study credits (summative PT-group). In a mixed-methods study design, we triangulated quantitative questionnaire data (n = 264), logging data of an online PT feedback system (n = 618), and qualitative interview data (n = 21) to compare feedback use, and test-taking motivation between the formative PT-group (n = 316), and the summative PT-group (n = 302). Self-reported, and actual feedback consultation was higher in the summative PT-group. Test preparation, and active feedback use were relatively low and similar in both groups. Both quantitative, and qualitative results showed that the motivation to prepare and consult feedback relates to how students value the assessment. In the interview data, a link could be made with goal orientation theory, as performance-oriented students perceived the formative PT as not important due to the lack of study credits. This led to low test-taking effort, and feedback consultation after the formative PT. In contrast, learning-oriented students valued the formative PT, and used it for self-study or self-assessment to gain feedback. Our results indicate that most students are less motivated to put effort in the test, and use feedback when there are no direct consequences. A supportive assessment environment that emphasizes recognition of the value of formative testing is required to motivate students to use feedback for learning.

Correlation Between Student Performances on Case-Based Constructed-Response Formative Assessment and Summative Assessment.

OBJECTIVES: This study aimed to evaluate the impact of formative assessment with case-based constructed-response question (CRQ) formats on student performance on the final summative assessment in the second-year periodontics course. METHODS: Classroom quizzes with case-based CRQs were implemented as the formative assessment during the course. Each student received feedback on their responses from the course director. After all students (N = 128) took the second-year final examination, the Friedman test was conducted to compare student performances in each assessment over time. The multiple linear regression (MLR) model was used to evaluate the association between the second-year final examination score and plausible predictors-student gender, the second-year formative and midterm examination scores, and time spent on the final examination. RESULTS: The mean % scores in the formative assessment (51) and midterm (84) examination were significantly lower than that of the final (87) examination (P < .01). The number of students who failed the final (6) examination was significantly lower than the midterm (16) examination (P = .03). The midterm (P < .0001) and the formative assessment (P = .0009) scores significantly affected the second-year final examination score while student gender (P = .59) and time spending (P = .83) showed no correlations. CONCLUSION: Within the limitations of the study, student performance on case-based CRQs was correlated with student performance on the summative assessment.

Why teachers do (or do not) implement recommended teaching practices? An application of the theory of planned behavior.

Introduction: In Luxembourg, competency-based practices (CBP), differentiated instruction (DI), and formative assessment (FA) have been imposed by the 2009 school law. Referring to the Theory of Planned Behavior (TPB), this study examined factors influencing the implementation of these practices in classrooms. Methods: Teachers participated in an online survey assessing their attitudes, subjective norm, perception of behavioral control, intention, and pedagogical practices regarding CBP, DI, or FA. Measurement models were used in structural equation models testing the TPB. Results: If the main relationships postulated by the theory were confirmed, some inconstancies were observed depending on the targeted practices. Structural equation TPB models controlling for gender, experience, teaching level, and socio-economic level of the school population explained between 20 and 45% of the variance in teachers' practices, and between 65 and 75% of the variance in teachers' intention to use these practices. Discussion: The relevance of the TPB for studying teaching practices and implications for professional training are discussed.

The Impact of Script Concordance Testing on Clinical Decision-Making in Paramedic Education.

This study investigates the effectiveness of the Script Concordance Test (SCT) in enhancing clinical reasoning skills within paramedic education. Focusing on the Medical University of Lublin, we evaluated the SCT's application across two cohorts of paramedic students, aiming to understand its potential to improve decision-making skills in emergency scenarios. Our approach, informed by Van der Vleuten's assessment framework, revealed that while the SCT's correlation with traditional methods like multiple-choice questions (MCQs) was limited, its formative nature significantly contributed to improved performance in summative assessments. These findings suggest that the SCT can be an effective tool in paramedic training, particularly in strengthening cognitive abilities critical for emergency responses. The study underscores the importance of incorporating innovative assessment tools like SCTs in paramedic curricula, not only to enhance clinical reasoning but also to prepare students for effective emergency responses. Our research contributes to the ongoing efforts in refining paramedic education and highlights the need for versatile assessment strategies in preparing future healthcare professionals for diverse clinical challenges.

Evaluation of student readiness for clinical dental education at the end of preclinical education.

OBJECTIVES: This study examined results of a summative objective structured clinical examination (OSCE) at the end of preclinical periodontal education to identify deficient areas for dental students in patient care and to explore factors affecting the probability of passing the OSCE. METHODS: The summative OSCE was administered to two consecutive cohorts, Classes A (class of 2024; n1 = 134) and B (class of 2025; n2 = 129). The questions for each station in the OSCEs were available to both classes 1 week before the OSCEs. Descriptive statistics were used to identify deficient areas. The multiple logistic regression model was built to predict the probability of passing the OSCE based on the cohort, gender, and the practical and written examination scores. RESULTS: Fifty-one (38%) students in Class A and 66 (51%) students in Class B completed the OSCE by passing all stations. Students undergoing remediation showed deficiencies in demonstrating how to detect tooth mobility, performing periodontal probing, drawing the healthy positive bony architecture and the mucogingival junction, and using a universal and a Gracey 13/14 curette. The probability of passing the OSCE was significantly correlated with Class B (p = 0.035) and the practical examination score (p = 0.03) while not associated with gender (p = 0.53) and the written examination score (p = 0.11). CONCLUSION: Students showed deficiencies in assessment skills at the end of preclinical education. The study findings suggest that the implementation of the OSCE at the conclusion of preclinical education would be beneficial since the written examination score might not accurately reflect student readiness for clinical patient care.

Feasibility, usability, and acceptability of MobileCoach-Teen: A smartphone app-based preventative intervention for risky adolescent drinking behavior.

BACKGROUND: Older adolescence (ages 15-18) is a critical period for experimentation with substance use, especially alcohol. Adolescent drinking poses hazards to physical and mental health, amplifies risk associated with other activities typically initiated during this life stage (e.g., driving, sexual activity), and is associated with adverse outcomes in adolescence and adulthood. Existing preventative interventions are expensive and have questionable long-term efficacy. Digital interventions may represent an accessible and personalized approach to providing preventative intervention content to youth. METHODS: This study recruited 29 adolescents aged 16-18 (M = 17.24, SD = 0.74) for a pilot feasibility trial of the MobileCoach-Teen (MC-Teen) smartphone app-based intervention. The study team randomized participants to receive either the alcohol intervention (MC-Teen) or attention control pseudo-intervention (MC-Fit). MC-Teen participants received 12 weeks of content adapted from a prior Swiss-based trial of a preventative alcohol intervention. Participants provided qualitative and quantitative feedback at baseline, via six biweekly surveys during and post-intervention. RESULTS: Both groups rated the application as easy to download (M = 4.31, SD = 0.93; 5-point Likert). All participants completed the baseline survey in less than the estimated time of 10 min (M = 7:42, SD = 2:15) and rated the survey as easy to complete (M = 4.69, SD = 0.60; 5-point Likert). MC-Teen participants favorably assessed application user experience, message user experience, and digital working alliance with application. Qualitative themes included a desire for increased rate/amount and diversity of content, greater representation via coach options, user interface/user experience improvements, and additional features. CONCLUSION: The MC-Teen intervention is feasible and acceptable based on a pilot feasibility trial with a sample of U.S. adolescents.