Development of an analytical rubric and estimation of its validity and inter-rater reliability for assessing reflective narrations.

Background Reflective practice is an integral component of continuing professional development. However, assessing the written narration is complex and difficult. Rubric is a potential tool that can overcome this difficulty. We aimed to develop, validate and estimate inter-rater reliability of an analytical rubric used for assessing reflective narration. Methods A triangulation type of mixed-methods design (Qual-Nominal group Technique, Quan-Analytical follow-up design and Qual-Open-ended response) was adopted to achieve the study objectives. Faculties involved in the active surveillance of Covid-19 participated in the process of development of assessment rubrics. The reflective narrations of medical interns were assessed by postgraduates with and without the rubric. Steps recommended by the assessment committee of the University of Hawaii were followed to develop rubrics. Content validity index and inter-rater reliability measures were estimated. Results An analytical rubric with eight criteria and four mastery levels yielding a maximum score of 40 was developed. There was a significant difference in the mean score obtained by interns when rated without and with the developed rubrics. Kendall's coefficient of concordance, which is a measure of concordance of scorers among more than two scorers, was higher after using rubrics. Conclusion Our attempt to develop an analytical rubric for assessing reflective narration was successful in terms of the high content validity index and better inter-rater concordance. The same process can be replicated to develop any such analytical rubric in the future.

Student support system for medical undergraduates: A qualitative exploration of stakeholder perspectives.

BACKGROUND: The demands and learning challenges in medical schools are not efficiently overcome by all learners. Despite the gravity of the problem, there is a dearth of studies to identify, define, and address the needs of learners. Thus, the present study was designed to do a situational analysis to identify and define the problems of learners and to develop a model for student support system in our institution. MATERIALS AND METHODS: A phenomenological type of qualitative research was undertaken. One-to-one in-depth interviews (IDIs) were conducted, 10 each among undergraduate medical students, faculties and parents (n = 30) to understand the problems of students and their suggested solutions from each one's perspective. The interviews were audio-recorded, transcribed verbatim, and manual thematic analyses were performed. RESULTS: Manual thematic analysis of the transcripts yielded 16 subcategories and 7 categories. The various categories that emerged are (1) curriculum related; (2) interpersonal adjustment problems; (3) personal issues and family problems; (4) cognitive learning disabilities; (5) poor organizational skills; (6) students' lack of motivation; and (7) miscellaneous. Based on the problems and their suggested solutions, a model for the essential components of a student support system for our college was developed. It outlines the principal roles of four key stakeholders, namely students, faculties, parents, and college administration. CONCLUSION: It has been found that students face various academic problems, personal, interpersonal and family level issues. We developed the support system model suitable for our context. In future, it may be implemented and evaluated to check if it achieves the desired purpose.