ABSTRACT
Faculty feedback program (FFP) at CMH Multan Institute of Medical Sciences (CIMS) was conducted for obtaining feedback for basic medical sciences faculty and evaluated to highlight its weaknesses for future improvement. The evaluation design was utilization-focused evaluation (UFE) keeping in mind its two essential elements. First element is the primary intended users (PIU) of the evaluation, namely the college faculty and students which were clearly identified and personally engaged to investigate intended use of the evaluation. Second element required the evaluator to ensure that the intended use of evaluation by PIU guide all other decisions made about the evaluation process. It was a mixed method study (qualitative and quantitative methods both) conducted from August 2018 to August 2019 in CIMS Multan with IRB approval following the steps of UFE. The whole program evaluation was conducted in two parts - first part constituted the 2018 manual FFP evaluation that provided suggestions for a FFP conducted in 2019 online. In step 2 the 2019 online FFP was evaluated again forming basis for future recommendations. Hence the PIUs response was recorded twice in the evaluation cycle - initially after the manual 2018 basic science FFP (response rate: 53%) - after which based on our findings a report was generated and recommendations suggested which were implemented in the 2019 online FFP and response observed again (response rate: 85.7%) to complete the evaluation cycle. Open-end questions were asked from faculty (qualitative analysis) with three themes emerging regarding FFP procedure, questionnaire and timing. An acknowledgement of shift of FFP procedure from manual (2018) to online system (2019) was observed in which faculty praised the ease (72.2%), confidentiality (66.6%), anonymity (50%) and transparency (33.3%) of the online system compared to manual FFP, which was reported to be a rather tense experience (83%). Regarding questionnaire, 38% faculty members reported feedback questions asked from students to be vague and 66.6% claimed that the timing was inappropriate and should have been end of academic year. When asked for suggestions for improvement in 2018 FFP, 72% faculty suggested training students on providing feedback and making the procedure user friendly (83%). Student response regarding both feedback was obtained online by a survey with closed ended questions (quantitative study). Fifty-three percent college students were satisfied with the online FFP giving an average rating of 3.2 to the software user interface and 85% affirmed that using the online software aided in providing anonymous responses helping them provide candid feedback. Seventy-five percent students agreed that online feedback system in 2019 had streamlined the feedback process and made it more efficient compared to the paper-based manual survey of 2018. After evaluating 2019 online FFP, few suggestions were recommended for future FFP including obtaining formative as well as summative faculty feedback, supplementing feedback with teacher's self-assessment/pen picture and incorporating 360 multi-source feedback.
