Construction of the evaluation index system for pharmacist training effectiveness based on Kirkpatrick evaluation model / 药学实践杂志

Objective To construct the evaluation index system for pharmacist training effectiveness in China. Methods The initial framework of index system based on Kirkpatrick evaluation model was constructed through literature research and expert interview method. Delphi method was used to analyze the index system. Results The evaluation index system for pharmacist training effectiveness in China included 4 first-level indicators, reaction evaluation, learning evaluation, behavior evaluation and result evaluation, 9 second-level indicators, training scheme, teaching staff, training conditions, theoretical knowledge, practical skills, the change of work ability, the change of professional attitude, personal income and organizational income, as well as 32 third-level indexes such as pharmaceutical professional knowledge. Conclusion This evaluation index system for pharmacist training result is highly scientific and systematic. It covers the whole-process and follow-up evaluation of the training activities. It can optimize the pharmacist training schedule and improve the program design for the effective pharmacist training evaluation.

The Asian Sarcoma Consortium sarcoma preceptorship program: A program evaluation study utilizing the kirkpatrick model (level 1 and 2)

Background@#Sarcomas, a heterogenous group of malignancies consisting of tumors, are rarely encountered in most general oncology clinics. As a result, most general oncologists have very little experience in dealing with these tumors and resulting in poorer treatment and survival outcomes. Having successful collaborative efforts in research, the Asian Sarcoma Consortium (ASC) pioneered a Sarcoma Preceptorship Program with its main goal of increasing awareness of the multidisciplinary approach in the management of sarcomas.@*Objective@#The objective of this study is to evaluate the ASC Sarcoma Preceptorship Program using the first 2 levels of the Kirkpatrick Evaluation Model: Level 1) Reaction – through using satisfaction surveys; and Level 2) Knowledge – focusing on the amount of information gained from the activity evaluated by test-retest method.@*Methods@#The 2-day preceptorship program held in Singapore utilized educational activities such as didactic lectures, multidisciplinary case discussions, and case presentations. The program was evaluated using: 1) pre-test and posttest; 2) satisfaction survey from ASC; and 3) satisfaction survey made by the sponsor, Novartis Pharma.@*Results@#The preceptorship program enrolled 30 participants, most from the Asia-Pacific region. The overall results from the satisfaction surveys were generally optimistic, with results showing high levels of satisfaction with regard to the time allotment, scope of topics, and how helpful each session was. Test results showed that participants scored lower in post-test as compared to the pre-test. Computation of the Coefficient of Test reliability showed that although the pre-test was highly reliable, this was not true of the post-test.@*Conclusions@#Based on the Level 1 evaluation, the program was successful in terms of both participant and faculty satisfaction. However, certain areas for improvement were identified and recommendations were made in order to effect improvements for future iterations of this preceptorship.

Evaluation of the effectiveness of written communication training of a public higher education institution using the Kirkpatrick model

BACKGROUND@#This study aims to evaluate the effectiveness of the “Effective Written Communication Training” provided among 16 UP Manila administrative employees following Kirkpatrick's training evaluation model.@*OBJECTIVES@#This study specifically examines the reaction of the employees to the training program (Level 1), changes in the level of learning (Level 2), and transfer of training in the workplace or behavioral changes (Level 3).@*METHODOLOGY@#The study used a cross-sectional design, and the primary data was collected through survey questionnaires, a pretest, and a posttest. For the Level 1 evaluation, a training assessment survey was given to the participants. For the Level 2 evaluation, tests were given to the participants before and after the conduct of the training program. For the Level 3 evaluation, a survey questionnaire was given to the participants and their immediate supervisor five months after the end of the training.@*RESULTS@#The Level 1 results showed that the participants' overall reaction to the training was Excellent in terms of training design, facilitation, resource speaker/facilitator, and overall experience. For the Level 2 evaluation, results showed that there is a significant change (t(15)=11.32, ρ<0.05) in the participants' learning about written communication, as shown in the difference between the pretest (M=16.56, SD=3.01) and posttest scores (M=20.25, SD=2.49). Results of the Level 3 evaluation showed that written communication competencies have been applied to the workplace. Employees confidently compose written correspondences, construct formal and official letters, and relay queries using a more appropriate choice of words, which the participants' supervisors also observed.@*CONCLUSION@#Overall, the training program was found to be effective in improving the participants' written communication skills, following Kirkpatrick's training evaluation model.

Application research of segmented teaching combined with Kirkpatrick model evaluation in the teaching of PICC specialized nurses / 中华医学教育探索杂志

Objective:To explore the application effect of segmented teaching combined with Kirkpatrick model evaluation in the teaching of specialized nurses of peripherally implanted central venous catheter (PICC).Methods:A total of 60 trainees in the PICC specialized nurse training class in PICC Clinic of Cancer Hospital Affiliated to Nantong University from June 2020 to September 2020 were selected in the study, and they were randomly divided into a control group and a study group, with 30 nurses in each group. The control group conducted traditional training for PICC specialized nurses, and the research group used segmented teaching. After training, the Kirkpatrick model was used to evaluate the response level, learning level, behavior level and result level of the training effect of the two groups of PICC specialized nurses. SPSS 22.0 was used for t test and chi-square test. Results:Through the segmented teaching training, the evaluation results of the Kirkpatrick model showed that the effect of the PICC specialized nurse training was significantly better in the study group than that in the control group in terms of response level, learning level, behavior level and result level. The study group's satisfaction with the training program and teaching content scored higher than that of the control group, with statistically significance ( P<0.001). The research group nurses' mastery of theoretical knowledge and practical skills and humanistic care scores were higher than those of the control group, with significant differences ( P<0.001). After the training, the self-behavior assessment scores of the two groups increased, and the improvement of the research group was more obvious. When the nurses of the research group returned to the hospital clinical work after training, their professional identity, PICC teaching and guidance ability, treatment, nursing service quality, the recognition of patient and family members and the effectiveness of the department were improved to varying degrees, the corresponding measures could be carried out in the hospital, and the development of PICC technology was promoted. Conclusion:The combination of segmented teaching and Kirkpatrick model evaluation can effectively meet the training needs of PICC specialized nurses and make a full range of effect evaluations to improve the quality of PICC specialized nursing services.

Construction of a training evaluation index system of sterility concept for nursing interns in the operating room based on Kirkpatrick model / 中华医学教育探索杂志

Objective:Based on the Kirkpatrick model, construct an evaluation index system for the teaching effect of aseptic concept for interns in operating room.Methods:Based on the Kirkpatrick model as the theoretical framework, combined with literature review and group discussion, the first draft of the indicator system was drawn up, and the expert consultation questionnaire was compiled. 30 experts with nursing teaching experience from several tertiary hospitals and medical undergraduate colleges in Chengdu were selected for research. Delphi conducts 2 rounds of consultation with experts. The collected data were input by Excel, and analyzed by SPSS 23.0.Results:The positive coefficients of the two rounds of consulting experts were l00.00% and 93.33%; the expert authority coefficients were both>0.7; the overall coordination coefficient of expert opinions Kendall's W was 0.321 and 0.427, respectively, with statistical significance ( P<0.05). The final result index system for training nurses in operating room aseptic concept includes 4 first-level indicators, 11 second-level indicators, and 45 third-level indicators. Conclusion:The construction method of aseptic concept assessment index system for nursing students in operating room is scientific and reasonable, and the content is set appropriately, which can provide a basis for the evaluation of aseptic concept training for nursing students in operating room.

Action research on optimizing nurse training of "Internet+ TCM nursing" based on CIPP and Kirkpatrick Model / 中华医学教育探索杂志

Objective:To explore whether the action research on the pre-service training evaluation system of "Internet + TCM Nursing" guided by CIPP (context, input, process and product) and Kirkpatrick Model can improve the training effect of nursing staff under the background of internet and TCM nursing service.Methods:Taking 211 nurses of "Internet + TCM Nursing" in Guangdong Provincial Hospital of TCM from 2019 to 2020 as the research objects, action teams were made, and the data were collected and analyzed by various methods. According to the collected data, the action teams constructed the training scheme of "Internet + TCM Nursing", the compilation requirements of "internet + TCM Nursing", the case optimization scheme of "internet + TCM Nursing" process, and the evaluation model of "internet + TCM Nursing". After the training, the nurses' knowledge, skills and related attitudes, the general reaction and feelings to the training programs, and the evaluation of nurses' work performance by the competent leaders, medical staff and clients were compared by comparing their scores and issuing self-made questionnaires. SPSS 20.0 software was used for statistical analysis.Results:The results showed that the theoretical scores of nurses after training were (85.16±6.22) points, and the case assessment scores after training [(81.22±7.03) points] were higher than the previous theoretical scores [(61.23±12.90) points] and the case assessment scores [(59.54±14.41) points], with statistical significance ( P<0.001). The results of self-made questionnaire showed that there were significant differences in theoretical knowledge, skills and related attitudes among nurses after training ( P<0.001). In the process of training, there were statistically significant differences in training content, teacher training, training implementation and training guarantee at the end of training ( P<0.001). After training, there were significant differences in the evaluation of nurses' job performance by leaders in charge, medical staff and clients improved in knowledge and skills, professional ethics, working process and learning ability, and the improvement rate ( P<0.001). Conclusion:Action research is an effective research method to continuously improve nursing training. The optimization scheme of "internet + TCM Nursing" nurses based on CIPP and Kirkpatrick Model through action research can effectively improve the training effect.

Learning Outcome of Continued Education Based on Kirkpatrick Model for Professionals in Rehabilitation Institutes / 中国康复理论与实践

Objective:To evaluate the learning outcome of continued education based on Kirkpatrick model for professionals in rehabilitation institutes. Methods:A total of 190 rehabilitation trainees from 2018 to 2019 were investigated with Kirkpatrick model, from the reaction layer, learning layer, behavior layer and results layer, to evaluate the training effect. Results:From reaction layer, the total satisfaction was high from three aspects including general feeling, hospital management and department management. From learning layer, the score of theoretical results significantly improved (t = 32.476, P < 0.001), and the score of practical skill was (81.99±9.59). From behavior layer, their abilities of doctor-patient communication, risk management, writing, rehabilitation evaluation, identifying key points of problems, formulating rehabilitation plans and implementing operations improved (t > 14.364, P < 0.001). From results layer, the complaint rate was 0%, and the accident rate was 0%; three trainees published papers. Conclusion:Kirkpatrick model could evaluate the training comprehensively. Rehabilitation training could improve the quality and ability of trainees.

The Asian Sarcoma Consortium Sarcoma Preceptorship Program: A program evaluation study utilizing the Kirkpatrick Model (Level 1 and 2)

Background@#Sarcomas, a heterogenous group of malignancies consisting of tumors, are rarely encountered in most general oncology clinics. As a result, most general oncologists have very little experience in dealing with these tumors and resulting in poorer treatment and survival outcomes. Having successful collaborative efforts in research, the Asian Sarcoma Consortium (ASC) pioneered a Sarcoma Preceptorship Program with its main goal of increasing awareness of the multidisciplinary approach in the management of sarcomas. @*Objective@#The objective of this study is to evaluate the ASC Sarcoma Preceptorship Program using the first 2 levels of the Kirkpatrick Evaluation Model: Level 1) Reaction – through using satisfaction surveys; and Level 2) Knowledge – focusing on the amount of information gained from the activity evaluated by test-retest method. @*Methods@#The 2-day preceptorship program held in Singapore utilized educational activities such as didactic lectures, multidisciplinary case discussions, and case presentations. The program was evaluated using: 1) pre-test and posttest; 2) satisfaction survey from ASC; and 3) satisfaction survey made by the sponsor, Novartis Pharma. @*Results@#The preceptorship program enrolled 30 participants, most from the Asia-Pacific region. The overall results from the satisfaction surveys were generally optimistic, with results showing high levels of satisfaction with regard to the time allotment, scope of topics, and how helpful each session was. Test results showed that participants scored lower in post-test as compared to the pre-test. Computation of the Coefficient of Test reliability showed that although the pre-test was highly reliable, this was not true of the post-test.@*Conclusions@#Based on the Level 1 evaluation, the program was successful in terms of both participant and faculty satisfaction. However, certain areas for improvement were identified and recommendations were made in order to effect improvements for future iterations of this preceptorship.

Evaluation index system of training effectiveness for new recruiting nurses was constructed based on the theory of Kirkpatrick Model / 中国实用护理杂志

Objective To construct evaluation index system of training effectiveness for new recruiting nurses based on the theory of Kirkpatrick Model. Methods The draft of evaluation index system of training effectiveness for new recruiting nurses was drawn up through literature survey and interview method first, and then it was constructed by using Delphi method and statistical analysis. Results Through two rounds of expert consultations, the experts′ suggestions were revised and summarized, and then the evaluation index system was constructed, including 4 grade I indexes, 14 gradeⅡindexes, and 56 gradeⅢindexes. The positive coefficients of the two rounds of expert consultation were 96.15% and 100.00% , respectively, with authoritative coefficient of 0.877. The Kendall Harmony Coefficients of importance and operability of grade I, Ⅱ, and Ⅲ indexes were 0.640, 0.378, 0.296, and 0.707, 0.515, 0.416, respectively (P< 0.01). Conclusions The enthusiasm, authority level and coordination degree of the experts were all at a high level. The evaluation index system of training effectiveness for new recruiting nurses was of high importance and operability.

Evaluation index system of training effectiveness for new recruiting nurses was constructed based on the theory of Kirkpatrick Model / 中国实用护理杂志

Objective@#To construct evaluation index system of training effectiveness for new recruiting nurses based on the theory of Kirkpatrick Model.@*Methods@#The draft of evaluation index system of training effectiveness for new recruiting nurses was drawn up through literature survey and interview method first, and then it was constructed by using Delphi method and statistical analysis.@*Results@#Through two rounds of expert consultations, the experts′ suggestions were revised and summarized, and then the evaluation index system was constructed, including 4 grade I indexes, 14 grade Ⅱindexes, and 56 grade Ⅲ indexes. The positive coefficients of the two rounds of expert consultation were 96.15% and 100.00%, respectively, with authoritative coefficient of 0.877. The Kendall Harmony Coefficients of importance and operability of grade I, Ⅱ, and Ⅲ indexes were 0.640, 0.378, 0.296, and 0.707, 0.515, 0.416, respectively (P< 0.01).@*Conclusions@#The enthusiasm, authority level and coordination degree of the experts were all at a high level. The evaluation index system of training effectiveness for new recruiting nurses was of high importance and operability.

Application effect of mixed reality in the teaching of hepatobiliary surgery / 中华医学教育探索杂志

Objective@#To explore the application effect of mixed reality technology in the teaching of hepatobiliary surgery.@*Methods@#A total of 120 professional postgraduates majored in surgery who had received the four-month standardized training of residents in hepatobiliary surgery department of the First Affiliated Hospital of Army Military Medical University and the First Affiliated Hospital of Chongqing Medical University during 2016-2018 were selected and were equally allocated into the control group and the research group via random number table method. Students in the control group were trained with traditional surgical teaching mode and in the research group were trained with three-dimensional visualization model which was constructed by mixed reality. Students in both groups received assessment and questionnaires at the end of training.@*Results@#Scores of theoretical examination and surgical operation assessment of students in the research group were higher than those in the control group (P<0.05). The error rate for separating gallbladder artery and bile duct in the laparoscopic simulated system of students in the research group was significantly lower than that in the control group (P<0.05). Students-teacher satisfaction degree in the research group was higher than that in the control group (P<0.05).@*Conclusion@#Using mixed reality technology to construct a three-dimensional visualization model and to implement hepatobiliary surgery teaching enables the trainees to better master anatomical key points in the hepatobiliary system and the hepatobiliary surgical approach, with good feedback.

Application effect of mixed reality in the teaching of hepatobiliary surgery / 中华医学教育探索杂志

Objective To explore the application effect of mixed reality technology in the teaching of hepatobiliary surgery.Methods A total of 120 professional postgraduates majored in surgery who had received the four-month standardized training of residents in hepatobiliary surgery department of the First Affiliated Hospital of Army Military Medical University and the First Affiliated Hospital of Chongqing Medical University during 2016-2018 were selected and were equally allocated into the control group and the research group via random number table method.Students in the control group were trained with traditional surgical teaching mode and in the research group were trained with three-dimensional visualization model which was constructed by mixed reality.Students in both groups received assessment and questionnaires at the end of training.Results Scores of theoretical examination and surgical operation assessment of students in the research group were higher than those in the control group(P＜0.05).The error rate for separating gallbladder artery and bile duct in the laparoscopic simulated system of students in the research group was significantly lower than that in the control group(P＜0.05).Students-teacher satisfaction degree in the research group was higher than that in the control group(P＜0.05).Conclusion Using mixed reality technology to construct a three-dimensional visualization model and to implement hepatobiliary surgery teaching enables the trainees to better master anatomical key points in the hepatobiliary system and the hepatobiliary surgical approach,with good feedback.

Evaluation of effects of Traditional Chinese Medicine nursing training based on Kirkpatrick's model / 中华护理杂志

Objective To evaluate the effects of Traditional Chinese Medicine (TCM) nursing training based on Kirkpatrick's model relying on TCM hospitals.Methods Totally 77 clinical nurses from 50 medical institutions in Hangzhou were enrolled for TCM training.The four levels of Kirkpatrick's model including reaction,learning,behavior and results were applied for the evaluation together with questionnaires,paper-based exams,scene simulation method and semi-structured interviews.Results Nurses' attendance was over 85％,and average score of training satisfaction was 96.32±5.91 which lied between highly satisfied and very satisfied.TCM knowledge and skills were higher after training(P＜0.05).The scores of self-evaluation and peer evaluation were higher after training(P＜0.05).The numbers of projectsand personnel to practice TCM in the department or outpatient clinic were higher after training (P＜0.05).The patient satisfaction was over 82％.The themes of training benefits among nurses were reaching the expected values,growth and harvest,and expectation of continuous learning.Conclusion Using Kirkpatrick's model as the evaluation tool can evaluate the effects of TCM nursing training relying on TCM hospitals in a comprehensive and long-term manner,and improve the quality of training.

The quality analysis of the merged culture of the training for professional master degree students of clinical medicine and resident standardization training in Zhejiang Province / 中华医学教育探索杂志

Objective To improve the training system of the combined education mode between professional master degree students of clinical medicine and resident standardization training (RST) in Zhe-jiang province, and improve the professional quality of medical graduate students. Methods A question-naire was designed through literature review and expert interview, and the data of 77 clinical training post-graduates were collected, and the effect of the training was evaluated by taking the reaction level, learning level, behavior level and achievement level of Ke's evaluation as the breakthrough point. Statistical analysis was performed by SPSS 20.0, and the mean score was used as the standard deviation. The scores were compared with t test, and the scores of multiple groups were made variance analysis. Results The RST attitude value in reaction layer was 9.26±1.08, RST content value was 29.29±4.36, RST teacher enthusiasm value was 11.13±2.17, RST experience value was 17.38±3.10, which indicated that residents' overall satisfaction to RST was high, but the satisfaction on teachers, evaluation and compensation was relatively low;The grade difference before and after RST in learning layer was statistically significant (P<0.05), indicating that before and after RST the grades were obviously improved, and the sense of belonging is high. The behave layer showed the residents' behavior changed obviously before and after RST (P<0.05), and their abilities in various aspects such as ward round and dealing with common diseases were largely improved;The result layer showed 77 graduate students had no medical accident and complaint , and all of them passed the annual assessment and participated in the research projects. Conclusion In general, RST graduate students are relatively satisfied with the plan, content and methods of the training, on the other hand, there are problems such as the absence of timely information feedback, teachers' poor enthusiasm, lack of effective competition mechanism, etc. Kirkpatrick Model made a comprehensive and objective eval-uation on graduate students RST from a overall perspective, and it can be introduced into the effectiveness evaluation for graduate students RST.

Research on training effect of the standardized training program for resident physicians by using Kirkpatrick's model / 中华医学教育探索杂志

Objective To evaluate the effect of the standardized training program for resident physicians in Ruijin Hospital,and analyse the problems so as to provide reference to improve the training.Methods Questionnaire surveys were conducted among 113 resident physicians,31 teachers and 43 head nurses in Rujin Hospital by using the simple random sampling and 300.The data was analysed by Kirkpatrick's model in four layers including reaction layer,learning layer,behaviors layer and results layer.Data of reation layer was analysed by ANOVA and data of behaviors layer was compared by paired t-test.Results Reaction layer:the resident physicians' overall satisfaction score for the training is 3.45.Learning layer:all resident doctors participating in the training passed all the exams organised by the hospital.Behaviors layer:Residents made a great progress in many aspects after the standardized training program,and the difference was statistically significant (P＜0.01) according to the analysis of all questionnaires written by residents,clinical teachers and head nurses.Results layer:both the patient complaint rate and the accident rate of the 113 resident physicians was 0 while they all passed the National Medical Licensing Examination and the employment rate was 100％.Conclusions the standardized training program for resident physicians in Ruijin Hospital gets fairly good effect.Resident doctors' quality and ability in many aspects are improved and the overall satisfaction for the training is high,but in salary,benefits and sense of belonging to the hospital,the satisfaction is relatively low.

Effects of Therapist Training:Evaluated with Kirkpatrick's Model / 中国康复理论与实践

Objective To evaluate the effect of training for therapists based on Kirkpatrick's Four-level Model. Methods Totally 39 ther-apists who completed 2014 Beijing Clinical Shortage of Manpower Training Programs (Therapist) were investigated with Kirkpatrick's Four-level Model, named reaction, learning, behavior and results. Results For reaction level, the total satisfaction was high, but different with the degree of education, especially in the items of Participate in Training, Type of Cases, Organizing on Practice Training, Organizing Case Discussions and Get A Lot of Benefits from the Training (F>4.59, P9.53, P4.44, P<0.001). For results level, eleven therapists passed the rehabilitation therapist test, two therapists participated in some researches, and one published an article. Conclusion The thera-pists satisfied in the training, that may improve the ability of therapists to contribute to their organization.

Evaluation of a continuing professional development training program for physicians and physician assistants in hospitals in Laos based on the Kirkpatrick model / 보건의료교육평가

PURPOSE: Medical professionals from Korea and Laos have been working together to develop a continuing professional development training program covering the major clinical fields of primary care. This study aimed to evaluate the effectiveness of the program from 2013 to 2014 using the Kirkpatrick model. METHODS: A questionnaire was used to evaluate the reaction of the trainees, and the trainers assessed the level of trainees' performance at the beginning and the end of each clinical section. The transfer (behavioral change) of the trainees was evaluated through the review of medical records written by the trainees before and after the training program. RESULTS: The trainees were satisfied with the training program, for which the average score was 4.48 out of 5.0. The average score of the trainees' performance at the beginning was 2.39 out of 5.0, and rose to 3.88 at the end of each section. The average score of the medical records written before the training was 2.92 out of 5.0, and it rose to 3.34 after the training. The number of patient visits to the district hospitals increased. CONCLUSION: The continuing professional development training program, which was planned and implemented with the full engagement and responsibility of Lao health professionals, proved to be effective.

Determination of Satisfaction Index as a tool in evaluation of CME Program.

Continuing Medical Education is an indispensable part of physician's learning. Well designed program based on andragogy principles can enhance learning by motivating the learner and providing platform to encourage self directed learning. The present study aimed to explore the impact of program “NAMS-AIIMS Regional Symposium on Sleep Medicine” in changing the behavior and attitude of participants using “Satisfaction Index” and descriptive analysis of responses as evaluation tools for program effectiveness. This descriptive cross sectional study captured the response of participants through a pre-tested and validated questionnaire administered at the end of symposium. The result showed almost equal sex distribution (M: F- 27: 34) with majority being UG students (86%). Reliability of data showed Cronbach's Alpha of 0.98 indicating high reliability. Satisfaction index (SI) calculated as per WHO Educational Handbook for Health Personnel showed highest satisfaction for conducive environment of symposium (87.87 %) followed by provision for time to seek clarifications (87.21%), provision of appropriate Learning Resource material (85.90 %) and handling of critical comments by organizers (85.57%). Descriptive analysis showed majority responses as highly positive to our questionnaire with suggestions for more such activity, inclusion of clinical cases and other aspects of practical relevance.