A guide to outcome evaluation of simulation-based education programmes in low and middle-income countries.

Evaluation is a vital part of any learning activity and is essential to optimize and improve educational programmes. It should be considered and prioritized prior to the implementation of any learning activity. However, comprehensive programme evaluation is rarely conducted, and there are numerous barriers to high-quality evaluation. This review provides a framework for conducting outcome evaluation of simulation-based education programmes in low and middle-income countries (LMICs). The basis of evaluation, including core ideas of theory, purpose and structure are outlined, followed by an examination of the levels and healthcare applications of the Kirkpatrick model of evaluation. Then, methods of conducting evaluation of simulation-based education in LMICs are discussed through the lens of a successful surgical simulation programme in Myanmar, a lower-middle-income country. The programme involved the evaluation of 11 courses over 4 years in Myanmar and demonstrated evaluation at the highest level of the Kirkpatrick model. Reviewing this programme provides a bridge between evaluation theory and practical implementation. A range of evaluation methods are outlined, including surveys, interviews, and clinical outcome measurement. The importance of a mixed-methods approach, enabling triangulation of quantitative and qualitative analysis, is highlighted, as are methods of analysing data, including statistical and thematic analysis. Finally, issues and challenges of conducting evaluation are considered, as well as strategies to overcome these barriers. Ultimately, this review informs readers about evaluation theory and methods, grounded in a practical application, to enable other educators in low-resource settings to evaluate their own activities.

A Bi-national Needs Assessment to Identify and Prioritise Procedures in Paediatric Surgery for Simulation-based Training.

BACKGROUND: Simulation-based education (SBE) has led to significant changes in healthcare education. However, SBE has often been based on available resources and local expertise rather than a systematic approach to curriculum development. The aim of this study was to perform a bi-national needs assessment to identify and prioritise procedures in a paediatric surgery curriculum that can be supported using SBE. METHOD: A modified 3-round Delphi technique was used to gather consensus from education leaders and trainees in paediatric surgery in Australia and Aotearoa New Zealand (ANZ). Round 1 identified all procedures a newly specialised paediatric surgeon should be able to perform. In Round 2, each procedure was explored for the need for SBE using the Copenhagen Academy for Medical Education and Simulation (CAMES) Needs-Assessment Formula (NAF). This pre-prioritised list from Round 2 was sent back to participants for final exclusion and ranking in Round 3. Results 88 participants were identified and invited. From 174 procedures identified in Round 1, 71 procedures were grouped and categorised for Round 2 using the CAMES NAF. In Round 3, 17 procedures were eliminated resulting in 54 procedures. Appendicectomy, inguinal herniotomy, and central venous access were the highest rank procedures after prioritisation in Round 3. There was a strong correlation (r = 0.99) between the NAF score and the prioritised ranking, as well as between consultants and trainees (r = 0.92 in Round 2 and 0.98 in Round 3). CONCLUSION: The prioritised list represents a consensus document decided upon by education leaders and stakeholders in paediatric surgery. These procedures should be an integral part of the SBE of paediatric surgeons in the region. LEVEL OF EVIDENCE: Level V.

Management of thoracic empyema in children: a survey of the Australia and New Zealand Association of Paediatric Surgeons (ANZAPS).

PURPOSE: To define the spectrum of management for thoracic empyema in children in Australia and New Zealand. METHODS: Online survey of members of the Australian and New Zealand Association of Paediatric Surgeons (ANZAPS), limited to consultant/attending paediatric surgeons. RESULTS: A total of 54/80 (67.5%) members, from 16 paediatric surgical centres, responded. The majority (33/54, 61%) preferred chest drain with fibrinolytics, whilst 21/54 (39%) preferred video-assisted thoracoscopic surgery (VATS) with drain insertion. Urokinase was the most commonly used fibrinolytic (64%). There were no significant differences in management preferences between practising surgeons in Australia and New Zealand (p = 0.54), nor between consultants who had been practising a shorter (< 5 years) or longer (> 20 years) amount of time (p = 0.21). The practices described by the surveyed ANZAPS members were in line with the Thoracic Society of Australia and New Zealand recommendations for the management of paediatric empyema. CONCLUSION: Across Australia and New Zealand there exists significant variation surrounding the intra- and post-intervention management of thoracic empyema in children. The surveyed paediatric surgeons demonstrated a preference for fibrinolytics over the use of VATS. All management regimens were within published local guidelines.

Overview of a novel paediatric surgical simulation-based medical education programme in Myanmar.

BACKGROUND: Simulation-based medical education (SBME) is an integral part of undergraduate and postgraduate training in high-income countries (HICs). Despite potential benefits to low- and middle-income countries (LMICs), it has not been widely applied. Our aim was to use SBME to address some essential paediatric surgery learning needs in a LMIC. METHODS: Eleven SBME courses were designed, implemented and evaluated over a 4-year period in partnership with local paediatric surgeons and the University of Medicine 1 in Yangon, Myanmar. All courses were simulation-based and different major SBME modalities were utilized. Evaluation included pre- and post-course questionnaires, other evaluation assessments including Likert scale self-rated confidence in different domains, as well as, mixed method evaluation and Kirkpatrick's hierarchy of evaluation. RESULTS: Over 4 years, a multidisciplinary team consisting of surgical consultants, fellows, and educational specialists delivered 11 courses at a tertiary LMIC paediatric surgical centre. Attendance varied between 23 and 50 healthcare professionals, with some participants attending all of the educational activities. SBME modalities were utilized to meet each courses' learning objectives. All educational courses scored highly and showed statistically significant differences in all the self-rated pre and post-course confidence Likert scale domains. SBME was accepted and embraced by local participants and faculty, and transition to local delivery of educational content has begun. Level 4 of Kirkpatrick's hierarchy of evaluation was demonstrated. CONCLUSION: SBME can be used to meet essential learning objectives of local staff in a LMIC. Through various modalities, it offers a reliable, proven and affordable means of teaching multiple aspects of paediatric surgical clinical practice. By employing innovative simulation-based solutions, it can be adapted by local faculty to continue meeting ongoing learning needs.

Clinical Impact of the Introduction of Pediatric Intussusception Air Enema Reduction Technology in a Low- to Middle-Income Country Using Low-Cost Simulation-Based Medical Education.

INTRODUCTION: Pediatric intussusception is a common cause of bowel obstruction in infants. Air enema (AE) reduction is routine first-line management in many countries; however, there is a high rate of operative intervention in low- and middle-income countries. The aims of the study were to use simulation-based medical education with an intussusception simulator to introduce AE reduction to Myanmar and to assess its effect on provider behaviors and the resulting clinical care. METHODS: Clinical evaluation was conducted by comparing clinical outcomes data for children with intussusception 12 months before implementation with that from 12 months subsequent to implementation. These included the following: AE success rates, recurrence rates, length of stay, intestinal resection, and operative intervention rates. An educational workshop was developed that used a low-cost mannequin to facilitate practice at the reduction of intussusception using AE. Curriculum evaluation was performed through 5-point rating scale self-assessment in several domains. Data analysis was performed with Mann-Whitney U test, Student t test, or Wilcoxon signed-ranks test as appropriate; a P value of less than 0.05 was considered to be significant. RESULTS: After implementation, there was a significant reduction in the overall operative intervention rates [82.5% (85/103) vs. 58.7% (44/75), P = 0.006]. Intestinal resection rates increased [15.3% (13/85) vs. 35.9% (14/39), P = 0.02]. The success rate with attempted AE reduction was 94.4% (34/36), with a recurrence rate of 5.6% (2/36). The simulation-based medical education workshop was completed by 25 local participants. There was a significant difference in the confidence of performing (1.9 vs. 3.6, P ≤ 0.0001) or assisting (2.8 vs. 3.7, P = 0.018) an AE reduction before and after the workshop. CONCLUSIONS: Simulation-based educational techniques can be successfully applied in a low- and middle-income country to facilitate the safe introduction of new equipment and techniques with significant beneficial impact on provider behaviors and the resulting clinical care.

Evaluation of a novel low-cost laparoscopic training model for core laparoscopic skills.

The study involves the testing of a novel laparoscopic training model by surgeons of various levels of experience. BACKGROUND: There has been an increasing requirement for low-cost simulation. Our aim was to evaluate the construct validity of a low-cost model for teaching core laparoscopic skills. METHODS: The Double-Glove model was made from 2 latex gloves, one placed in the other. The inner glove was filled with water and 3 ovals were drawn on the outer glove. Participants were required to dissect the middle oval out without perforating the inner glove or leaving the line border of the middle oval. The task was assessed using a previously validated scoring system (minimum -120; maximum 80). RESULTS: Ninety-five participants completed the task: 40 novices, 45 intermediates, and 10 experts. The model revealed statistical significance between the three groups. Experts scored higher than novices (58/80 vs 11.7/80; p < 0.0001) and intermediates (58/80 vs 29.1/80; p = 0.0004), and intermediates scored higher than novices (29.1/80 vs 11.7/80; p = 0.014). Novices took more time to complete the task compared to intermediates (10 min vs 7.87 min; p < 0.0001) and experts (10 min vs 6.98 min; p < 0.0001). No correlation between time taken and score obtained was seen (r = -0.06, r = 0.01, r = -0.2 for novice, intermediate, and expert groups). CONCLUSION: By differentiating between groups of variable experience, the model demonstrated construct validity. It offers an inexpensive model that can be utilized in low-cost laparoscopic simulation. TYPE OF STUDY: Study of a diagnostic test. LEVEL OF EVIDENCE: II.

Surgical skill levels: Classification and analysis using deep neural network model and motion signals.

BACKGROUND AND OBJECTIVES: Currently, the assessment of surgical skills relies primarily on the observations of expert surgeons. This may be time-consuming, non-scalable, inconsistent and subjective. Therefore, an automated system that can objectively identify the actual skills level of a junior trainee is highly desirable. This study aims to design an automated surgical skills evaluation system. METHODS: We propose to use a deep neural network model that can analyze raw surgical motion data with minimal preprocessing. A platform with inertial measurement unit sensors was developed and participants with different levels of surgical experience were recruited to perform core open surgical skills tasks. JIGSAWS a publicly available robot based surgical training dataset was used to evaluate the generalization of our deep network model. 15 participants (4 experts, 4 intermediates and 7 novices) were recruited into the study. RESULTS: The proposed deep model achieved an accuracy of 98.2%. With comparison to JIGSAWS; our method outperformed some existing approaches with an accuracy of 98.4%, 98.4% and 94.7% for suturing, needle-passing, and knot-tying, respectively. The experimental results demonstrated the applicability of this method in both open surgery and robot-assisted minimally invasive surgery. CONCLUSIONS: This study demonstrated the potential ability of the proposed deep network model to learn the discriminative features between different surgical skills levels.

Circumcision in the paediatric patient: A review of indications, technique and complications.

Circumcision is one of the most commonly performed surgical procedures in the world. Despite this, the practice of paediatric circumcision remains highly controversial, and continues to generate ongoing debate. This debate has become more relevant recently with the provisional guidelines from the Centers for Disease Control and Prevention recommending a change of practice. In this review article, we provide an overview of the history and incidence of circumcision, normal preputial development, types of phimosis, the absolute and relative indications for circumcision as well as the evidence base for its use as a preventative measure. Our aim is to provide paediatricians with a greater understanding of this common surgical procedure and the conditions it treats, to guide their clinical practice and parent counselling.

The laparoscopic inguinal and diaphragmatic defect (LIDD) model: a validation study of a novel box trainer model.

BACKGROUND: Paediatric laparoscopic procedures are now becoming routine practice. Therefore, there is a need for simulated laparoscopic models to acquire part-procedural competency prior to direct patient contact in a safe learning environment. For this reason, we chose two paediatric conditions; inguinal hernia (IH) and congenital diaphragmatic hernia (CDH), which were combined to create the laparoscopic inguinal and diaphragmatic defect (LIDD) model. Our aim was to assess this novel surgical simulation model by determining its construct and content validity. METHODS: A total of 107 participants completed the validation study: volunteer medical students (novices), surgical trainees (intermediate) and consultant surgeons (experts). Basic demographic data were collected. Subjects were shown a pre-recorded video of both exercises. The assessment exercise involved closing both the simulated inguinal or diaphragmatic hernial orifice. The task was assessed using a novel scoring system with a maximum score of 21 for IH model and 15 for the CDH. The content validity was assessed by a 6-point Likert scale of the expert group. RESULTS: 105/107 participants successfully completed the two exercises. Both aspects of the LIDD model revealed a statistical significance between the scores obtained by the three groups of subjects. Experts scored 20.3/21 for the IH and 14.8/15 for the CDH models which significantly higher than medical students (6.3/21 and 5.3/15; p < 0.05 for both) and trainees (11.2/21 and 9.3/15; p < 0.05 for both). Similarly, trainees performed significantly better than medical students in both models (p < 0.05). Therefore, the LIDD model was found to have a good construct validity. It was, however, unable to differentiate between the various levels of trainees in the intermediate group. Content validity from the experts revealed that there was a high score for the potential of both aspects of LIDD (4.8 and 4.8). There was also a high level of functional fidelity for task completion (4.0 and 4.0). CONCLUSIONS: We have demonstrated both the construct and content validity of the LIDD model for both laparoscopic IH and CDH repair. It was able to successfully differentiate between the expert, trainees and inexperienced laparoscopic surgeons.

Acute pancreatitis complicating choledochal cysts in children.

AIM: To analyse the characteristics of patients with choledochal cysts presenting with acute pancreatitis. METHODS: Multicenter retrospective review of all paediatric patients (<18 years) with choledochal cysts managed over a 14-year period (2001-2014) at two tertiary paediatric surgical centres. Patient data were analysed for demographics, presentation, radiological classification of cyst type (Todani), operative interventions, complications and long-term follow-up. RESULTS: A total of 49 patients with choledochal cysts were identified with 15 (31%) being Type I fusiform, 18 (37%) Type I cystic and 16 (32%) Type IV-A. Seventeen (35%) patients presented with acute pancreatitis, one having had an ante-natally diagnosed choledochal cyst. Patients presenting with pancreatitis were older when compared to the non-pancreatitis group (5.1 vs. 1.2 years, P = 0.005). Nine out of 16 (53%) patients with Type IV-A cysts presented with pancreatitis compared to five (33%) of Type I fusiform and three (17%) of Type I cystic. There was however no statistically significant association between Todani types and the development of pancreatitis (Type I fusiform, P = 1.0; Type I cystic, P = 0.063; Type IV-A, P = 0.053). The rate of complications was similar in both groups. CONCLUSION: Pancreatitis was a common presentation in children with a choledochal cyst, however, there was no clear statistically significant association with Todani types and pancreatitis.