If You Build It, Will They Come? Exploring the Impact of Medical Student Engagement on Clerkship Curriculum Outcomes.

Introduction: Inconsistent or superficial access to workplace learning experiences can impede medical students' development. Well-designed clerkship curricula provide comprehensive education by offering developmental opportunities in and out of the workplace, explicitly linked to competency objectives. Questions remain about how students engage with clerkship curriculum offerings and how this affects their achievement. This study investigated student engagement as the source of an apparent clerkship curriculum malfunction: increasing rate of substandard summative clinical competency exam (SCCX) performance over 3 years following curriculum reform. Materials and Methods: We sampled from three cohorts of US medical students (classes of 2018-2020) based on their post-clerkship SCCX performance: substandard (N = 33) vs. exemplary (N = 31). Using a conceptually based, locally developed rubric, a five-person team rated students' engagement in a curriculum offering designed to provide standardized deliberate practice on the clerkship's competency objectives. We examined the association between engagement and SCCX performance, taking prior academic performance into account. Results: Rate of substandard SCCX performance could not be explained by cohort differences in prior academic performance. Student engagement differed across cohorts and was significantly associated with SCCX performance. However, engagement did not meaningfully predict individual students' SCCX performance, particularly in light of prior academic performance. Discussion: Engagement with a particular learning opportunity may not affect clerkship outcomes, but may reflect students' priorities when navigating curricular offerings, personal learning goals, and curriculum policy. Proposing four patterns of engagement in clerkship learning, this study prompts reflection on the complex interaction among factors that affect engagement and outcomes.

Attendings' Perceptions of Authentic Evaluation Criteria for Effective Surgical Consults.

OBJECTIVE: The authors aimed to investigate faculty evaluation criteria for an effective oral surgical presentation in actual patient care contexts. DESIGN: We conducted a 2-step observation-based qualitative study. Residents audiotaped oral presentations of a surgical consult to an attending. Evaluation panels listened to the recordings and discussed to develop joint feedback for the resident. The panel discussions were recorded and served as the data source for this study. We analyzed the data following the grounded theory approach using open coding and axial coding. SETTING: The study setting was at Southern Illinois University School of Medicine, a 5-year general surgery residency program in Springfield, Illinois. PARTICIPANTS: Thirteen residents out of 19 in the program participated by virtue of having submitted recordings of a patient care consult presentation via phone. Evaluation panels consisted of general surgery academic and community faculty, as well as senior residents. RESULTS: Several criteria for effective oral presentations emerged that have rarely been discussed in prior literature. Themes included: (1) The strategic opening is critical as it "sets the stage" and frames how the attending will listen. Situational factors, such as consideration of time of the day and urgency, should be accounted for in the opening. (2) A deductive structure defines the relevance of the presented information. Clinical judgement should precede supporting evidence. Attending physicians perceive important information as unnecessary if provided outside of this framework. (3) Established trust between a resident and a surgeon determines the level of detail expected of the presenting resident. With increasing trust, surgeons expect residents to present fewer details; if too much detail is included, the presentation may be assessed as ineffective. (4) Surgical descriptions are appreciated for their value in promoting the attending's visualization or mental picture of the patient condition. (5) Oral emphasis using voice tone and pace can be helpful for capturing attending attention. CONCLUSIONS: These findings can be utilized to improve the current training program and assessment rubrics toward contextualized work-based assessment practices in surgery. Oral patient presentation skills are neither static nor universal, but fluid and reflexive, based on trust, and situational factors.

Mock Pages Are a Valid Construct for Assessment of Clinical Decision Making and Interprofessional Communication.

IMPORTANCE: Answering pages from nurses about patients in need of immediate attention is one of the most difficult challenges a resident faces during their first days as a physician. A Mock Page program has been developed and adopted into a national surgical resident preparatory curriculum to prepare senior medical students for this important skill. OBJECTIVE: The purpose of this study is to assess standardized mock page cases as a valid construct to assess clinical decision making and interprofessional communication skills. DESIGN, SETTING, PARTICIPANTS: Mock page cases (n = 16) were administered to 213 senior medical students from 12 medical schools participating in a national surgical resident preparatory curriculum in 2013 and 2014. MAIN OUTCOME MEASURES: Clinical decision making and interprofessional communication were measured by case-specific assessments evaluating these skills which have undergone rigorous standard-setting to determine pass/fail cut points. RESULTS: Students' performance improved in general for both communication and clinical decision making over the 4-week course. Cases have been identified that seem to be best suited for differentiating high- from low-performing students. Chest pain, pulmonary embolus, and mental status change cases posed the greatest difficulty for student learners. CONCLUSIONS AND RELEVANCE: Simulated mock pages demonstrate an innovative technique for training students in both effective interprofessional communication and management of common postoperative conditions they will encounter as new surgical interns.

How prepared are your interns to take calls? Results of a multi-institutional study of simulated pages to prepare medical students for surgery internship.

BACKGROUND: This study evaluated a simulated pages curriculum that was developed to assess communication and clinical decision making in medical students and interns. METHODS: A curriculum consisting of 14 simulated pages was administered across 5 institutions to 150 senior medical students. A 3-case subset was administered to interns who did not participate in the curriculum. Six expert surgeons identified critical fails and set passing scores for case-specific assessments using the Graphical Hofstee Method. RESULTS: Participants in the curriculum demonstrated superior clinical decision making compared with non-participants across all cases scenarios (P < .01). Average medical student scores for clinical decision making were 46.9%. Global ratings averaged 6.0 for communication and 5.2 for patient care. Passing rates averaged 46%. CONCLUSIONS: Participation in a mock page curriculum improved performance. The performance of participants based on expert standards set for simulated page performance highlight the need for innovative approaches to improve interns' preparedness to take calls.

The impact of brief team communication, leadership and team behavior training on ad hoc team performance in trauma care settings.

BACKGROUND: Communication breakdowns and care coordination problems often cause preventable adverse patient care events, which can be especially acute in the trauma setting, in which ad hoc teams have little time for advanced planning. Existing teamwork curricula do not address the particular issues associated with ad hoc emergency teams providing trauma care. METHODS: Ad hoc trauma teams completed a preinstruction simulated trauma encounter and were provided with instruction on appropriate team behaviors and team communication. Teams completed a postinstruction simulated trauma encounter immediately afterward and 3 weeks later, then completed a questionnaire. Blinded raters rated videotapes of the simulations. RESULTS: Participants expressed high levels of satisfaction and intent to change practice after the intervention. Participants changed teamwork and communication behavior on the posttest, and changes were sustained after a 3-week interval, though there was some loss of retention. CONCLUSIONS: Brief training exercises can change teamwork and communication behaviors on ad hoc trauma teams.

The effect of model fidelity on colonoscopic skills acquisition. A randomized controlled study.

INTRODUCTION: Colonoscopic simulators offer the opportunity for skill acquisition in the preclinical setting. Currently available simulators vary widely with respect to level of fidelity and technological sophistication. Despite the belief that more realistic is better, there is a paucity of evidence regarding the relative effectiveness of simulator fidelity (high vs low) on the acquisition of basic colonoscopic skills. We hypothesized that novice learners can acquire basic colonoscopic skills using simulators, however fidelity of the simulator does not make a difference. METHODS: We randomly assigned novice third-year and fourth-year medical students to practice on either a low-fidelity or high-fidelity colonoscopy model. The low-fidelity model used is described in the module 16 of the American College of Surgeons/Association of Program Directors in Surgery surgical skills curriculum for residents, Phase 1: basic or core skills and tasks < http://elearning.facs.org/mod/resource/view.php?1d=450 >. The high-fidelity model was the AccuTouch colonoscopy simulator, Immersion Medical (AccuTouch CS) that has 6 different simulated scenarios for diagnostic colonoscopy (level 1-6). Both groups had 16 students and were given standard instruction by an expert with respect to the procedure and instrument handling on both models. Both groups were pretested and posttested on level 1 of the AccuTouch CS. The high-fidelity group practiced on level 2 and 4 of the AccuTouch CS, whereas the low-fidelity group practiced on the low-fidelity model for 2 sessions of 1 hour each. The computer-based evaluation parameters available on the AccuTouch CS were used to compare performances. RESULTS: Both groups had similar demographics. There were no significant differences in the baseline performances of either group. Each group demonstrated significant improvement for insertion time and percentage of mucosa visualized. However, there were no significant differences between the groups on posttesting on any of the measured parameters. CONCLUSIONS: Colonoscopic skill training on a low-fidelity model appears to be as effective as high-fidelity model training for basic endoscopic skill acquisition for novice learners.

The case of the entitled resident: a composite case study of a resident performance problem syndrome with interdisciplinary commentary.

BACKGROUND: Residents with performance problems create substantial burden on programs and institutions. Understanding the nature and quality of performance problems can help in learning to address performance problems. AIM: We sought to illuminate the effects of resident performance problems and the potential solutions for those problems from the perspectives of people with various roles in health care. METHODS: We created a composite portrait from several residents who demonstrated a cluster of common performance characteristics and whose chronic or serious maladaptive behavior and response to situations created problems for themselves, for their clinical colleagues, and for faculty of their residency program. The composite was derived from in-depth interviews of program directors and review of resident records. We solicited practitioners from multiple fields to respond to the portrait by answering a series of questions about severity, prognosis, and how and whether one could reliably remediate a person with these performance characteristics. We present their perspectives in a manner borrowed from the New England Journal of Medicine's "Case Records of the Massachusetts General Hospital." RESULTS: We created a composite portrait of a resident whose behavior suggested he felt entitled to benefits his peers were not entitled to. Experts reflecting on his behavior varied in their opinion about the effect the resident would have on the health care system. They suggested approaches to remediation that required substantial time and effort from the faculty. CONCLUSION: Programs must balance the needs of individual residents to adjust their behaviors with the needs of the health care system and other people within it.

A surgical simulation curriculum for senior medical students based on TeamSTEPPS.

OBJECTIVE: To investigate whether the existing Team Strategies and Tools to Enhance Performance and Patient Safety (TeamSTEPPS) curriculum can effectively teach senior medical students team skills. DESIGN Single-group preintervention and postintervention study. SETTING AND INTERVENTION: We integrated a TeamSTEPPS module into our existing resident readiness elective. The curriculum included interactive didactic sessions, discussion groups, role-plays, and videotaped immersive simulation scenarios. MAIN OUTCOME MEASURES: Improvement of self-assessment scores, multiple-choice examination scores, and performance ratings of videotaped simulation scenarios before and after intervention. The videos were rated by masked reviewers on the basis of a global rating instrument (TeamSTEPPS) and a more detailed nontechnical skills evaluation tool(NOTECHS). PARTICIPANTS: Seventeen students participated and completed the study. RESULTS: The self-evaluation scores improved from 12.76 to 16.06 (P < .001). The increase was significant for all of the TeamSTEPPS competencies and highest for leadership skills (from 2.2 to 3.2; P < .001). The multiple-choice score rose from 84.9% to 94.1% (P < .01). The postintervention video ratings were significantly higher for both instruments (TeamSTEPPS, from 2.99 to 3.56; P < .01; and NOTECHS, from 4.07 to 4.59; P < .001). CONCLUSIONS: The curriculum led to improved self-evaluation and multiple-choice scores as well as improved team skills during simulated immersive patient encounters. The TeamSTEPPS framework may be suitable for teaching medical students teamwork concepts and improving their competencies. Larger studies using this framework should be considered to further evaluate the generalizability of our results and the effectiveness of TeamSTEPPS for medical students.

Pursuing professional accountability: an evidence-based approach to addressing residents with behavioral problems.

OBJECTIVE: To develop an evidence-based approach to the identification, prevention, and management of surgical residents with behavioral problems. DESIGN: The American College of Surgeons and Southern Illinois University Department of Surgery hosted a 1-day think tank to develop strategies for early identification of problem residents and appropriate interventions. Participants read a selection of relevant literature before the meeting and reviewed case reports. SETTING: American College of Surgeons headquarters, Chicago, Illinois. PARTICIPANTS: Medical and nursing leaders in the field of resident education; individuals with expertise in dealing with academic law, mental health issues, learning deficiencies, and disruptive physicians; and surgical residents. MAIN OUTCOME MEASURES: Evidence-based strategies for the identification, prevention, and management of problem residents. RESULTS: Recommendations based on the literature and expert opinions have been made for the identification, remediation, and reassessment of problem residents. CONCLUSIONS: It is essential to set clear expectations for professional behavior with faculty and residents. A notice of deficiency should define the expected acceptable behavior, timeline for improvement, and consequences for noncompliance. Faculty should note and address systems problems that unintentionally reinforce and thus enable unprofessional behavior. Complaints, particularly by new residents, should be investigated and addressed promptly through a process that is transparent, fair, and reasonable. The importance of early intervention is emphasized.

Engaging medical students in the feedback process.

BACKGROUND: There are potential advantages to engaging medical students in the feedback process, but efforts to do so have yielded mixed results. The purpose of this study was to evaluate a student-focused feedback instructional session in an experimental setting. METHODS: Medical students were assigned randomly to either the intervention or control groups and then assigned randomly to receive either feedback or compliments. Tests of knowledge, skills, and attitudes were given before and after the intervention. RESULTS: There was a significant gain of knowledge and skill in the group that received instruction. Satisfaction was higher after compliments in the control group but higher after feedback in the instructional group. There was no change in the subject's willingness to seek feedback. CONCLUSIONS: A student-focused component should be carefully included as part of an overall effort to improve feedback in surgical education. The role of medical student attitudes about feedback requires further investigation.

Knowledge and attitudes regarding colorectal cancer screening among medical students: a tale of two schools.

Colorectal cancer (CRC) screening has been shown to decrease the incidence of CRC cancers and decrease mortality. Studies show that the most important predictor of patient compliance with CRC screening is physician recommendation. We assessed the knowledge and attitudes of medical students regarding cancer screening. A study-specific questionnaire was distributed to medical students (MS) at two medical schools. There was a significant difference in the percentage of correctly answered questions regarding screening recommendations between first year MS and all other years for both schools. However, MS attitudes towards CRC screening were consistent between classes and schools. Although most MS had positive attitudes regarding cancer screening our survey identified several important deficits in knowledge.

Use of simulated pages to prepare medical students for internship and improve patient safety.

PURPOSE: During the transition from medical school to internship, trainees experience high levels of stress related to pages on the inpatient wards. The steep learning curve during this period may also affect patient safety. The authors piloted the use of simulated pages to improve medical student preparedness, decrease stress related to pages, and familiarize medical students with common patient problems. METHOD: A multidisciplinary team at Southern Illinois University School of Medicine developed simulated pages that were tested among senior medical students. Sixteen medical students were presented with 11 common patient scenarios. Data on assessment, management, and global performance were collected. Mean confidence levels were evaluated pre- and postintervention. Students were also surveyed on how the simulated pages program influenced their perceived comfort in managing patient care needs and the usefulness of the exercise in preparing them to handle inpatient pages. RESULTS: Mean scores on the assessment and management portions of the scenarios varied widely depending on the scenario (range -15.6 ± 41.6 to 95.7 ± 9.5). Pass rates based on global performance ranged from 12% to 93%. Interrater agreement was high (mean kappa = 0.88). Students' confidence ratings on a six-point scale increased from 1.87 preintervention to 3.53 postintervention (P < .0001). CONCLUSIONS: Simulated pages engage medical students and may foster medical student preparedness for internship. Students valued the opportunity to simulate "on call" responsibilities, and exposure to simulated pages significantly increased their confidence levels. Further studies are needed to determine effects on patient safety outcomes.

Robot-assisted laparoscopic skills development: formal versus informal training.

INTRODUCTION: The learning curve for robotic surgery is not completely defined, and ideal training components have not yet been identified. We attempted to determine whether skill development would be accelerated with formal, organized instruction in robotic surgical techniques versus informal practice alone. MATERIALS AND METHODS: Forty-three medical students naive to robotic surgery were randomized into two groups and tested on three tasks using the robotic platform. Between the testing sessions, the students were given equally timed practice sessions. The formal training group participated in an organized, formal training session with instruction from an attending robotic surgeon, whereas the informal training group participated in an equally timed unstructured practice session with the robot. The results were compared based on technical score and time to completion of each task. RESULTS: There was no difference between groups in prepractice testing for any task. In postpractice testing, there was no difference between groups for the ring transfer tasks. However, for the suture placement and knot-tying task, the technical score of the formal training group was significantly better than that of the informal training group (p < 0.001), yet time to completion was not different. CONCLUSION: Although formal training may not be necessary for basic skills, formal instruction for more advanced skills, such as suture placement and knot tying, is important in developing skills needed for effective robotic surgery. These findings may be important in formulating potential skills labs or training courses for robotic surgery.

The nature of general surgery resident performance problems.

BACKGROUND: Residents with performance problems are challenging to program directors and complicate the work of other residents and health care providers. Having an effective, targeted remediation process to address these problems is dependent on being able to diagnose their nature. The purpose of this study was to identify residents who had serious, substantive, and recurring academic, clinical, and/or professional behavior problems, and to describe and classify their nature. METHODS: Raters performed a retrospective record review of general surgery categorical residents in one program over 30 years. Residents with substantial problems were reviewed, described, and classified independently by 3 raters. RESULTS: Seventeen residents had serious enough performance problems to be included. Four had only 1 class of problem (3 were professional behavior problems). Eight residents had 2 problems each (5 academic and clinical performance problems; 3 clinical performance and professional behavior problems). Five residents had 3 problems. The number of performance problem facets per case varied from 11 to 2 with an average of 5.9 facets. Relations with health care workers was identified most frequently, followed closely by insufficient knowledge. Poor communication was third. Performance problems of 14 residents (82%) were identified in their first year. For 15 cases, the resident had unresolved performance problems at the end of the program. CONCLUSION: There are 2 possible explanations for our findings: (1) resident problems similar to these are refractory to remediation; (2) treatments used historically are not well designed for the problems. Choosing among the 2 explanations will require developing remediation strategies targeted to specific patterns of performance problems.

A theory-based curriculum for enhancing surgical skillfulness.

BACKGROUND: Curricula for surgical technical skills laboratories have traditionally been designed to accommodate the clinical activities of residents, so they typically consist of individual, episodic training sessions. We believe that the skills laboratory offers an opportunity to design a surgical skills curriculum based on the fundamental elements known to be important for motor skill instruction. We hypothesized that training novices with such a curriculum for a 1-month period would yield skills performance levels equivalent to those of second year surgery residents who had trained in a traditional program. STUDY DESIGN: Fourth-year medical students served as study subjects (novice group) during a 4-week senior elective. They were taught each skill during a 1-week period. Subjects received instruction by a content expert followed by a 1-week period of deliberate practice with feedback. The novice performances were videotaped both before and after the intervention, and each videotape was evaluated in a blinded fashion by experts using a validated evaluation instrument. These results were compared with skill performance ratings of first- and second-year surgery residents that had been accumulated over the previous 3 years. RESULTS: Average performance ratings for the novices substantially improved for all four skills after training. There was no marked difference between average performance ratings of postintervention novice scores when compared with the average scores in the resident group. Inter-rater agreement in scoring for the videotaped novice performances exceeded 0.87 (intraclass correlation) for all ratings of pre- and posttraining. CONCLUSIONS: These results demonstrate the effectiveness of a laboratory-based training program that includes fundamentals of motor skills acquisition.

Surgeon information transfer and communication: factors affecting quality and efficiency of inpatient care.

OBJECTIVE: To determine the nature of surgeon information transfer and communication (ITC) errors that lead to adverse events and near misses. To recommend strategies for minimizing or preventing these errors. SUMMARY BACKGROUND DATA: Surgical hospital practice is changing from a single provider to a team-based approach. This has put a premium on effective ITC. The Information Transfer and Communication Practices (ITCP) Project is a multi-institutional effort to: 1) better understand surgeon ITCP and their patient care consequences, 2) determine what has been done to improve ITCP in other professions, and 3) recommend ways to improve these practices among surgeons. METHODS: Separate, semi-structured focus group sessions were conducted with surgical residents (n = 59), general surgery attending physicians (n = 36), and surgical nurses (n = 42) at 5 medical centers. Case descriptions and general comments were classified by the nature of ITC lapses and their effects on patients and medical care. Information learned was combined with a review of ITC strategies in other professions to develop principles and guidelines for re-engineering surgeon ITCP. RESULTS: : A total of 328 case descriptions and general comments were obtained and classified. Incidents fell into 4 areas: blurred boundaries of responsibility (87 reports), decreased surgeon familiarity with patients (123 reports), diversion of surgeon attention (31 reports), and distorted or inhibited communication (67 reports). Results were subdivided into 30 contributing factors (eg, shift change, location change, number of providers). Consequences of ITC lapses included delays in patient care (77% of cases), wasted surgeon/staff time (48%), and serious adverse patient consequences (31%). Twelve principles and 5 institutional habit changes are recommended to guide ITCP re-engineering. CONCLUSIONS: Surgeon communication lapses are significant contributors to adverse patient consequences, and provider inefficiency. Re-engineering ITCP will require significant cultural changes.

An investigation of medical student reactions to feedback: a randomised controlled trial.

BACKGROUND: Medical educators have indicated that feedback is one of the main catalysts required for performance improvement. However, medical students appear to be persistently dissatisfied with the feedback that they receive. The purpose of this study was to evaluate learning outcomes and perceptions in students who received feedback compared to those who received general compliments. METHODS: All subjects received identical instruction on two-handed surgical knot-tying. Group 1 received specific, constructive feedback on how to improve their knot-tying skill. Group 2 received only general compliments. Performance was videotaped before and after instruction and after feedback. Subjects completed the study by indicating their global level of satisfaction. Three faculty evaluators observed and scored blinded videotapes of each performance. Intra-observer agreement among expert ratings of performance was calculated using 2-way random effects intraclass correlation (ICC) methods. Satisfaction scores and performance scores were compared using paired samples t-tests and independent samples t-tests. RESULTS: Performance data from 33 subjects were analysed. Inter-rater reliability exceeded 0.8 for ratings of pre-test, pre-intervention and post-intervention performances. The average performance of students who received specific feedback improved (21.98 versus 15.87, P<0.001), whereas there was no significant change in the performance score in the group who received only compliments (17.00 versus 15.39, P=0.181) The average satisfaction rating in the group that received compliments was significantly higher than the group that received feedback (6.00 versus 5.00, P=0.005). DISCUSSION: Student satisfaction is not an accurate measure of the quality of feedback. It appears that satisfaction ratings respond to praise more than feedback, while learning is more a function of feedback.

The effects of group dynamics on resident progress committee deliberations.

BACKGROUND: Prior investigations suggest that resident progress decisions by committee provide a broader perspective on performance and result in less grade inflation. There is countervailing concern that group dynamics may compromise quality of progress decisions. PURPOSE: To determine whether and how group dynamics compromise decision making about resident progress. METHODS: Researchers recorded and analyzed participant comments during a resident progress committee meeting. End-of-rotation (EOR) evaluations were analyzed and compared to progress committee meeting results. RESULTS: EOR and progress committee comments were similar in content. The ratio of specific to general comments was higher for EOR evaluations (5:1) than for progress committee meetings (2:1). EOR evaluation comments provided more supporting evidence for assertions. Individual progress committee participants did not dominate discussion or sway decision making. Participant progress committee meeting comments were consistent with their EOR comments failing to support the presence of progress committee meeting "feeding frenzies." CONCLUSIONS: Results suggest that progress committee meeting group dynamics do not seriously compromise the validity of resident progress decisions.

A senior elective designed to prepare medical students for surgical residency.

BACKGROUND: The transition from medical student to surgery internship can be stressful. The goal of this project was to design, implement, and evaluate a 1-month long elective course that would meet the majority of the American College of Surgeons Graduate Medical Education Committee prerequisites for graduate surgical education METHODS: The major elements of the curriculum included faculty- and resident-facilitated case-based sessions and cadaver dissections. In addition, the students participated in skills laboratory experiences, Intensive Care Unit rounds, and mock interviews and clinical pages. The students took a knowledge pretest and post-test that was compared with the performance of 8 surgical interns on the same examination. RESULTS: The highest rated elements of the course were those that provided hands-on experience or practical knowledge. The post-test knowledge examination scores were significantly higher than pretest scores and surgical intern scores. CONCLUSIONS: It was possible to develop a 1-month senior medical student elective course that provided students with the essential prerequisites believed to be essential for all surgical interns.