Mock Pages Are a Valid Construct for Assessment of Clinical Decision Making and Interprofessional Communication.

IMPORTANCE: Answering pages from nurses about patients in need of immediate attention is one of the most difficult challenges a resident faces during their first days as a physician. A Mock Page program has been developed and adopted into a national surgical resident preparatory curriculum to prepare senior medical students for this important skill. OBJECTIVE: The purpose of this study is to assess standardized mock page cases as a valid construct to assess clinical decision making and interprofessional communication skills. DESIGN, SETTING, PARTICIPANTS: Mock page cases (n = 16) were administered to 213 senior medical students from 12 medical schools participating in a national surgical resident preparatory curriculum in 2013 and 2014. MAIN OUTCOME MEASURES: Clinical decision making and interprofessional communication were measured by case-specific assessments evaluating these skills which have undergone rigorous standard-setting to determine pass/fail cut points. RESULTS: Students' performance improved in general for both communication and clinical decision making over the 4-week course. Cases have been identified that seem to be best suited for differentiating high- from low-performing students. Chest pain, pulmonary embolus, and mental status change cases posed the greatest difficulty for student learners. CONCLUSIONS AND RELEVANCE: Simulated mock pages demonstrate an innovative technique for training students in both effective interprofessional communication and management of common postoperative conditions they will encounter as new surgical interns.

Do Surgeons and Gastroenterologists Describe Endoscopic Retrograde Cholangiopancreatography Differently? A Qualitative Study.

INTRODUCTION: Endoscopic retrograde cholangiopancreatography (ERCP) is a technically challenging procedure performed by both surgeons and gastroenterologists. There is controversy in the field regarding the training necessary to perform ERCP. Widely disparate requisite volumes of experience to achieve proficiency have been published by representatives of each specialty. The basis for these differences has not been fully explored, with particular reference to the cognitive mindset of the different specialties. METHODS: Structured cognitive task analytic interviews were conducted with 7 expert gastroenterologists and 4 expert surgeons from 4 institutions, each of whom performs ERCP as a common procedure in their clinical practice. A qualitative analysis and grounded theory approach was used, focusing specifically on duct cannulation as a critical procedural element. Transcripts were analyzed using Atlas.ti software. RESULTS: The qualitative analysis of 11 transcripts identified 173 unique codes from a total of 653 quotes. In all, 5 themes were found to describe the codes: judgment, teaching, techniques, principles, and equipment. Significant differences were noted between gastroenterologists and surgeons across these themes. Gastroenterologists placed emphasis on issues of judgment including rationale, and emphasized explanation and clarification in teaching. Surgeons placed more emphasis on use of visual cues, and emphasized technique and equipment nuances. CONCLUSION: The data suggest that gastroenterologists deconstruct ERCP competence based on application of rules and rationale through reflection. Surgeons focus more on visual and tactile cues in task deconstruction, and may be more likely to measure proficiency based on technical response to such cues. Based on this study, it is proposed that both specialties might have complementary roles in training therapeutic endoscopists.

How prepared are your interns to take calls? Results of a multi-institutional study of simulated pages to prepare medical students for surgery internship.

BACKGROUND: This study evaluated a simulated pages curriculum that was developed to assess communication and clinical decision making in medical students and interns. METHODS: A curriculum consisting of 14 simulated pages was administered across 5 institutions to 150 senior medical students. A 3-case subset was administered to interns who did not participate in the curriculum. Six expert surgeons identified critical fails and set passing scores for case-specific assessments using the Graphical Hofstee Method. RESULTS: Participants in the curriculum demonstrated superior clinical decision making compared with non-participants across all cases scenarios (P < .01). Average medical student scores for clinical decision making were 46.9%. Global ratings averaged 6.0 for communication and 5.2 for patient care. Passing rates averaged 46%. CONCLUSIONS: Participation in a mock page curriculum improved performance. The performance of participants based on expert standards set for simulated page performance highlight the need for innovative approaches to improve interns' preparedness to take calls.

Foundations for teaching surgeons to address the contributions of systems to operating room team conflict.

BACKGROUND: Prior research has shown that surgeons who effectively manage operating room conflict engage in a problem-solving stage devoted to modifying systems that contribute to team conflict. The purpose of this study was to clarify how systems contributed to operating room team conflict and clarify what surgeons do to modify them. METHODS: Focus groups of circulating nurses and surgeons were conducted at 5 academic medical centers. Narratives describing the contributions of systems to operating room conflict and behaviors used by surgeons to address those systems were analyzed using the constant comparative approach associated with a constructivist grounded theory approach. RESULTS: Operating room team conflict was affected by 4 systems-related factors: team features, procedural-specific staff training, equipment management systems, and the administrative leadership itself. Effective systems problem solving included advocating for change based on patient safety concerns. CONCLUSIONS: The results of this study provide clarity about how systems contribute to operating room conflict and what surgeons can do to effectively modify these systems. This information is foundational material for a conflict management educational program for surgeons.

Surgeons managing conflict in the operating room: defining the educational need and identifying effective behaviors.

BACKGROUND: Developing an operating room conflict management educational program for surgeons requires a formal needs assessment and information about behaviors that represent effective conflict management. METHODS: Focus groups of circulating room nurses and surgeons were conducted at 5 participating centers. Participants responded to queries about conflict management training, conflict consequences, and effective conflict management behaviors. Transcripts of these sessions served as the data for this study. RESULTS: Educational preparation for conflict management was inadequate consisting of trial and error with observed behaviors. Conflict and conflict mismanagement had negative consequences for team members and team performance. Four behaviors emerge as representing effective ways for surgeons to manage conflict. CONCLUSIONS: There is a clear educational need for conflict management education. Target behaviors have now been identified that can provide the basis for a theoretically grounded and contextually adapted instruction and assessment of surgeon conflict management.

Pursuing professional accountability: an evidence-based approach to addressing residents with behavioral problems.

OBJECTIVE: To develop an evidence-based approach to the identification, prevention, and management of surgical residents with behavioral problems. DESIGN: The American College of Surgeons and Southern Illinois University Department of Surgery hosted a 1-day think tank to develop strategies for early identification of problem residents and appropriate interventions. Participants read a selection of relevant literature before the meeting and reviewed case reports. SETTING: American College of Surgeons headquarters, Chicago, Illinois. PARTICIPANTS: Medical and nursing leaders in the field of resident education; individuals with expertise in dealing with academic law, mental health issues, learning deficiencies, and disruptive physicians; and surgical residents. MAIN OUTCOME MEASURES: Evidence-based strategies for the identification, prevention, and management of problem residents. RESULTS: Recommendations based on the literature and expert opinions have been made for the identification, remediation, and reassessment of problem residents. CONCLUSIONS: It is essential to set clear expectations for professional behavior with faculty and residents. A notice of deficiency should define the expected acceptable behavior, timeline for improvement, and consequences for noncompliance. Faculty should note and address systems problems that unintentionally reinforce and thus enable unprofessional behavior. Complaints, particularly by new residents, should be investigated and addressed promptly through a process that is transparent, fair, and reasonable. The importance of early intervention is emphasized.

Using the hidden curriculum to teach professionalism during the surgery clerkship.

BACKGROUND: It has been shown that medical student professionalism is influenced by the hidden curriculum, although the extent to which this occurs during the surgery clerkship is unknown. Furthermore, the processes within the hidden curriculum have been used to teach professionalism to medical students, but this strategy has not been used during the surgery clerkship. The purpose of this study was to review a 2-year experience with a surgery clerkship instructional session where the hidden curriculum was used to teach professionalism to medical students. STUDY DESIGN: Medical student essays were analyzed to evaluate the influence of the hidden curriculum on their ideas about professionalism and to identify specific behaviors that they regarded as professional and unprofessional. The instructional session was evaluated using the average satisfaction session ratings and through an analysis of medical student session evaluation comments. RESULTS: Seventy-five percent of medical students reported that their ideas about professionalism changed. This change involved their general concepts about professionalism, identifying specific behaviors that they planned to adopt or avoid, or developing opinions about the professionalism of surgeons. The average satisfaction rating was consistently high throughout the study period, and the most helpful session feature was reported as the opportunity to share and discuss their observations. CONCLUSIONS: The hidden curriculum has a substantial influence on the development of professionalism of medical students during the surgery clerkship. It was possible to illuminate and use the hidden curriculum to create an instructional session devoted to professionalism for medical students on the surgery clerkship.

Engaging medical students in the feedback process.

BACKGROUND: There are potential advantages to engaging medical students in the feedback process, but efforts to do so have yielded mixed results. The purpose of this study was to evaluate a student-focused feedback instructional session in an experimental setting. METHODS: Medical students were assigned randomly to either the intervention or control groups and then assigned randomly to receive either feedback or compliments. Tests of knowledge, skills, and attitudes were given before and after the intervention. RESULTS: There was a significant gain of knowledge and skill in the group that received instruction. Satisfaction was higher after compliments in the control group but higher after feedback in the instructional group. There was no change in the subject's willingness to seek feedback. CONCLUSIONS: A student-focused component should be carefully included as part of an overall effort to improve feedback in surgical education. The role of medical student attitudes about feedback requires further investigation.

Teaching operating room conflict management to surgeons: clarifying the optimal approach.

CONTEXT: Conflict management has been identified as an essential competence for surgeons as they work in operating room (OR) teams; however, the optimal approach is unclear. Social science research offers two alternatives, the first of which recommends that task-related conflict be managed using problem-solving techniques while avoiding relationship conflict. The other approach advocates for the active management of relationship conflict as it almost always accompanies task-related conflict. Clarity about the optimal management strategy can be gained through a better understanding of conflict transformation, or the inter-relationship between conflict types, in this specific setting. The purpose of this study was to evaluate conflict transformation in OR teams in order to clarify the approach most appropriate for an educational conflict management programme for surgeons. METHODS: A constructivist grounded theory approach was adopted to explore the phenomenon of OR team conflict. Narratives were collected from focus groups of OR nurses and surgeons at five participating centres. A subset of these narratives involved transformation between and within conflict types. This dataset was analysed. RESULTS: The results confirm that misattribution and the use of harsh language cause conflict transformation in OR teams just as they do in stable work teams. Negative emotionality was found to make a substantial contribution to responses to and consequences of conflict, notably in the swiftness with which individuals terminated their working relationships. These findings contribute to a theory of conflict transformation in the OR team. CONCLUSIONS: There are a number of behaviours that activate conflict transformation in the OR team and a conflict management education programme should include a description of and alternatives to these behaviours. The types of conflict are tightly interwoven in this setting and thus the most appropriate management strategy is one that assumes that both types of conflict will exist and should be managed actively.

Use of simulated pages to prepare medical students for internship and improve patient safety.

PURPOSE: During the transition from medical school to internship, trainees experience high levels of stress related to pages on the inpatient wards. The steep learning curve during this period may also affect patient safety. The authors piloted the use of simulated pages to improve medical student preparedness, decrease stress related to pages, and familiarize medical students with common patient problems. METHOD: A multidisciplinary team at Southern Illinois University School of Medicine developed simulated pages that were tested among senior medical students. Sixteen medical students were presented with 11 common patient scenarios. Data on assessment, management, and global performance were collected. Mean confidence levels were evaluated pre- and postintervention. Students were also surveyed on how the simulated pages program influenced their perceived comfort in managing patient care needs and the usefulness of the exercise in preparing them to handle inpatient pages. RESULTS: Mean scores on the assessment and management portions of the scenarios varied widely depending on the scenario (range -15.6 ± 41.6 to 95.7 ± 9.5). Pass rates based on global performance ranged from 12% to 93%. Interrater agreement was high (mean kappa = 0.88). Students' confidence ratings on a six-point scale increased from 1.87 preintervention to 3.53 postintervention (P < .0001). CONCLUSIONS: Simulated pages engage medical students and may foster medical student preparedness for internship. Students valued the opportunity to simulate "on call" responsibilities, and exposure to simulated pages significantly increased their confidence levels. Further studies are needed to determine effects on patient safety outcomes.

Skills coaches as part of the educational team: a randomized controlled trial of teaching of a basic surgical skill in the laboratory setting.

BACKGROUND: The aim of this study was to compare the laboratory teaching of a basic technical skill by a nonphysician skills coach and a faculty surgeon. METHODS: Medical students were randomized to instruction of skin suturing in the skills laboratory by a faculty surgeon or by a nonphysician skills coach. Testing of performance occurred at 3 time points. Other faculty surgeons, blinded to identities and training groups, rated performance. RESULTS: Forty-nine students participated. Baseline fourth-year student mean scores showed no significant difference between training groups. Third-year and fourth-year student performance showed no difference between training groups on postintervention testing. Delayed testing also showed no difference in third-year student scores. CONCLUSIONS: Training by either a nonsurgeon skills coach or a faculty surgeon resulted in no difference in performance on a basic surgical skill. This was true for students with and without prior experience and was also true after subsequent clinical experiences. Nonphysician coaches may ease the teaching burden of surgical faculty members while providing similar quality of instruction for trainees.

Refining the evaluation of operating room performance.

PURPOSE: An accurate and consistent evaluation of resident operative performance is necessary but difficult to achieve. This study continues the examination of the Southern Illinois University (SIU) operative performance rating system (OPRS) by studying additional factors that may influence reliability, accuracy, and interpretability of results. METHODS: OPRS evaluations of surgical residents by faculty at SIU, from 2001 to 2008, were analyzed for the most frequently rated procedures to determine (1) the elapsed time from the procedure until completion of rating, (2) the patterns in responses of procedure-specific and global surgical skills items, and (3) whether particular evaluating surgeons differed in their stringency of ratings of resident operative performance. RESULTS: In all, 566 evaluations were analyzed, which consisted of open colectomy (n = 125), open inguinal hernia (n = 103), laparoscopic cholecystectomy (n = 199), and excisional biopsy (n = 139). The number of residents evaluated per training level (PGY) ranged from 88 to 161. The median time to completion of evaluations was 11 days, 9 hours. The quickest evaluation was 18 hours after assignment. Most were completed within 4.5 to 22 days. Procedure-specific and global scale scores resulted in similar rank-ordering of performances (single-measure intraclass correlation using the consistency model = 0.88; 95% confidence interval [CI] = 0.87-0.90) and similar absolute OPRS scores (single-measure intraclass correlation using the consistency model = 0.89; 95% CI, 0.87-0.90). Evaluating surgeons differed in stringency of ratings across procedures (average difference = 1.4 points of 5 possible points). Resident performance improved with increasing PGY level for all 4 procedures. CONCLUSIONS: Substantial time elapses between performance in the operating room and the completion of the evaluation. This raises the question of whether surgeons remember the nuances of the procedure well enough to rate performance accurately. The item type used for rating does not affect the absolute rating assigned or the rank ordering of the performance. Differences in stringency of evaluators indicate the need for multiple resident performance observations by multiple surgeons. These findings are the foundation for an upcoming multi-institutional trial.

A theory-based curriculum for enhancing surgical skillfulness.

BACKGROUND: Curricula for surgical technical skills laboratories have traditionally been designed to accommodate the clinical activities of residents, so they typically consist of individual, episodic training sessions. We believe that the skills laboratory offers an opportunity to design a surgical skills curriculum based on the fundamental elements known to be important for motor skill instruction. We hypothesized that training novices with such a curriculum for a 1-month period would yield skills performance levels equivalent to those of second year surgery residents who had trained in a traditional program. STUDY DESIGN: Fourth-year medical students served as study subjects (novice group) during a 4-week senior elective. They were taught each skill during a 1-week period. Subjects received instruction by a content expert followed by a 1-week period of deliberate practice with feedback. The novice performances were videotaped both before and after the intervention, and each videotape was evaluated in a blinded fashion by experts using a validated evaluation instrument. These results were compared with skill performance ratings of first- and second-year surgery residents that had been accumulated over the previous 3 years. RESULTS: Average performance ratings for the novices substantially improved for all four skills after training. There was no marked difference between average performance ratings of postintervention novice scores when compared with the average scores in the resident group. Inter-rater agreement in scoring for the videotaped novice performances exceeded 0.87 (intraclass correlation) for all ratings of pre- and posttraining. CONCLUSIONS: These results demonstrate the effectiveness of a laboratory-based training program that includes fundamentals of motor skills acquisition.

An investigation of medical student reactions to feedback: a randomised controlled trial.

BACKGROUND: Medical educators have indicated that feedback is one of the main catalysts required for performance improvement. However, medical students appear to be persistently dissatisfied with the feedback that they receive. The purpose of this study was to evaluate learning outcomes and perceptions in students who received feedback compared to those who received general compliments. METHODS: All subjects received identical instruction on two-handed surgical knot-tying. Group 1 received specific, constructive feedback on how to improve their knot-tying skill. Group 2 received only general compliments. Performance was videotaped before and after instruction and after feedback. Subjects completed the study by indicating their global level of satisfaction. Three faculty evaluators observed and scored blinded videotapes of each performance. Intra-observer agreement among expert ratings of performance was calculated using 2-way random effects intraclass correlation (ICC) methods. Satisfaction scores and performance scores were compared using paired samples t-tests and independent samples t-tests. RESULTS: Performance data from 33 subjects were analysed. Inter-rater reliability exceeded 0.8 for ratings of pre-test, pre-intervention and post-intervention performances. The average performance of students who received specific feedback improved (21.98 versus 15.87, P<0.001), whereas there was no significant change in the performance score in the group who received only compliments (17.00 versus 15.39, P=0.181) The average satisfaction rating in the group that received compliments was significantly higher than the group that received feedback (6.00 versus 5.00, P=0.005). DISCUSSION: Student satisfaction is not an accurate measure of the quality of feedback. It appears that satisfaction ratings respond to praise more than feedback, while learning is more a function of feedback.

Feasibility, reliability and validity of an operative performance rating system for evaluating surgery residents.

BACKGROUND: Resident evaluation traditionally involves global assessments including clinical performance, professional behavior, technical skill, and number of procedures performed. These evaluations lack objective assessment of operative skills. We describe an operative performance rating system (OPRS) designed to provide objective operative performance ratings using a sentinel procedure format. METHODS: Ten-item procedure-specific rating instruments were developed. Items included technical skills, operative decision making, and general items. A 1 to 5 (5 = excellent) scale was used for evaluation. Six procedures had sufficient forms returned to allow evaluation. Inter-rater reliability was determined by having faculty evaluators view 2 videotaped operations. RESULTS: Return rates for the Internet-based form were full-time faculty (92%), volunteer faculty (27%), and overall (67%). Reliability, (average interitem correlation), and total procedures evaluated were excisional biopsy, 0.90, (0.48), 77; open inguinal herniorraphy, 0.94, (0.62), 51; laparoscopic cholecystectomy, 0.95, (0.64), 75; small-bowel and colon resection, 0.92, (0.58), 30; parathyroidectomy, 0.70, (0.19), 30; and lumpectomy, 0.92, (0.51), 38. Years of training accounted for 25% to 57% of the variation in scores. Inter-rater variability was observed; however, the average rater agreement was reliable. CONCLUSIONS: Internet-based management made obtaining the data feasible. The OPRS complements traditional evaluations by providing objective assessment of operative decision-making and technical skills. Interitem correlations indicate the average rating of items provides a reliable indicator of resident performance. The OPRS is useful in tracking resident development throughout postgraduate training and offers a structured means of certifying operative skills.

A senior elective designed to prepare medical students for surgical residency.

BACKGROUND: The transition from medical student to surgery internship can be stressful. The goal of this project was to design, implement, and evaluate a 1-month long elective course that would meet the majority of the American College of Surgeons Graduate Medical Education Committee prerequisites for graduate surgical education METHODS: The major elements of the curriculum included faculty- and resident-facilitated case-based sessions and cadaver dissections. In addition, the students participated in skills laboratory experiences, Intensive Care Unit rounds, and mock interviews and clinical pages. The students took a knowledge pretest and post-test that was compared with the performance of 8 surgical interns on the same examination. RESULTS: The highest rated elements of the course were those that provided hands-on experience or practical knowledge. The post-test knowledge examination scores were significantly higher than pretest scores and surgical intern scores. CONCLUSIONS: It was possible to develop a 1-month senior medical student elective course that provided students with the essential prerequisites believed to be essential for all surgical interns.

Do individual attendings' post-rotation performance ratings detect residents' clinical performance deficiencies?

PURPOSE: To determine whether attending physicians' post-rotation performance ratings and written comments detect surgery residents' clinical performance deficits. METHOD: Residents' performance records from 1997-2002 in the Department of Surgery, Southern Illinois University School of Medicine, were reviewed to determine the percentage of times end-of-rotation performance ratings and/or comments detected deficiencies leading to negative end-of-year progress decisions. RESULTS: Thirteen of 1,986 individual post-rotation ratings (0.7%) nominally noted a deficit. Post-rotation ratings of "good" or below were predictive of negative end-of-year progress decisions. Eighteen percent of residents determined to have some deficiency requiring remediation received no post-rotation performance ratings indicating that deficiency. Written comments on post-rotation evaluation forms detected deficits more accurately than did numeric ratings. Physicians detected technical skills performance deficits more frequently than applied knowledge and professional behavior deficits. More physicians' post-rotation numeric ratings contradicted performance deficits than supported them. More written comments supported deficits than contradicted them in the technical skills area. In the applied knowledge and professional behavior areas, more written comments contradicted deficits than supported them. CONCLUSIONS: A large percentage of performance deficiencies only became apparent when the attending physicians discussed performance at the annual evaluation meetings. Annual evaluation meetings may (1) make patterns of residents' behavior apparent that were not previously apparent to individual physicians, (2) provide evidence that strengthens the individual attending's preexisting convictions about residents' performance deficiencies, or (3) lead to erroneous conclusions. The authors believe deficiencies were real and that their findings can be explained by a combination of reasons one and two.

Variables influencing medical student learning in the operating room.

BACKGROUND: The operating room (OR) is an important venue where surgeons do much of medical student teaching and yet there has been little work evaluating variables that influence learning in this unique environment. We designed this study to identify variables that affected medical student learning in the OR. METHODS: We developed a questionnaire based on surgery faculty observations of learning in the OR. The medical students completed the questionnaire on 114 learning episodes in the OR. Pearson correlation coefficient was used to establish the strength of association between various variables and the student's overall perception of learning. RESULTS: The students evaluated 27 variables that might impact their learning in the OR. Strong correlations were identified between the attending physician's attitude, interactions and teaching ability in the OR and the environment being conducive to learning. CONCLUSIONS: Surgical faculty behavior is a powerful determinant of student perceptions of what provides for a favorable learning environment in the OR.

Who are the surgery clerkship directors and what are their educational needs?

BACKGROUND: The surgery clerkship director is a key individual in the surgery department's educational mission and yet there has been no prior effort to describe this group or identify their learning needs. The purpose of this study was to develop a demographic profile and an educational needs assessment for surgery clerkship directors. METHODS: A survey instrument was designed based on existing literature and distributed to surgery clerkship directors in the United States and Canada. RESULTS: Surveys were returned from 108 subjects (77%). The majority of clerkship directors strongly agree that directing is a positive experience but express concern that the job demands may impede their professional careers. The perceived educational needs identified related primarily to the development and management of the student education curriculum. CONCLUSIONS: Surgery clerkship directors are experienced academic surgeons who report high levels of satisfaction. They identify a number of important educational needs of the position and express concern about the requirements of the position on their academic careers.

Medical student contact with patients on a surgery clerkship: is there a chance to learn?

BACKGROUND: Earlier studies of medical students on nonsurgical rotations have shown that clinical clerks usually first interact with their patients late in the clinical course. This would seem disadvantageous to the student's learning because they would have less opportunity to generate diagnoses or a management plan. STUDY DESIGN: A questionnaire designed to assess the nature of medical student-patient interactions in all potential clinical sites was administered to third year medical students during their surgical clerkship. Students received questionnaires each day to evaluate their clinical experiences from the previous day. RESULTS: The results from 311 student-patient encounters were collected and analyzed by clinical site as follows: outpatient clinics, outpatient surgery, inpatient surgery, day of surgery admission, inpatient consults, or emergency room consults. Students reported significantly more opportunities to elicit chief complaint, generate potential diagnosis, develop or suggest a management plan, and perform the initial examination when in the clinic setting. CONCLUSIONS: Overall, students were given relatively few opportunities to be the first to interact with any patient in any setting. They infrequently had an opportunity to independently generate a hypothesis or generate a management plan. Currently, the clinic offers the best opportunity for the student to complete these processes.