World Health Organization Surgical Safety Checklist Modification: Do Changes Emphasize Communication and Teamwork?

BACKGROUND: The World Health Organization's (WHO) surgical safety checklist is meant to be customized to facilitate local implementation, encourage full-team participation, and promote a culture of safety. Although it has been globally adopted, little is known about the extent of checklist modification and the type of changes made. METHODS: Nonsubspecialty surgical checklists were obtained through online search and targeted hospital requests. A detailed coding scheme was created to capture modifications to checklist content and formatting. Descriptive statistics were performed. RESULTS: Of 155 checklists analyzed, all were modified. Compared with the WHO checklist, those in our sample contained more lines of text (median: 63 [interquartile range: 50-73] versus 56) and items (36 [interquartile range: 30-43] versus 28). A median of 13 new items were added. Items most frequently added included implants/special equipment (added by 84%), deep vein thrombosis prophylaxis/anticoagulation (added by 75%), and positioning (added by 63%). Checklists removed a median of 5 WHO items. The most frequently removed item was the pulse oximeter check (removed in 75%), followed by 4 items (each removed in 39%-48%) that comprise part of the WHO Checklist's "Anticipated Critical Events" section, which is intended for exchanging critical information. The surgeon was not explicitly mentioned in the checklist in 12%; the anesthesiologist/certified registered nurse anesthetist in 14%, the circulator in 10%, and the surgical tech/scrub in 79%. CONCLUSIONS: Checklists are highly modified but often enlarged with items that may not prompt discussion or teamwork. Of concern is the frequent removal of items from the WHO's "Anticipated Critical Events" section.

Scaling Safety: The South Carolina Surgical Safety Checklist Experience.

Proven patient safety solutions such as the World Health Organization's Surgical Safety Checklist are challenging to implement at scale. A voluntary initiative was launched in South Carolina hospitals in 2010 to encourage use of the checklist in all operating rooms. Hospitals that reported completing implementation of the checklist in their operating rooms by 2017 had significantly higher levels of CEO and physician participation and engaged more in higher-touch activities such as in-person meetings and teamwork skills trainings than comparison hospitals did. Based on our experience and the participation data collected, we suggest three considerations for hospital, hospital association, state, and national policy makers: Successful programs must be designed to engage all stakeholders (CEOs, physicians, nurses, surgical technologists, and others); offering a variety of program activities-both lower-touch and higher-touch-over the duration of the program allows more hospital and individual participation; and change takes time and resources.

Mortality Trends After a Voluntary Checklist-based Surgical Safety Collaborative.

OBJECTIVE: To determine whether completion of a voluntary, checklist-based surgical quality improvement program is associated with reduced 30-day postoperative mortality. BACKGROUND: Despite evidence of efficacy of team-based surgical safety checklists in improving perioperative outcomes in research trials, effective methods of population-based implementation have been lacking. The Safe Surgery 2015 South Carolina program was designed to foster state-wide engagement of hospitals in a voluntary, collaborative implementation of a checklist program. METHODS: We compared postoperative mortality rates after inpatient surgery in South Carolina utilizing state-wide all-payer discharge claims from 2008 to 2013, linked with state vital statistics, stratifying hospitals on the basis of completion of the checklist program. Changes in risk-adjusted 30-day mortality were compared between hospitals, using propensity score-adjusted difference-in-differences analysis. RESULTS: Fourteen hospitals completed the program by December 2013. Before program launch, there was no difference in mortality trends between the completion cohort and all others (P = 0.33), but postoperative mortality diverged thereafter (P = 0.021). Risk-adjusted 30-day mortality among completers was 3.38% in 2010 and 2.84% in 2013 (P < 0.00001), whereas mortality among other hospitals (n = 44) was 3.50% in 2010 and 3.71% in 2013 (P = 0.3281), reflecting a 22% difference between the groups on difference-in-differences analysis (P = 0.0021). CONCLUSIONS: Despite similar pre-existing rates and trends of postoperative mortality, hospitals in South Carolina completing a voluntary checklist-based surgical quality improvement program had a reduction in deaths after inpatient surgery over the first 3 years of the collaborative compared with other hospitals in the state. This may indicate that effective large-scale implementation of a team-based surgical safety checklist is feasible.

Perception of Safety of Surgical Practice Among Operating Room Personnel From Survey Data Is Associated With All-cause 30-day Postoperative Death Rate in South Carolina.

OBJECTIVE: To evaluate whether the perception of safety of surgical practice among operating room (OR) personnel is associated with hospital-level 30-day postoperative death. BACKGROUND: The relationship between improvements in the safety of surgical practice and benefits to postoperative outcomes has not been demonstrated empirically. METHODS: As part of the Safe Surgery 2015: South Carolina initiative, a baseline survey measuring the perception of safety of surgical practice among OR personnel was completed. We evaluated the relationship between hospital-level mean item survey scores and rates of all-cause 30-day postoperative death using binomial regression. Models were controlled for multiple patient, hospital, and procedure covariates using supervised principal components regression. RESULTS: The overall survey response rate was 38.1% (1793/4707) among 31 hospitals. For every 1 point increase in the hospital-level mean score for respect [adjusted relative risk (aRR) 0.78, 95% CI 0.65-0.93, P = 0.0059], clinical leadership (aRR 0.86, 95% CI 0.74-0.9932, P = 0.0401), and assertiveness (aRR 0.71, 95% CI 0.54-0.93, P = 0.01) among all survey respondents, there were associated decreases in the hospital-level 30-day postoperative death rate after inpatient surgery ranging from 14% to 29%. Higher hospital-level mean scores for the statement, "I would feel safe being treated here as a patient," were associated with significantly lower hospital-level 30-day postoperative death rates (aRR 0.83, 95% CI 0.70-0.97, P = 0.02). Although most findings seen among all OR personnel were seen among nurses, they were often absent among surgeons. CONCLUSIONS: Perception of OR safety of surgical practice was associated with hospital-level 30-day postoperative death rates.

Relationship Between Operating Room Teamwork, Contextual Factors, and Safety Checklist Performance.

BACKGROUND: Studies show that using surgical safety checklists (SSCs) reduces complications. Many believe SSCs accomplish this by enhancing teamwork, but evidence is limited. Our study sought to relate teamwork to checklist performance, understand how they relate, and determine conditions that affect this relationship. STUDY DESIGN: Using 2 validated tools for observing and coaching operating room teams, we evaluated the association between checklist performance with surgeon buy-in and 4 domains of surgical teamwork: clinical leadership, communication, coordination, and respect. Hospital staff in 10 South Carolina hospitals observed 207 procedures between April 2011 and January 2013. We calculated levels of checklist performance, buy-in, and measures of teamwork, and evaluated their relationship, controlling for patient and case characteristics. RESULTS: Few teams completed most or all SSC items. Teams more often completed items considered procedural "checks" than conversation "prompts." Surgeon buy-in, clinical leadership, communication, a summary measure of teamwork overall, and observers' teamwork ratings positively related to overall checklist completion (multivariable model estimates from 0.04, p < 0.05 for communication to 0.17, p < 0.01 for surgeon buy-in). All measures of teamwork and surgeon buy-in related positively to completing more conversation prompts; none related significantly to procedural checks (estimates from 0.10, p < 0.01 for communication to 0.27, p < 0.001 for surgeon buy-in). Patient age was significantly associated with completing the checklist and prompts (p < 0.05); only case duration was positively associated with performing more checks (p < 0.10). CONCLUSIONS: Surgeon buy-in and surgical teamwork characterized by shared clinical leadership, open communication, active coordination, and mutual respect were critical in prompting case-related conversations, but not in completing procedural checks. Findings highlight the importance of surgeon engagement and high-quality, consistent teamwork for promoting checklist use and ensuring a safe surgical environment.

Implementation of the Surgical Safety Checklist in South Carolina Hospitals Is Associated with Improvement in Perceived Perioperative Safety.

BACKGROUND: Previous research suggests that surgical safety checklists (SSCs) are associated with reductions in postoperative morbidity and mortality as well as improvement in teamwork and communication. These findings stem from evaluations of individual or small groups of hospitals. Studies with more hospitals have assessed the relationship of checklists with teamwork at a single point in time. The objective of this study was to evaluate the impact of a large-scale implementation of SSCs on staff perceptions of perioperative safety in the operating room. STUDY DESIGN: As part of the Safe Surgery 2015 initiative to implement SSCs in South Carolina hospitals, we administered a validated survey designed to measure perception of multiple dimensions of perioperative safety among clinical operating room personnel before and after implementation of an SSC. RESULTS: Thirteen hospitals administered baseline and follow-up surveys, separated by 1 to 2 years. Response rates were 48.4% at baseline (929 of 1,921) and 42.7% (815 of 1,909) at follow-up. Results suggest improvement in all of the 5 dimensions of teamwork (relative percent improvement ranged from +2.9% for coordination to +11.9% for communication). These were significant after adjusting for respondent characteristics, hospital fixed-effects, multiple comparisons, and clustering robust standard errors by hospital (all p < 0.05). More than half of respondents (54.1%) said their surgical teams always used checklists effectively; 73.6% said checklists had averted problems or complications. CONCLUSIONS: A large-scale initiative to implement SSCs is associated with improved staff perceptions of mutual respect, clinical leadership, assertiveness on behalf of safety, team coordination and communication, safe practice, and perceived checklist outcomes.

Surgical team member assessment of the safety of surgery practice in 38 South Carolina hospitals.

We assessed surgical team member perceptions of multiple dimensions of safe surgical practice in 38 South Carolina hospitals participating in a statewide initiative to implement surgical safety checklists. Primary data were collected using a novel 35-item survey. We calculated the percentage of 1,852 respondents with strongly positive, positive, and neutral/negative responses about the safety of surgical practice, compared results by hospital and professional discipline, and examined how readiness, teamwork, and adherence related to staff perception of care quality. Overall, 78% of responses were positive about surgical safety at respondent's hospitals, but in each survey dimension, from 16% to 40% of responses were neutral/negative, suggesting significant opportunity to improve surgical safety. Respondents not reporting they would feel safe being treated in their operating rooms varied from 0% to 57% among hospitals. Surgeons responded more positively than nonsurgeons. Readiness, teamwork, and practice adherence related directly to staff perceptions of patient safety (p < .001).

The Surgical Safety Checklist and Teamwork Coaching Tools: a study of inter-rater reliability.

OBJECTIVE: To assess the inter-rater reliability (IRR) of two novel observation tools for measuring surgical safety checklist performance and teamwork. SUMMARY BACKGROUND: Data surgical safety checklists can promote adherence to standards of care and improve teamwork in the operating room. Their use has been associated with reductions in mortality and other postoperative complications. However, checklist effectiveness depends on how well they are performed. METHODS: Authors from the Safe Surgery 2015 initiative developed a pair of novel observation tools through literature review, expert consultation and end-user testing. In one South Carolina hospital participating in the initiative, two observers jointly attended 50 surgical cases and independently rated surgical teams using both tools. We used descriptive statistics to measure checklist performance and teamwork at the hospital. We assessed IRR by measuring percent agreement, Cohen's κ, and weighted κ scores. RESULTS: The overall percent agreement and κ between the two observers was 93% and 0.74 (95% CI 0.66 to 0.79), respectively, for the Checklist Coaching Tool and 86% and 0.84 (95% CI 0.77 to 0.90) for the Surgical Teamwork Tool. Percent agreement for individual sections of both tools was 79% or higher. Additionally, κ scores for six of eight sections on the Checklist Coaching Tool and for two of five domains on the Surgical Teamwork Tool achieved the desired 0.7 threshold. However, teamwork scores were high and variation was limited. There were no significant changes in the percent agreement or κ scores between the first 10 and last 10 cases observed. CONCLUSIONS: Both tools demonstrated substantial IRR and required limited training to use. These instruments may be used to observe checklist performance and teamwork in the operating room. However, further refinement and calibration of observer expectations, particularly in rating teamwork, could improve the utility of the tools.

Crisis checklists for the operating room: development and pilot testing.

BACKGROUND: Because operating room crises are rare events, failure to adhere to critical management steps is common. We sought to develop and pilot a tool to improve adherence to lifesaving measures during operating room crises. STUDY DESIGN: We identified 12 of the most frequently occurring operating room crises and corresponding evidence-based metrics of essential care for each (46 total process measures). We developed checklists for each crisis based on a previously defined method, which included literature review, multidisciplinary expert consultation, and simulation. After development, 2 operating room teams (11 participants) were each exposed to 8 simulations with random assignment to checklist use or working from memory alone. Each team managed 4 simulations with a checklist available and 4 without. One of the primary outcomes measured through video review was failure to adhere to essential processes of care. Participants were surveyed for perceptions of checklist use and realism of the scenarios. RESULTS: Checklist use resulted in a 6-fold reduction in failure of adherence to critical steps in management for 8 scenarios with 2 pilot teams. These results held in multivariate analysis accounting for clustering within teams and adjusting for learning or fatigue effects (11 of 46 failures without the checklist vs 2 of 46 failures with the checklist; adjusted relative risk = 0.15, 95% CI, 0.04-0.60; p = 0.007). All participants rated the overall quality of the checklists and scenarios to be higher than average or excellent. CONCLUSIONS: Checklist use can improve safety and management in operating room crises. These findings warrant broader evaluation, including in clinical settings.

Effective surgical safety checklist implementation.

BACKGROUND: Research suggests that surgical safety checklists can reduce mortality and other postoperative complications. The real world impact of surgical safety checklists on patient outcomes, however, depends on the effectiveness of hospitals' implementation processes. STUDY DESIGN: We studied implementation processes in 5 Washington State hospitals by conducting semistructured interviews with implementation leaders and surgeons from September to December 2009. Interviews were transcribed, analyzed, and compared with findings from previous implementation research to identify factors that distinguish effective implementation. RESULTS: Qualitative analysis suggested that effectiveness hinges on the ability of implementation leaders to persuasively explain why and adaptively show how to use the checklist. Coordinated efforts to explain why the checklist is being implemented and extensive education regarding its use resulted in buy-in among surgical staff and thorough checklist use. When implementation leaders did not explain why or show how the checklist should be used, staff neither understood the rationale behind implementation nor were they adequately prepared to use the checklist, leading to frustration, disinterest, and eventual abandonment despite a hospital-wide mandate. CONCLUSIONS: The impact of surgical safety checklists on patient outcomes is likely to vary with the effectiveness of each hospital's implementation process. Further research is needed to confirm these findings and reveal additional factors supportive of checklist implementation.