Increased adherence to perioperative safety guidelines associated with improved patient safety outcomes: a stepped-wedge, cluster-randomised multicentre trial.

BACKGROUND: National Dutch guidelines have been introduced to improve suboptimal perioperative care. A multifaceted implementation programme (IMPlementatie Richtlijnen Operatieve VEiligheid [IMPROVE]) has been developed to support hospitals in applying these guidelines. This study evaluated the effectiveness of IMPROVE on guideline adherence and the association between guideline adherence and patient safety. METHODS: Nine hospitals participated in this unblinded, superiority, stepped-wedge, cluster RCT in patients with major noncardiac surgery (mortality risk ≥1%). IMPROVE consisted of educational activities, audit and feedback, reminders, organisational, team-directed, and patient-mediated activities. The primary outcome of the study was guideline adherence measured by nine patient safety indicators on the process (stop moments from the composite STOP bundle, and timely administration of antibiotics) and on the structure of perioperative care. Secondary safety outcomes included in-hospital complications, postoperative wound infections, mortality, length of hospital stay, and unplanned care. RESULTS: Data were analysed for 1934 patients. The IMPROVE programme improved one stop moment: 'discharge from recovery room' (+16%; 95% confidence interval [CI], 9-23%). This stop moment was related to decreased mortality (-3%; 95% CI, -4% to -1%), fewer complications (-8%; 95% CI, -13% to -3%), and fewer unscheduled transfers to the ICU (-6%; 95% CI, -9% to -3%). IMPROVE negatively affected one other stop moment - 'discharge from the hospital' - possibly because of the limited resources of hospitals to improve all stop moments together. CONCLUSIONS: Mixed implementation effects of IMPROVE were found. We found some positive associations between guideline adherence and patient safety (i.e. mortality, complications, and unscheduled transfers to the ICU) except for the timely administration of antibiotics. CLINICAL TRIAL REGISTRATION: NTR3568 (Dutch Trial Registry).

Development of a tool for identifying and addressing prioritised determinants of quality improvement initiatives led by healthcare professionals: a mixed-methods study.

BACKGROUND: Several frameworks have been developed to identify essential determinants for healthcare improvement. These frameworks aim to be comprehensive, leading to the creation of long lists of determinants that are not prioritised based on being experienced as most important. Furthermore, most existing frameworks do not describe the methods or actions used to identify and address the determinants, limiting their practical value. The aim of this study is to describe the development of a tool with prioritised facilitators and barriers supplemented with methods to identify and address each determinant. The tool can be used by those performing quality improvement initiatives in healthcare practice. METHODS: A mixed-methods study design was used to develop the tool. First, an online survey was used to ask healthcare professionals about the determinants they experienced as most facilitating and most hindering during the performance of their quality improvement initiative. A priority score was calculated for every named determinant, and those with a priority score ≥ 20 were incorporated into the tool. Semi-structured interviews with implementation experts were performed to gain insight on how to analyse and address the determinants in our tool. RESULTS: The 25 healthcare professionals in this study experienced 64 facilitators and 66 barriers when performing their improvement initiatives. Of these, 12 facilitators and nine barriers were incorporated into the tool. Sufficient support from management of the department was identified as the most important facilitator, while having limited time to perform the initiative was considered the most important barrier. The interviews with 16 experts in implementation science led to various inputs for identifying and addressing each determinant. Important themes included maintaining adequate communication with stakeholders, keeping the initiative at a manageable size, learning by doing and being able to influence determinants. CONCLUSIONS: This paper describes the development of a tool with prioritised determinants for performing quality improvement initiatives with suggestions for analysing and addressing these determinants. The tool is developed for those engaged in quality improvement initiatives in practice, so in this way it helps in bridging the research to practice gap of determinants frameworks. More research is needed to validate and develop the tool further.

Reducing work pressure and IT problems and facilitating IT integration and audit & feedback help adherence to perioperative safety guidelines: a survey among 95 perioperative professionals.

BACKGROUND: To improve perioperative patient safety, guidelines for the preoperative, peroperative, and postoperative phase were introduced in the Netherlands between 2010 and 2013. To help the implementation of these guidelines, we aimed to get a better understanding of the barriers and drivers of perioperative guideline adherence and to explore what can be learned for future implementation projects in complex organizations. METHODS: We developed a questionnaire survey based on the theoretical framework of Van Sluisveld et al. for classifying barriers and facilitators. The questionnaire contained 57 statements derived from (a) an instrument for measuring determinants of innovations by the Dutch Organization for Applied Scientific Research, (b) interviews with quality and safety policy officers and perioperative professionals, and (c) a publication of Cabana et al. The target group consisted of 232 perioperative professionals in nine hospitals. In addition to rating the statements on a five-point Likert scale (which were classified into the seven categories of the framework: factors relating to the intervention, society, implementation, organization, professional, patients, and social factors), respondents were invited to rank their three most important barriers in a separate, extra open-ended question. RESULTS: Ninety-five professionals (41%) completed the questionnaire. Fifteen statements (26%) were considered to be barriers, relating to social factors (N = 5), the organization (N = 4), the professional (N = 4), the patient (N = 1), and the intervention (N = 1). An integrated information system was considered an important facilitator (70.4%) as well as audit and feedback (41.8%). The Barriers Top-3 question resulted in 75 different barriers in nearly all categories. The most frequently reported barriers were as follows: time pressure (16% of the total number of barriers), emergency patients (8%), inefficient IT structure (4%), and workload (3%). CONCLUSIONS: We identified a wide range of barriers that are believed to hinder the use of the perioperative safety guidelines, while an integrated information system and local data collection and feedback will also be necessary to engage perioperative teams. These barriers need to be locally prioritized and addressed by tailored implementation strategies. These results may also be of relevance for guideline implementation in general in complex organizations. TRIAL REGISTRATION: Dutch Trial Registry: NTR3568.

Does a Long-Term Quality and Safety Curriculum for Health Care Professionals Improve Clinical Practice? An Evaluation of Quality Improvement Projects.

INTRODUCTION: Little is known about the effects on clinical practice of continuing education quality and safety curricula. The aim of this study is to gain insight into learning outcomes on the fourth level of the Kirkpatrick evaluation model for systematically deployed quality improvement projects performed by health care professionals during a Masters in Healthcare Quality and Safety in the Netherlands. METHODS: The researchers reviewed 35 projects led by health care professionals in 16 different hospitals to determine their scopes and effects. Afterward, professionals took an online survey to determine the extent of their project's sustainability and spread. RESULTS: Improving health care safety was the most prevalent quality dimension (n = 11, 31%). A positive change was measured by professionals for 64% (n = 35) of the primary outcomes. Statistical significance was measured in 19 (35%) of the outcomes, of which nine (47%) were found to have a statistically significant effect. A minority of professionals (17%) judged their project as sustained by the department, while some stated that the intervention (37%) or the results of the project (11%) had spread. DISCUSSION: Although most projects indicated an improvement in their primary outcomes, only a few resulted in statistically significant changes. Teaching professionals in using evaluation methods that take into account the complex context where these projects are performed and teaching them leadership skills is needed to reduce the likelihood of unmeasured outcomes. Analyzing learning experiences of professionals in performing the project is important to see what they learned from performing quality improvement projects, providing experiences that may lead to sustainable effects in future projects.

Important factors for effective patient safety governance auditing: a questionnaire survey.

BACKGROUND: Audits are increasingly used for patient safety governance purposes. However, there is little insight into the factors that hinder or stimulate effective governance based on auditing. The aim of this study is to quantify the factors that influence effective auditing for hospital boards and executives. METHODS: A questionnaire of 32 factors was developed using influencing factors found in a qualitative study on effective auditing. Factors were divided into four categories. The questionnaire was sent to the board of directors, chief of medical staff, nursing officer, medical department head and director of the quality and safety department of 89 acute care hospitals in the Netherlands. RESULTS: We approached 522 people, of whom 211 responded. Of the 32 factors in the questionnaire, 30 factors had an agreement percentage higher than 50%. Important factors per category were 'audit as an improvement tool as well as a control tool', 'department is aware of audit purpose', 'quality of auditors' and 'learning culture at department'. We found 14 factors with a significant difference in agreement between stakeholders of at least 20%. Amongst these were 'medical specialist on the audit team', 'soft signals in the audit report', 'patients as auditors' and 'post-audit support'. CONCLUSION: We found 30 factors for effective auditing, which we synthesised into eight recommendations to optimise audits. Hospitals can use these recommendations as a framework for audits that enable boards to become more in control of patient safety in their hospital.

Evaluation of the organisation and effectiveness of internal audits to govern patient safety in hospitals: a mixed-methods study.

OBJECTIVES: Hospital boards are legally responsible for safe healthcare. They need tools to assist them in their task of governing patient safety. Almost every Dutch hospital performs internal audits, but the effectiveness of these audits for hospital governance has never been evaluated. The aim of this study is to evaluate the organisation of internal audits and their effectiveness for hospitals boards to govern patient safety. DESIGN AND SETTING: A mixed-methods study consisting of a questionnaire regarding the organisation of internal audits among all Dutch hospitals (n=89) and interviews with stakeholders regarding the audit process and experienced effectiveness of audits within six hospitals. RESULTS: Response rate of the questionnaire was 76% and 43 interviews were held. In every responding hospital, the internal audits followed the plan-do-check-act cycle. Every hospital used interviews, document analysis and site visits as input for the internal audit. Boards stated that effective aspects of internal audits were their multidisciplinary scope, their structured and in-depth approach, the usability to monitor improvement activities and to change hospital policy and the fact that results were used in meetings with staff and boards of supervisors. The qualitative methods (interviews and site visits) used in internal audits enable the identification of soft signals such as unsafe culture or communication and collaboration problems. Reported disadvantages were the low frequency of internal audits and the absence of soft signals in the actual audit reports. CONCLUSION: This study shows that internal audits are regarded as effective for patient safety governance, as they help boards to identify patient safety problems, proactively steer patient safety and inform boards of supervisors on the status of patient safety. The description of the Dutch internal audits makes these audits replicable to other healthcare organisations in different settings, enabling hospital boards to complement their systems to govern patient safety.

Improving the implementation of perioperative safety guidelines using a multifaceted intervention approach: protocol of the IMPROVE study, a stepped wedge cluster randomized trial.

BACKGROUND: This study is initiated to evaluate the effects, costs, and feasibility at the hospital and patient level of an evidence-based strategy to improve the use of Dutch perioperative safety guidelines. Based on current knowledge, expert opinions and expertise of the project team, a multifaceted implementation strategy has been developed. METHODS/DESIGN: This is a stepped wedge cluster randomized trial including nine representative hospitals across The Netherlands. Hospitals are stratified into three groups according to hospital type and geographical location and randomized in terms of the period for receipt of the intervention. All adult surgical patients meeting the inclusion criteria are assessed for patient outcomes. The implementation strategy includes education, audit and feedback, organizational interventions (e.g., local embedding of the guidelines), team-directed interventions (e.g., multi-professional team training), reminders, as well as patient-mediated interventions (e.g., patient safety cards). To tailor the implementation activities, we developed a questionnaire to identify barriers for effective guideline adherence, based on (a) a theoretical framework for classifying barriers and facilitators, (b) an instrument for measuring determinants of innovations, and (c) 19 semi-structured interviews with perioperative key professionals. Primary outcome is guideline adherence measured at the hospital (i.e., cluster) and patient levels by a set of perioperative Patient Safety Indicators (PSIs), which was developed parallel to the perioperative guidelines. Secondary outcomes at the patient level are in-hospital complications, postoperative wound infections and mortality, length of hospital stay, and unscheduled transfer to the intensive care unit, non-elective readmission to the hospital and unplanned reoperation, all within 30 days after the initial surgery. Also, patient safety culture and team climate will be studied as potential determinants. Finally, a process evaluation is conducted to identify the compliance with the implementation strategy, as well as an economic evaluation to assess the costs. Data sources are registered clinical data and surveys. There is no form of blinding. DISCUSSION: The perioperative setting is an unexplored area with respect to implementation issues. This study is expected to yield important new evidence about the effects of a multifaceted approach on guideline adherence in the perioperative care setting. TRIAL REGISTRATION: Dutch trial registry: NTR3568.

Evaluation of physicians' professional performance: an iterative development and validation study of multisource feedback instruments.

BACKGROUND: There is a global need to assess physicians' professional performance in actual clinical practice. Valid and reliable instruments are necessary to support these efforts. This study focuses on the reliability and validity, the influences of some sociodemographic biasing factors, associations between self and other evaluations, and the number of evaluations needed for reliable assessment of a physician based on the three instruments used for the multisource assessment of physicians' professional performance in the Netherlands. METHODS: This observational validation study of three instruments underlying multisource feedback (MSF) was set in 26 non-academic hospitals in the Netherlands. In total, 146 hospital-based physicians took part in the study. Each physician's professional performance was assessed by peers (physician colleagues), co-workers (including nurses, secretary assistants and other healthcare professionals) and patients. Physicians also completed a self-evaluation. Ratings of 864 peers, 894 co-workers and 1960 patients on MSF were available. We used principal components analysis and methods of classical test theory to evaluate the factor structure, reliability and validity of instruments. We used Pearson's correlation coefficient and linear mixed models to address other objectives. RESULTS: The peer, co-worker and patient instruments respectively had six factors, three factors and one factor with high internal consistencies (Cronbach's alpha 0.95 - 0.96). It appeared that only 2 percent of variance in the mean ratings could be attributed to biasing factors. Self-ratings were not correlated with peer, co-worker or patient ratings. However, ratings of peers, co-workers and patients were correlated. Five peer evaluations, five co-worker evaluations and 11 patient evaluations are required to achieve reliable results (reliability coefficient ≥ 0.70). CONCLUSIONS: The study demonstrated that the three MSF instruments produced reliable and valid data for evaluating physicians' professional performance in the Netherlands. Scores from peers, co-workers and patients were not correlated with self-evaluations. Future research should examine improvement of performance when using MSF.

Development of generic quality indicators for patient-centered cancer care by using a RAND modified Delphi method.

BACKGROUND: Despite growing attention to patient-centered care, the needs of cancer patients are not always met. OBJECTIVE: Using a RAND modified Delphi method, this study aimed to systematically develop evidence-based indicators, to be used to measure the quality of patient-centered cancer care as a first step toward improvement. METHODS: First, key recommendations were identified from literature and were distributed over 5 domains of patient-centered cancer care: communication, physical support, psychosocial care, after-care, and organization of care. Generic key recommendations, with best available evidence, were selected from guidelines. A multidisciplinary panel of patients and medical professionals (n = 14) rated and prioritized these recommendations in a written procedure. Subsequently, the panel discussed the recommendations at a consensus meeting. RESULTS: Key recommendations were identified for communication (n = 32), physical support (n = 13), psychosocial care (n = 25), after-care (n = 11), and organization of care (n = 11). For all domains, recommendations based on high-level evidence were identified except for after-care and physical support. The panel developed 17 indicators concerning criteria for communication and informed consent, evaluation of communication skills, provision of information, examination of emotional health, appointment of a care coordinator, physical complaints, follow-up, rehabilitation, psychosocial effects of waiting times, and self-management. CONCLUSIONS: A set of 17 indicators for patient-centered cancer care resulted from this study. Evidence support was available for most indicators. IMPLICATIONS FOR PRACTICE: This set provides an opportunity to measure and improve the quality of patient-centered cancer care. It is generic and therefore applies to many patients.

Development and measurement of guideline-based indicators for patients with non-Hodgkin's lymphoma.

PURPOSE: Patients with cancer are not always treated according to available guidelines. Factors such as age and comorbidities are frequently used as arguments for nonadherence. The aim of this study was to measure guideline adherence with guideline-based indicators for patients with non-Hodgkin's lymphoma (NHL) and to examine the need for improvement, considering relevant arguments. METHODS: A RAND-modified Delphi procedure was used to systematically develop NHL indicators. We evaluated their improvement potential (defined as < 90% score) in a random sample of patients with NHL (N = 431) diagnosed in 2006-2007 in 22 hospitals in the Netherlands with data from medical records. Multilevel logistic regression analyses were used to estimate the relationship between indicator scores and factors: comorbidity index (combined with age), stage, patient's objections, and lymphoma type. Scores were adjusted for significant factors. RESULTS: Of the 20 indicators developed, 16 had improvement potential. Scores were lowest for assessment of International Prognostic Index, 21%; imaging of neck, thorax, and abdomen and bone marrow examination during the diagnostic process, 23%, and after chemotherapy, 37%; adequate pathology reporting, 11%; and multidisciplinary discussion of patients, 21%. Scores for eight indicators were better for patients with a low Charlson index, stage III or IV disease, no objections to care, and aggressive lymphoma. After adjustments, adherence to all but one indicator (administration of the combination of rituximab and cyclophosphamide-doxorubicin-vincristine-prednisone) remained < 90%. CONCLUSION: In the Netherlands, almost all indicators for NHL needed improvement. This should be evaluated in other countries as well. International efforts should be undertaken to improve the quality of care of this often curable malignancy.

How to promote healthy behaviours in patients? An overview of evidence for behaviour change techniques.

To identify the evidence for the effectiveness of behaviour change techniques, when used by health-care professionals, in accomplishing health-promoting behaviours in patients. Reviews were used to extract data at a study level. A taxonomy was used to classify behaviour change techniques. We included 23 systematic reviews: 14 on smoking cessation, 6 on physical exercise, and 2 on healthy diets and 1 on both exercise and diets. None of the behaviour change techniques demonstrated clear effects in a convincing majority of the studies in which they were evaluated. Techniques targeting knowledge (n = 210 studies) and facilitation of behaviour (n = 172) were evaluated most frequently. However, self-monitoring of behaviour (positive effects in 56% of the studies), risk communication (52%) and use of social support (50%) were most often identified as effective. Insufficient insight into appropriateness of technique choice and quality of technique delivery hinder precise conclusions. Relatively, however, self-monitoring of behaviour, risk communication and use of social support are most effective. Health professionals should avoid thinking that providing knowledge, materials and professional support will be sufficient for patients to accomplish change and consider alternative strategies which may be more effective.

Peer mentoring in doctor performance assessment: strategies, obstacles and benefits.

CONTEXT: Mentors are increasingly involved in doctor performance assessments. Mentoring seems to be a key determinant in achieving the ultimate goal of those assessments, namely, improving doctor performance. Little is known, however, about how mentors perceive and fulfil this role. OBJECTIVE: The aim of this paper is to expand understanding of the role of mentors in performance assessment. METHODS: Thirty-eight mentors undertook formative performance assessments of their peers in a pilot study. A mixed-methods design was used, consisting of a postal survey (n = 28) and qualitative interviews with a subset of mentors (n = 11). Individual semi-structured interviews were completed and transcripts were analysed by two researchers using a grounded theory approach. RESULTS: The results of the survey showed that 89% of mentors intended to continue in their mentorship role. Interviews revealed that mentors used several strategies in the assessments, including: contrasting and collating information; posing reflective questions, and goal setting. Mentors experienced difficulty in disregarding their views of the doctors evaluated. Some mentors noticed obstacles with specific interview skills such as 'paying attention to their colleagues' strengths' and 'enabling doctors to find their own solutions'. Mentors reported that they and their organisations benefited from the assessments. The perceived benefits included: improved interview skills; increased solidarity, and increased mutual respect. CONCLUSIONS: The study provides insights into what mentors can do to increase the chance that externally derived information is integrated into doctors' self-assessments. Mainly, mentors used strategies aimed at effectively delivering feedback and encouraging reflection. However, we found that mentors who took part in our study appeared to struggle with a number of obstacles related to: time investment; familiarity with the doctor assessed, and the acquiring of specific interview skills.

Impact of an integrated care program for patients with head and neck cancer on the quality of care.

BACKGROUND: The management of patients with head and neck cancer is complex, and implementation of an integrated care program might improve the quality of care. METHODS: A prospective before-after study was performed in 1 clinic for head and neck oncology on 311 adults with head and neck cancer to evaluate an integrated care program. RESULTS: Scores on the integrated care indicators showed that the implementation of the integrated care program led to relevant improvements, eg, waiting time for diagnostic procedures less than 10 days (improvement of 37%), support for stopping smoking (+37%), nutrition support (+44%), assessment of CT and MRI scans by a an expert radiologist (+23%), and number of patients in contact with the specialist nurses (+37%). The program had no relevant effects on the outcome indicators. CONCLUSION: An integrated care program can improve several aspects of the management of patients with head and neck cancer.

Doctor performance assessment in daily practise: does it help doctors or not? A systematic review.

CONTEXT: Continuous assessment of individual performance of doctors is crucial for life-long learning and quality of care. Policy-makers and health educators should have good insights into the strengths and weaknesses of the methods available. The aim of this study was to systematically evaluate the feasibility of methods, the psychometric properties of instruments that are especially important for summative assessments, and the effectiveness of methods serving formative assessments used in routine practise to assess the performance of individual doctors. METHODS: We searched the MEDLINE (1966-January 2006), PsychINFO (1972-January 2006), CINAHL (1982-January 2006), EMBASE (1980-January 2006) and Cochrane (1966-2006) databases for English language articles, and supplemented this with a hand-search of reference lists of relevant studies and bibliographies of review articles. Studies that aimed to assess the performance of individual doctors in routine practise were included. Two reviewers independently abstracted data regarding study design, setting and findings related to reliability, validity, feasibility and effectiveness using a standard data abstraction form. RESULTS: A total of 64 articles met our inclusion criteria. We observed 6 different methods of evaluating performance: simulated patients; video observation; direct observation; peer assessment; audit of medical records, and portfolio or appraisal. Peer assessment is the most feasible method in terms of costs and time. Little psychometric assessment of the instruments has been undertaken so far. Effectiveness of formative assessments is poorly studied. All systems but 2 rely on a single method to assess performance. DISCUSSION: There is substantial potential to assess performance of doctors in routine practise. The longterm impact and effectiveness of formative performance assessments on education and quality of care remains hardly known. Future research designs need to pay special attention to unmasking effectiveness in terms of performance improvement.

Quality of integrated care for patients with nonsmall cell lung cancer: variations and determinants of care.

BACKGROUND: In the current study, the authors focused on determinants influencing the quality of care and variations in the actual quality of integrated care for patients with nonsmall cell lung cancer (NSCLC) to estimate whether there is room for improvement. METHODS: The authors tested the quality of integrated care for 276 NSCLC patients with 14 quality indicators of professional (4 indicators), organizational (3 indicators), and patient-oriented quality (7 indicators). Patient characteristics and actual care data were derived from medical record data, patient-oriented care was derived from patient questionnaires, and professional and hospital characteristics were derived from questionnaires for professionals. The performance measure was the proportion of patients to whom the indicator applied who had positive scores on the indicator. Multilevel logistic regression analysis determined the influence of patient, professional, and hospital characteristics on care. RESULTS: With regard to professional quality, the proportions of patients who underwent fluorodeoxyglucose-positron emission tomography or cervical mediastinoscopy according to the guideline criteria were 88% and 84%, respectively. Only 50% of the biopsies were adequately obtained during mediastinoscopy, and in 3% of the patients with clinical stage III disease (based on the TNM classification) there was a search for brain metastases before the initiation of combination therapy. With regard to organizational quality, the diagnostic route of 79% of the patients was completed within 21 days; 51% of patients began therapy within 35 days and 57% were discussed during multidisciplinary consultation. All but 1 patient-oriented quality indicator scored /=20% with regard to 11 of the 14 indicators. The patient-related determinants "stage of disease," "age," and "comorbidity" were found to influence the indicator scores the most. CONCLUSIONS: The quality of integrated care (especially patient-oriented care) for NSCLC patients needs improvement. Patient characteristics appear to influence performance more than professional or hospital characteristics.

Quality of integrated care for patients with head and neck cancer: Development and measurement of clinical indicators.

BACKGROUND: To improve the quality of integrated care, we developed indicators for assessing current practice in a large reference center for head and neck oncology. METHODS: We defined a set of indicators based on integrated care literature, national evidence-based guidelines for patients with head and neck cancer, and the opinions of professionals and patients. We tested this set regarding assessment of current practice and clinimetric characteristics. RESULTS: The final set consisted of 8 integrated care indicators and 23 specific indicators for patients with head and neck cancer. Current practice assessment produced high scores for the integrated care indicators, but the specific indicators showed room for improvement. The practice test showed that 9 indicators had good applicability. CONCLUSIONS: The indicators, while based on evidence-based guidelines and the principles of integrated care, should incorporate patients' opinions and include a practice test. Our results show that the quality of integrated care for patients with head and neck cancer could be improved.