Evaluation of the culture of safety: survey of clinicians and managers in an academic medical center.

BACKGROUND: Despite the emphasis on patient safety in health care, few organizations have evaluated the extent to which safety is a strategic priority or their culture supports patient safety. In response to the Institute of Medicine's report and to an organizational commitment to patient safety, we conducted a systematic assessment of safety at the Johns Hopkins Hospital (JHH) and, from this, developed a strategic plan to improve safety. The specific aims of this study were to evaluate the extent to which the culture supports patient safety at JHH and the extent to which safety is a strategic priority. METHODS: During July and August 2001 we implemented two surveys in disparate populations to assess patient safety. The Safety Climate Scale (SCS) was administered to a sample of physicians, nurses, pharmacists, and other ICU staff. SCS assesses perceptions of a strong and proactive organizational commitment to patient safety. The second survey instrument, called Strategies for Leadership (SLS), evaluated the extent to which safety was a strategic priority for the organization. This survey was administered to clinical and administrative leaders. RESULTS: We received 395 completed SCS surveys from 82% of the departments and 86% of the nursing units. Staff perceived that supervisors had a greater commitment to safety than senior leaders. Nurses had higher scores than physicians for perceptions of safety. Twenty three completed SLS surveys were received from 77% of the JHH Patient Safety Committee members and 50% of the JHH Management Committee members. Management Committee responses were more positive than Patient Safety Committee, indicating that management perceived safety efforts to be further developed. Strategic planning received the lowest scores from both committees. CONCLUSIONS: We believe this is one of the first large scale efforts to measure institutional culture of safety and then design improvements in health care. The survey results suggest that strategic planning of patient safety needs enhancement. Several efforts to improve our culture of safety were initiated based on these results, which should lead to measurable improvements in patient safety.

Reasons for pediatrician nonadherence to asthma guidelines.

BACKGROUND: The 1997 National Heart, Lung, and Blood Institute (NHLBI) asthma guidelines include recommendations on how to improve the quality of care for asthma. OBJECTIVE: To identify barriers to physician adherence to the NHLBI guidelines. DESIGN: Cross-sectional survey. PARTICIPANTS: A national random sample of 829 primary care pediatricians. MAIN OUTCOME MEASURES: Self-reported adherence to 4 components of the NHLBI guidelines (steroid prescription, instructing peak flow meter use, screening and counseling patients with asthma for smoking, and screening and counseling parents for smoking). We also collected information on physician demographics, practice characteristics, and possible barriers to adherence. We defined adherence as following a guideline component more than 90% of the time. RESULTS: The response rate was 55% (456/829). Most of the responding pediatricians were aware of the guidelines (88%) and reported having access to a copy of the guidelines (81%). Self-reported rates of adherence were between 39% and 53% for the guideline components. After controlling for demographics and other barriers, we found that nonadherence was associated with specific barriers for each guideline component: for corticosteroid prescription, lack of agreement (odds ratio [OR], 6.8; 95% confidence interval [CI], 3.2-14.4); for peak flow meter use, lack of self-efficacy (OR, 3.4; 95% CI, 1.9-6.1) and lack of outcome expectancy (OR, 4.7; 95% CI, 2.5-8.9); and for screening and counseling of patients and parents for smoking, lack of self-efficacy (OR, 3.8; 95% CI, 1.7-6.2 and OR, 2.8; 95% CI, 1.3-5.9, respectively). CONCLUSIONS: Although pediatricians in this sample were aware of the NHLBI guidelines, a variety of barriers precluded their successful use. To improve NHLBI guideline adherence, tailored interventions that address the barriers characteristic of a given guideline component need to be implemented.

Benchmarking applications: linking state strategic planning, quality improvement, and consumer reporting.

This article demonstrates the value of using benchmark patient satisfaction data for Medicaid program quality improvement. The authors compare surveys of Maryland Medicaid and federal employees in Maryland, utilizing the latter as an external benchmark. Unadjusted and adjusted analyses found a significantly lower percentage of Medicaid than federal respondents rated telephone access excellent, very good, or good, whereas more Medicaid respondents rated advice on prevention and choice of primary care doctor highly. Patient satisfaction external benchmark data provide managed care organizations (MCOs) and state policy makers with goals to improve quality and standards to measure care objectively in vulnerable populations.

Developing a health-related quality-of-life measure for end-stage renal disease: The CHOICE Health Experience Questionnaire.

The Choices for Healthy Outcomes in Caring for End-Stage Renal Disease ([ESRD] CHOICE) Study was designed to evaluate the effectiveness of alternative dialysis prescriptions. As part of CHOICE, we developed an instrument for measuring health-related quality of life (HRQOL) for patients with ESRD that would complement the Medical Outcomes Study 36-Item Short-Form Survey (SF-36) and be sensitive to differences in dialysis modality (hemodialysis [HD] and peritoneal dialysis [PD]) and dialysis dose. The selection of HRQOL domains to be included was based on: (1) a structured literature review of 47 articles describing 53 different instruments; (2) content analysis of five focus groups with HD and PD patients, nephrologists, and other providers; (3) a survey of 110 dialysis providers about features of different modalities that affect patient HRQOL; and (4) a semistructured survey of 25 patients with ESRD on the effects of dialysis on functioning and HRQOL. To help prioritize domains and items identified by these methods, a representative sample of 136 dialysis patients rated each item for frequency and bother. A panel of nephrologists provided advice about the salience of items to modality or dose. Items and scales were selected with a preference for existing measures tested in patients with ESRD and were tested for reliability and validity. The first four steps yielded 22 HRQOL domains that included 96 items: 8 generic domains in the SF-36 (health perceptions, physical, social, physical and emotional role function, pain, mental health, and energy); 8 additional generic domains (cognitive functioning, sexual functioning, sleep, work, recreation, travel, finances, and general quality of life); and 6 ESRD-specific domains (diet, freedom, time, body image, dialysis access [catheters and/or vascular], and symptoms). New items were developed or adapted to assess ESRD-specific domains. Scales for these items showed adequate internal consistency (Cronbach's alpha > 0.70, except for time [alpha = 0.57] and quality of life [alpha = 0.68]), as well as convergent and discriminant construct validity in a sample of 928 patients. The final questionnaire included 21 domains (time was deleted) and 83 items. We have designed a patient-centered instrument, the CHOICE Health Experience Questionnaire, that addresses domains that may be sensitive to differences in dialysis modality and dose and shows evidence for reliability and validity as a measure of HRQOL in ESRD.

The advantages and disadvantages of process-based measures of health care quality.

As consumers, payers, and regulatory agencies require evidence regarding health care qualities the demand for process of care measures will grow. Although outcome measures of quality represent the desired end results of health care, validated process of care measures provide an important additional element to quality improvement efforts, as they illuminate exactly which provider actions could be changed to improve patient outcomes. In this essay, we discuss the advantages and disadvantages of process measures of quality, and outline some practical strategies and issues in implementing them.

From a process of care to a measure: the development and testing of a quality indicator.

This paper outlines the steps in developing and implementing process measures of quality. Developing a process measure includes defining the purpose of and audiences for the measures, choosing the clinical area to evaluate, organizing the assessment team, choosing the component of the process to measure, writing the indicator specifications, performing preliminary tests of feasibility, reliability and validity, and determining scoring and analytical specifications. Given the growing evidence in the literature regarding the impact of care, and an evolving understanding of how to develop and implement process of care measures as outlined here, the future should bring the development and implementation of quality indicators that are rigorously developed and that will provide insights into opportunities to improve the quality of care.

Assessing the relationship between quality of care and the characteristics of health care organizations.

In the past two decades, relationships among health plans, medical groups, and providers have grown more complex and the number of clinical management strategies has increased. In this context, determining the independent effect of a particular organizational strategy on quality of care has become more difficult. The authors review some of the issues a researcher must address when studying the relationship between organizational characteristics and quality of care. They offer criteria for selecting a research question, list organizational characteristics that may influence quality, and suggest sampling and study design techniques to reduce confounding. Since this type of research often requires a health care organization as collaborator, the authors discuss strategies for developing research partnerships and collecting data from the partner organization. Finally, they offer suggestions for translating research into policy.

Cost-utility analysis of the cochlear implant in children.

CONTEXT: Barriers to the use of cochlear implants in children with profound deafness include device costs, difficulty assessing benefit, and lack of data to compare the implant with other medical interventions. OBJECTIVE: To determine the quality of life and cost consequences for deaf children who receive a cochlear implant. DESIGN: Cost-utility analysis using preintervention, postintervention, and cross-sectional surveys conducted from July 1998 to May 2000. SETTING: Hearing clinic at a US academic medical center. PARTICIPANTS: Parents of 78 profoundly deaf children (average age, 7.5 years) who received cochlear implants. MAIN OUTCOME MEASURES: Direct and total cost to society per quality-adjusted life-year (QALY) using the time-trade-off (TTO), visual analog scale (VAS), and Health Utilities Index-Mark III (HUI), discounting costs and benefits 3% annually. Parents rated their child's health state at the time of the survey and immediately before and 1 year before implantation. RESULTS: Recipients had an average of 1.9 years of implant use. Mean VAS scores increased by 0. 27, from 0.59 before implantation to 0.86 at survey. In a subset of participants, TTO scores increased by 0.22, from 0.75 to 0.97 (n = 40) and HUI scores increased by 0.39, from 0.25 to 0.64 (n = 22). Quality-of-life scores were no different 1 year before and immediately before implantation. Discounted direct costs were $60,228, yielding $9,029 per QALY using the TTO, $7,500 per QALY using the VAS, and $5,197 per QALY using the HUI. Including indirect costs such as reduced educational expenses, the cochlear implant provided a savings of $53,198 per child. CONCLUSIONS: Cochlear implants in profoundly deaf children have a positive effect on quality of life at reasonable direct costs and appear to result in a net savings to society. JAMA. 2000;284:850-856

Predictors of pain control in patients undergoing flexible bronchoscopy.

The purpose of this study was to assess the extent to which patients undergoing flexible bronchoscopy (FOB) experience pain and to identify patient factors and process of care factors that are associated with pain. We conducted a prospective cohort study on 481 patients undergoing FOB. Overall control of pain during FOB was the primary outcome. The mean age of the patients was 48 yr, 50% were male, and 32% required supplemental oxygen prior to FOB. Pain control was excellent in 36% of patients, but 10% considered it to be fair or poor. Patient factors associated with excellent pain control were excellent health (versus poor health, OR = 6.25 [95% CI, 2.28-16.67]), more education (college education versus high school education, OR = 1.72 [95% CI, 1.05-2.86]), and not having asthma (OR = 2.86 [95% CI, 1.09-7.14]). Process of care factors associated with excellent pain control were not being bothered by scope insertion (versus bothered, OR = 3.65 [95% CI, 1.99-6.98]), no memory of FOB (versus some memory, OR = 2.33 [95% CI, 1.24-4.44]), and higher ratings of information about the procedure (per 1-point increase on a 12-point scale, OR = 1.57 [95% CI, 1.41-1.78]). This is the first large-scale, prospective study to evaluate patient and process of care factors that influence pain control during FOB. It demonstrated that there are patient characteristics and process of care factors that need to be considered when evaluating pain during bronchoscopy. Improved preparation of patients with lower education, inferior health status, and asthma may lead to decreased pain during FOB. Bronchoscopists may be able to reduce pain during FOB by identifying methods to decrease pain on scope insertion, by improving the information provided to patients, and by achieving greater levels of amnesia during FOB.

Barriers pediatricians face when using asthma practice guidelines.

OBJECTIVE: To describe barriers to the successful use of the 1997 National Heart, Lung, and Blood Institute (NHLBI) asthma guidelines. METHODS: We conducted 3 focus groups to understand barriers to the use of 4 recommendations within the NHLBI guidelines (prescription of inhaled corticosteroids, recommendation of daily peak flowmeter use, smoking cessation screening and counseling, and allergen exposure counseling). PARTICIPANTS: Twenty-one pediatricians and 1 nurse practitioner, who each followed an average of 47 patients with asthma, participated. Six participants (27%) had a faculty or adjunct appointment at a medical school. Nineteen (90%) of the 21 pediatricians were board certified. RESULTS: We identified 171 comments about barriers to adherence. Type of recommendation and physician year of graduation from medical school were related to which barrier was prominent. For corticosteroid prescription, senior physicians mentioned lack of agreement, whereas younger physicians described lack of confidence in dosing or recognizing contraindications. For peak flow-meter use, senior physicians emphasized lack of training. Only senior physicians described the inertia of previous practice as a barrier. All groups mentioned time limitations. CONCLUSIONS: Efforts to improve adherence to asthma guidelines should consider the range of barriers that pediatricians face, such as lack of awareness, familiarity, or agreement, and external barriers owing to environmental, guideline, or patient factors. In addition, this study documents barriers not previously considered, such as lack of self-efficacy, lack of outcome expectancy, and inertia of previous practice, that prevent adherence. Because type of recommendation and physician demographics are related to which barriers are prominent, interventions to improve NHLBI guideline adherence should be tailored to these factors.

Family satisfaction with end-of-life care in seriously ill hospitalized adults.

OBJECTIVE: To examine factors associated with family satisfaction with end-of-life care in the Study to Understand Prognoses and Preferences for Outcomes and Risks of Treatments (SUPPORT). DESIGN: A prospective cohort study with patients randomized to either usual care or an intervention that included clinical nurse specialists to assist in symptom control and facilitation of communication and decision-making. SETTING: Five teaching hospitals in the United States. PARTICIPANTS: Family members and other surrogate respondents for 767 seriously ill hospitalized adults who died. MEASUREMENTS: Eight questionnaire items regarding satisfaction with the patient's medical care expressed as two scores, one measuring satisfaction with patient comfort and the other measuring satisfaction with communication and decision-making. RESULTS: Sixteen percent of respondents reported dissatisfaction with patient comfort and 30% reported dissatisfaction with communication and decision-making. Factors found to be significantly associated with satisfaction with communication and decision-making were hospital site, whether death occurred during the index hospitalization (adjusted odds ratio (AOR) 2.2, 95% CI, 1.3-3.9), and for patients who died following discharge, whether the patient received the SUPPORT intervention (AOR 2.0, 1.2-3.2). For satisfaction with comfort, male surrogates reported less satisfaction (0.6, 0.4-1.0), surrogates who reported patients' preferences were followed moderately to not at all had less satisfaction (0.2, 0.1-0.4), and surrogates who reported the patient's illness had greater effect on family finances had less satisfaction (0.4, 0.2-0.8). CONCLUSIONS: Satisfaction scores suggest the need for improvement in end-of-life care, especially in communication and decision making. Further research is needed to understand how factors affect satisfaction with end-of-life care. An intervention like that used in SUPPORT may help family members.

Utility of on-site cytopathology assessment for bronchoscopic evaluation of lung masses and adenopathy.

STUDY OBJECTIVES: To determine the extent to which on-site cytopathology assessment improves diagnostic yield when sampling lung nodules or masses and/or hilar or mediastinal lymphadenopathy by fiberoptic bronchoscopy (FOB). DESIGN: Prospective cohort study. SETTING: Two teaching hospitals in Baltimore, MD. PATIENTS: Consecutive adult patients (>/= 18 years) undergoing FOB for evaluation of lung nodules or masses and/or hilar or mediastinal lymphadenopathy. INTERVENTION: Prospective collection of data on patient factors and details of the procedure on standardized report forms. MEASUREMENTS AND RESULTS: The primary outcome measure was a new diagnosis obtained by FOB. On-site assessment was used in 81 of 204 cases (40%), and overall diagnostic yield was 62%. Yield was greater when on-site cytopathology assessment was used, in unadjusted analysis (81% vs 50%, p < 0.001) and in a multivariate model (odds ratio, 4.5; 95% confidence interval, 2.1 to 10.0). Other significant predictors of a new diagnosis included older patient age, higher dose of narcotic used during FOB, and shorter procedure time. CONCLUSIONS: We conclude that diagnostic yield was greater when on-site cytopathology was used to assist FOB evaluation of intrathoracic adenopathy and/or lung nodules or masses. Increasing the use of on-site cytopathology assessment may improve the quality of FOB services.

The effect of advance care planning on completion of advance directives and patient satisfaction in people with HIV/AIDS.

The effects of advance care planning are poorly understood. The purpose of this study was to evaluate the effect of an advance care planning intervention on the completion of advance directives (ADs) and patient satisfaction. A volunteer sample of persons with HIV/AIDS received advance directive documents, watched an educational video and received individual counselling on completing an advance directive during three face-to-face interviews over approximately six months. The advance care planning intervention was associated with an increase in advance directive completion rates from 16.4% to 40.7% (p = 0.001), but 23.1% of advance directives reported as completed were legally invalid. There was a trend towards decreased overall patient satisfaction with health care (p = 0.07). Advance are planning increases the rate of AD completion but many 'completed' advance directives are legally invalid. Advance care planning did not improve patient satisfaction with health care.

Use of a clustered model to identify factors affecting hospital length of stay.

Predictive models have been used to identify factors that may prolong hospital length of stay (LOS). However, because predictors of LOS are collinear, the proportion of variance associated with each factor in a multivariate stepwise regression model may not reflect its mathematical contribution in explaining LOS. In an attempt to model factor contribution to LOS more realistically, we evaluated a clinically based clustered model. This model uses classes of candidate predictors, that is, patient attributes, adverse events, treatment modality, and health provider identity. Clusters of variables are permitted to enter into the model in a theoretically based predetermined sequence, so that the additional contribution of each cluster of factors can be assessed while the contribution of preceding factors is preserved. The clustered model was tested and compared with a free stepwise multivariate analysis in a cohort of patients undergoing prostatectomy for benign prostatic hypertrophy. We found that both models explained a similar proportion of the variance in LOS (56%-57%). However, some important differences were evident. Prostate size, associated with 12% of the variance in the clustered model, was not an independent predictor in the free model. A higher proportion of variance was associated with process variables, such as treatment modality in the free model. We conclude that use of a clustered model may facilitate more realistic assessment of the relative contribution of factors to LOS.

Why don't physicians follow clinical practice guidelines? A framework for improvement.

CONTEXT: Despite wide promulgation, clinical practice guidelines have had limited effect on changing physician behavior. Little is known about the process and factors involved in changing physician practices in response to guidelines. OBJECTIVE: To review barriers to physician adherence to clinical practice guidelines. DATA SOURCES: We searched the MEDLINE, Educational Resources Information Center (ERIC), and HealthSTAR databases (January 1966 to January 1998); bibliographies; textbooks on health behavior or public health; and references supplied by experts to find English-language article titles that describe barriers to guideline adherence. STUDY SELECTION: Of 5658 articles initially identified, we selected 76 published studies describing at least 1 barrier to adherence to clinical practice guidelines, practice parameters, clinical policies, or national consensus statements. One investigator screened titles to identify candidate articles, then 2 investigators independently reviewed the texts to exclude articles that did not match the criteria. Differences were resolved by consensus with a third investigator. DATA EXTRACTION: Two investigators organized barriers to adherence into a framework according to their effect on physician knowledge, attitudes, or behavior. This organization was validated by 3 additional investigators. DATA SYNTHESIS: The 76 articles included 120 different surveys investigating 293 potential barriers to physician guideline adherence, including awareness (n = 46), familiarity(n = 31), agreement (n = 33), self-efficacy (n = 19), outcome expectancy (n = 8), ability to overcome the inertia of previous practice (n = 14), and absence of external barriers to perform recommendations (n = 34). The majority of surveys (70 [58%] of 120) examined only 1 type of barrier. CONCLUSIONS: Studies on improving physician guideline adherence may not be generalizable, since barriers in one setting may not be present in another. Our review offers a differential diagnosis for why physicians do not follow practice guidelines, as well as a rational approach toward improving guideline adherence and a framework for future research.

Review of studies that compare the quality of cardiovascular care in HMO versus non-HMO settings.

OBJECTIVES: The authors compared the quality of cardiovascular care in health maintenance organizations (HMOs) versus traditional insurance arrangements through an analysis of existing literature. METHODS: Data were derived from all peer-reviewed studies published through November 1995 that used process or outcome measures to evaluate the quality of cardiovascular care in HMO versus non-HMO settings. A standardized form was used to extract information from each study on: condition studied, study time frame, type of study design, type of comparison groups, characteristics of patients and physicians, process and outcome measures used, data collection methods, reliability and validity of quality measurements, risk adjustment techniques, findings about quality of care, summary of other findings, study limitations, and other comments that explained the context of the research. RESULTS: Seven of the 11 studies that examined process measures for cardiovascular care in HMO versus non-HMO patients found more differences in one or more process measures that favored HMOs than non-HMOs. Seven of the 10 studies that examined outcome measures found no statistically significant differences in patient care between HMO and non-HMO settings. The other three studies presented contradictory results. CONCLUSIONS: The existing literature suggests that the outcomes of care for cardiovascular conditions do not differ between HMO and non-HMO settings, although selected measures of the process of cardiovascular care are actually better in HMO than in non-HMO settings.

Quality assessment through patient self-report of symptoms prefiberoptic and postfiberoptic bronchoscopy.

STUDY OBJECTIVE: To apply the principles of quality improvement to measure the frequency and severity of symptoms that result from fiberoptic bronchoscopy (FOB), and to identify opportunities to improve FOB practice by identifying factors about patients and the process of care that predict these symptoms. DESIGN: Concurrent longitudinal cohort study. PATIENTS: Four hundred ninety-three adult patients who underwent FOB. MEASUREMENTS AND RESULTS: Patients completed questionnaires just prior to FOB and again at 48 h postprocedure. Patients were asked to rate the severity of nose pain, throat pain, swallowing pain, and chest pain, and the frequency of coughing, hemoptysis, phlegm, shortness of breath, wheezing, difficulty swallowing, fever, and chills. Symptom severity was reported on a four-point ordinal scale. FINDINGS: Significant worsening was found for nose pain, throat pain, swallowing pain, and hemoptysis. Shorter patients experienced more throat pain and hemoptysis, and longer procedure time predicted nose pain and hemoptysis. CONCLUSIONS: Bronchoscopy causes nose pain, throat pain, swallowing pain, and hemoptysis to a larger extent than previously has been recognized. There are opportunities to improve the patient experience with bronchoscopy by using smaller bronchoscopes in shorter patients, shortening the procedure length, and reanesthetizing the nares in longer procedures.

Assessment of the validity and reliability of three systems of medical record screening for quality of care assessment.

OBJECTIVES: The authors compared the validity and reliability of three methods of medical record screening for detecting problems with quality of care of patients undergoing cardiovascular procedures: (1) the Uniform Clinical Data Set System (UCDSS), a software-based system developed by the Health Care Financing Administration (HCFA), (2) the HCFA generic screens, and (3) the Harvard Medical Practice Study (HMPS) screening criteria. METHODS: A random sample of 451 medical records of inpatients who underwent cardiovascular procedures were screened by trained nurses and were also examined by physicians masked to screening results. Physicians' judgment provided a reference standard for validity assessment of screening applied by nurses. RESULTS: Agreement between pairs of physician reviews about the quality of care was fair or slight (Kappa < 0.40). Agreement between nurses about presence of any screening flags was excellent for UCDSS (0.93), slight for HCFA (0.11), and fair for HMPS (0.41). The UCDSS was more sensitive to quality problems agreed on by two physicians than HCFA or HMPS (85%, 74%, and 69%, respectively), but was also less specific (48%, 57%, and 56%, respectively). The UCDSS was much more labor-intensive with an average of 6.7 hours of nurse review per quality problem disclosed versus 2 hours required by HCFA and HMPS. CONCLUSIONS: The UCDSS computerized screens brought about substantial improvements in reliability of medical record screening, but needed to reduce its false-positive rate and to increase efficiency before it is used. From the perspective of continuous quality improvement, the UCDSS computerized algorithms provided a starting point to developing disease-specific patterns of practice to guide interventions.