Contributing factors identified by hospital incident report narratives.

CONTEXT: A major purpose of incident reporting is to understand contributing factors so that causes of errors can be uncovered and systems made safer. For established reporting systems in US hospitals, little is known about how well the reports identify contributing factors. OBJECTIVE: To characterise the information incident report narratives provide about contributing factors using a taxonomy we developed for this purpose. DESIGN: Descriptive study examining 2228 reports for 16 575 randomly selected patients discharged from an academic and a community hospital in the US between 1 January and 31 December 2001. MAIN OUTCOMES MEASURED: Reports in which patient, system and provider (errors, mistakes and violations) factors were identifiable. RESULTS: 80% of reports described at least one contributing factor. Patient factors were identifiable in 32%, most frequently illness (61% of these reports) and behaviour (24%). System factors were identifiable in 32%, most commonly equipment malfunction or difficulty of use (38%), problems coordinating care among providers (31%), provider unavailability (24%) and tasks that were difficult to execute correctly (20%). Provider factors were evident in 46%, but half of these reports contained insufficient detail to determine which specific factor. When detail sufficed, slips (52%), exceptional violations (22%), lapses (15%) and applying incorrect rules (13%) were common. CONCLUSIONS: Contributing factors could be identified in most incident-report narratives from these hospitals. However, each category of factors was present in a minority of reports, and provider factors were often insufficently elucidated. Greater detail about contributing factors would make incident reports more useful for improving patient safety.

Underuse of cardiac procedures: do women, ethnic minorities, and the uninsured fail to receive needed revascularization?

BACKGROUND: Women, ethnic minorities, and uninsured persons receive fewer cardiac procedures than affluent white male patients do, but rates of use are crude indicators of quality. The important question is, Do women, minorities, and the uninsured fail to receive cardiac procedures when they need them? OBJECTIVE: To measure receipt of necessary coronary artery bypass graft (CABG) surgery and percutaneous transluminal coronary angioplasty (PTCA) overall; by patient sex, ethnicity, and payer status; and by availability of on-site revascularization. DESIGN: Retrospective, randomized medical record review. SETTING: 13 of the 24 hospitals in New York City that provide coronary angiography. PATIENTS: 631 patients who had coronary angiography in 1992 and met the RAND expert panel criteria for necessary revascularization. MEASUREMENTS: The percentage of patients who had CABG surgery or PTCA was measured, as were variations in use rates by sex, ethnic group, insurance status, and availability of on-site revascularization. Clinical and laboratory data were retrieved from medical records to identify patients who met the panel criteria for necessary revascularization. Receipt of revascularization was determined from state reports, medical records, and information provided by cardiologists. RESULTS: Overall, 74% (95% CI, 71% to 77%) of patients who met the panel criteria for necessary revascularization had CABG surgery or PTCA (underuse rate, 26%). No differences were found in use rates by patient sex, ethnic group, or payer status, but hospitals that provided on-site revascularization had higher use rates (76% [CI, 74% to 79%]) than hospitals that did not provide it (59% [CI, 56% to 65%]) (P < 0.01). In hospitals that did not provide on-site revascularization, uninsured patients were less likely to have revascularization recommended to them (52% [CI, 32% to 78%]); rates of recommendation for patients with private insurance, Medicare, and Medicaid were 82%, 91%, and 75%, respectively (P = 0.026). CONCLUSIONS: Although revascularization procedures are substantially underused, no variations in rate of use by sex, ethnic group, or payer status were seen among patients treated in hospitals that provide CABG surgery and PTCA. However, underuse was significantly greater in hospitals that do not provide these procedures, particularly among uninsured persons.

Are physicians' office laboratory results of comparable quality to those produced in other laboratory settings?

CONTEXT: In 1995, California adopted a bill that brought laboratory laws in line with the 1988 Clinical Laboratory Improvement Amendments' standards for clinical laboratories and mandated a study comparing results in physicians' office laboratories (POLs) with other settings. OBJECTIVE: To determine whether persons conducting tests in POLs produce accurate and reliable test results comparable to those produced by non-POLs. DESIGN: Survey of clinical laboratories using proficiency testing data. SETTING: All California clinical laboratories participating in the American Association of Bioanalysts proficiency testing program in 1996 (n=1110). MAIN OUTCOME MEASURES: "Unsatisfactory" (single testing event failure) and "unsuccessful" (repeated testing event failure) on proficiency testing samples. RESULTS: The unsatisfactory failure rate for POLs was nearly 3 times (21.5% vs 8.1%) the rate for the non-POLs and about 1.5 times (21.5% vs 14.0%) for POLs that used laboratory professionals as testing or supervisory personnel (P<.001). The POL unsuccessful rate was more than 4 times (4.4% vs 0.9%) the rate for non-POLs and more than twice (4.4% vs 1.8%) the rate for the POLs using laboratory professionals (P<.001). CONCLUSIONS: Significant differences exist among POLs, POLs using licensed clinical laboratory scientists (medical technologists), and non-POLs. Testing personnel in many POLs might lack the necessary education, training, and oversight common to larger facilities. We must better understand the contributing factors that result in the poorer results of POLs relative to non-POLs. In the meantime, patients should be aware that preliminary findings suggest that differences in quality of laboratory tests based on testing site may exist. Laboratory directors at all testing sites must ensure that they understand laboratory practice sufficiently to minimize errors and maximize accuracy and reliability. Directors must understand their obligation when they elect to oversee those assigned testing responsibility. Legislators may wish to reconsider the wisdom of further easing restrictions on those to whom we entrust our laboratory specimens.

Waiting for coronary revascularization: a comparison between New York State, The Netherlands and Sweden.

OBJECTIVE: To compare waiting times for percutaneous transluminal coronary angioplasty (PTCA) and coronary artery bypass graft (CABG) surgery in New York State, the Netherlands and Sweden and to determine whether queuing adversely affects patients' health. METHODS: We reviewed the medical records of 4487 chronic stable angina patients who underwent PTCA or CABG in one of 15 New York State hospitals (n = 1021) or were referred for PTCA or CABG to one of ten hospitals in the Netherlands (n = 1980) or to one of seven hospitals in Sweden (n = 1486). We measured the median waiting time between coronary angiography and PTCA or CABG. RESULTS: The median waiting time for PTCA in New York was 13 days compared with 35 and 42 days, respectively, in the Netherlands and Sweden (P < 0.001). For CABG, New York patients waited 17 days, while Dutch and Swedish patients waited 72 and 59 days, respectively (P < 0.001). The Swedish and Dutch waiting list mortality rate was 0.8% for CABG candidates and 0.15% for PTCA candidates. CONCLUSIONS: There were large variations in waiting time for coronary revascularization among these three sites. Patients waiting for CABG were at greatest risk of experiencing an adverse event. In both the Netherlands and Sweden, the capacity to perform coronary revascularization has been expanded since this study began. Further international cooperation may identify other areas where quality of care can be improved.

The appropriateness of coronary artery bypass graft surgery in academic medical centers. Working Group of the Appropriateness Project of the Academic Medical Center Consortium.

OBJECTIVE: To compare the appropriateness of use of coronary artery bypass graft (CABG) surgery in Academic Medical Center Consortium hospitals as judged 1) according to criteria developed by an expert panel, 2) according to revisions of those criteria made by cardiac surgeons from the Academic Medical Center Consortium, and 3) by review of cases by the surgeons responsible for those cases. DESIGN: Retrospective, randomized medical record review. SETTING: 12 Academic Medical Center Consortium hospitals. PATIENTS: Random sample of 1156 patients who had had isolated CABG surgery in 1990. MAIN OUTCOME MEASURES: 1) Percentage of patients with indications for which CABG surgery was classified as appropriate, Inappropriate, or of uncertain appropriateness and 2) percentage of cases in which CABG surgery was judged inappropriate or uncertain for which ratings changed after local case review. RESULTS: Data were retrieved from medical records by trained abstractors using an explicit data collection instrument. Cases in which CABG surgery was judged to be inappropriate or uncertain were individually reviewed by the responsible surgeons. According to the expert panel ratings, 83% of the CABG operations (95% CI, 81% to 85%) were necessary, 9% (CI, 8% to 10%) were appropriate, 7% (CI, 5% to 8%) were uncertain, and 1.6% (CI, 0.6% to 2.5%) were inappropriate. These rates are almost identical to those found in a previous study that was done in New York State and that used the same criteria (in that study, 91% of operations were classified as necessary or appropriate, 7% were classified as uncertain, and 2.4% were classified as inappropriate). Rates of inappropriate procedures varied from 0% to 5% among the 12 member hospitals (P = 0.02). The Academic Medical Center Consortium cardiac surgeons revised 568 (24%) of the indications used by the expert panel. However, because those revisions altered the appropriateness ratings in both directions and affected only 50 cases (4%), the net effect of the revisions was slight: The rate of inappropriate CABG surgery increased from 1.6% to 1.9%. Local review found that data collection errors had caused erroneous ratings in 12.5% of 64 cases in which surgery had been classified as inappropriate or uncertain. CONCLUSIONS: The Academic Medical Center Consortium hospitals had low rates of inappropriate and uncertain use of CABG surgery, regardless of the criteria used for assessment. Even though surgeons from the Consortium revised the appropriateness ratings extensively, their revisions had a negligible effect on the overall assessment of appropriateness. However, because of potential data collection errors, appropriateness criteria should be used for individual case audits only if supplemented by subsequent physician review.

Variations by specialty in physician ratings of the appropriateness and necessity of indications for procedures.

The authors compare the appropriateness ratings and mutual influence of panelists from different specialties rating a comprehensive set of indications for six surgical procedures. Nine-member panels rated each procedure: abdominal aortic aneurysm surgery, carotid endarterectomy, cataract surgery, coronary angiography, and coronary artery bypass graft surgery/percutaneous transluminal coronary angioplasty (common panel). Panelists individually rated the appropriateness of indications at home and then discussed and re-rated the indications during a 2-day meeting. Subsequently, they rated the necessity of those indications scored by the group as appropriate. There were 45 panelists, including specialists (either performers of the procedure or members of a related specialty) and primary care providers, all drawn from nominations by their respective specialty societies. Main outcome measures included: individual panelists' mean ratings over all indications, mean change and conformity scores between rounds of ratings, and the percentage of audited actual procedures rated appropriate or necessary. Performers had the highest mean ratings, followed by physicians in related specialties, trailed by primary care providers. One fifth of all actual procedures were for indications rated appropriate by performers and less than appropriate by primary care providers. At the panel meetings, primary care providers and related specialists showed no greater tendency to be influenced by other panelists than did performers. Multispecialty panels provide more divergent viewpoints than panels composed entirely of performers. This divergence means that fewer actual procedures are deemed performed for appropriate or necessary indications.

The appropriateness of use of cardiovascular procedures in women and men.

OBJECTIVE: To determine whether there are differences between women and men in the appropriateness of use of cardiovascular procedures. DESIGN: Retrospective chart review. SETTING: Thirty hospitals located in New York State. PATIENTS: Random sample of 3979 patients undergoing coronary angiography, percutaneous transluminal coronary angioplasty, or coronary artery bypass graft surgery in 1990. MEASURES: We evaluated two measures: (1) the percent of women and men who underwent cardiovascular procedures for appropriate, uncertain, and inappropriate indications and (2) for coronary angiography patients, the prognostic exercise stress treadmill score that predicts before the coronary angiogram the 5-year probability of death from a cardiovascular event. RESULTS: The inappropriate rate of use of cardiovascular procedures was low and not significantly different for men and women (4% vs 5% for coronary angiography; 4% vs 3% for percutaneous transluminal coronary angioplasty; and 2% vs 3% for coronary artery bypass graft surgery, respectively), and the use of these procedures for uncertain reasons also did not vary significantly by gender. There was also no significant gender difference in the predicted risk of death from a cardiovascular event for coronary angiography patients: 24% of men and 22% of women were at high risk (ie, < 75% 5-year survival rate) and 20% and 16%, respectively, were at low risk (ie, > or = 95% 5-year survival rate). CONCLUSION: Based on two indicators, the RAND appropriateness score and the Duke prognostic exercise treadmill score, we were unable to find any evidence of a difference in the clinical appropriateness of use of these three cardiovascular procedures between women and men.

Comparison of the appropriateness of coronary angiography and coronary artery bypass graft surgery between Canada and New York State.

OBJECTIVE: To compare the appropriateness of coronary angiography and coronary artery bypass graft (CABG) use between the United States and Canada. DESIGN: Retrospective randomized medical record review. SETTING: All hospitals performing coronary angiography and/or CABG surgery in two Canadian provinces (Ontario and British Columbia); in New York State, 15 randomly selected hospitals that provide coronary angiography and 15 randomly selected hospitals that provide CABG surgery. PATIENTS: All patients were randomly selected. For coronary angiography, 533 patients in Canada and 1333 patients in New York were selected; for CABG, 556 patients in Canada and 1336 patients in New York were selected. MAIN OUTCOME MEASURES: Percentage of patients in each country who had coronary angiography or CABG for necessary, appropriate, uncertain, or inappropriate indications as rated by criteria developed separately in each country and the complications of those procedures. RESULTS: For coronary angiography, 9% of Canadian cases and 10% of New York cases were rated inappropriate using Canadian criteria compared with 5% and 4%, respectively, using US criteria. For CABG, 4% of Canadian cases and 6% of New York cases were rated inappropriate by Canadian criteria compared with 3% and 2%, respectively, using US criteria. A lower proportion of procedures were performed on persons aged 75 years or older in Canada than in New York for both coronary angiography (5% vs 11%; P < .001) and CABG (6% vs 14%; P < .001). Women were also represented in lower proportions among angiography cases in Canada than in New York (28% vs 35%; P = .023). Canadian patients with left main coronary disease waited significantly longer between angiography and CABG than did New York patients (P < .0001). CONCLUSIONS: Rates of inappropriate use of cardiac procedures were low in Canada and New York, which suggests that the regionalization of cardiac procedures that characterizes both health care systems contributes to better clinical decision making. Differences in the use of cardiac procedures among the elderly in the two countries merits further comparative examination.

Measuring the necessity of medical procedures.

This is a report on the extension of the concept of the appropriateness of a procedure to the necessity, or crucial importance, of that procedure. To state that a procedure is crucial means that withholding the procedure would be deleterious to the patient's health. Appropriateness and necessity ratings for six procedures were obtained using a modified Delphi panel process developed in earlier work. Panels were composed of practicing clinicians who were recognized leaders in their fields. The panels included both performers and nonperformers of the procedure under discussion. For most procedures and panelists, necessity was related to appropriateness, but was distinct from it. The proportion of indications for which the procedure was crucial varied in clinically consistent ways both among and within procedures. However, panelists did not achieve a consensus on necessity. Further research is suggested to refine the method to promote consensus and to validate further the ratings of necessity. In conclusion, necessity ratings can be used together with appropriateness ratings to address not only the overuse of procedures, but also to indicate limited access to care through underuse of procedures.

Coronary angiography and revascularization: defining procedural indications through formal group processes. The Canadian Revascularization Panel, the Canadian Coronary Angiography Panel.

OBJECTIVES: To summarize the process and extent of interphysician agreement within two panels convened to derive indications for the appropriate use of coronary angiography and for coronary revascularization procedures. PARTICIPANTS: Two panels, each with nine practitioners. METHODS: Panelists rated the appropriateness of intervention for a comprehensive set of indications for each procedure. Indications were brief profiles created by combining and permuting clinical characteristics pertinent to case selection for intervention. Ratings were first made at home, with a second round at the panel meeting following open discussion. Final rankings of indications as 'appropriate', 'uncertain' or 'inappropriate' were based on the pattern of panelists' responses on a nine-point scale, including the median rating and extent of agreement among panelists. Agreement was defined as at least seven panelists' ratings within the three-point region containing the median rating. Panelists were later mailed a much-reduced list of indications for which there was agreement on appropriateness. These were re-rated on a necessity scale. A procedure was rated 'necessary' only if a physician was ethically obligated to recommend it as the preferred treatment option. RESULTS: For appropriateness of angiography, agreement occurred in 38.2% of indications in round 1 and 64.4% in round 2 (P < 0.0001). For coronary artery bypass graft (CABG) versus medical therapy, the corresponding increase was from 43.5 to 54.0% (P < 0.0001). Agreement on necessity of angiography occurred for 44.3% of scenarios. For indications where CABG alone was appropriate, agreement on necessity was 56%. However, for indications where percutaneous transluminal coronary angioplasty (PTCA) could be regarded as the first-line intervention, agreement on necessity was only 5%. CONCLUSIONS: A two-step panel process permitted considerable convergence of panelists' ratings, highlighting the importance of formal panel methods in setting utilization management criteria. However, the extent of continuing disagreement on ratings underscores the need to avoid a forced consensus; instead, divergent opinions should be taken as indicative of uncertainty about the appropriateness of intervention. Interpanelist agreement on necessity ratings was modest, but may help in setting benchmarks to assess possible underprovision of invasive cardiac services in Canada.

The appropriateness of use of coronary artery bypass graft surgery in New York State.

OBJECTIVE: To determine the appropriateness of use of coronary artery bypass graft surgery in New York State. DESIGN: Retrospective randomized medical record review. SETTING: Fifteen randomly selected hospitals in New York State that provide coronary artery bypass graft surgery. PATIENTS: Random sample of 1338 patients undergoing isolated coronary artery bypass graft surgery in New York State in 1990. MAIN OUTCOME MEASURES: Percentage of patients who had bypass surgery for appropriate, inappropriate, or uncertain indications; operative (30-day) mortality; and complications. RESULTS: Nearly 91% of the bypass operations were rated appropriate; 7%, uncertain; and 2.4%, inappropriate. This low inappropriate rate differs substantially from the 14% rate found in a previous study of patients operated on in 1979, 1980, and 1982. The difference in rates was not due to more lenient criteria but to changes in practice, the most important being that the fraction of patients receiving coronary artery bypass grafts for one- and two-vessel disease fell from 51% to 24%. Individual hospital rates of inappropriateness (0% to 5%) did not vary significantly. Rates of appropriateness also did not vary by hospital location, volume, or teaching status. Operative mortality was 2.0%; 17% of patients suffered a complication. Complication rates varied significantly among hospitals (P < .01) and were higher in downstate hospitals. CONCLUSIONS: The rates of inappropriate and uncertain use of coronary artery bypass graft surgery in New York State were very low. Rates of inappropriate use did not vary significantly among hospitals, or according to region, volume of bypass operations performed, or teaching status.

The appropriateness of use of percutaneous transluminal coronary angioplasty in New York State.

OBJECTIVE: To determine the appropriateness of use of percutaneous transluminal coronary angioplasty (PTCA) in New York State. DESIGN: Retrospective randomized medical record. SETTING: Fifteen randomly selected hospitals in New York State that provide PTCA. PATIENTS: Random sample of 1306 patients undergoing PTCA in New York State in 1990. MAIN OUTCOME MEASURES: Percentage of patients who underwent PTCA for indications rated appropriate, uncertain, and inappropriate. RESULTS: The majority of patients received PTCA for chronic stable angina, unstable angina, and in the post-myocardial infarction period (up to 3 weeks). Fifty-eight percent of PTCAs were rated appropriate; 38%, uncertain; and 4%, inappropriate. The inappropriate rate varied by hospital from 1% to 9% (P = .12); the uncertain rate, from 26% to 50% (P = .02); and the combined inappropriate and uncertain rate, from 29% to 57% (P < .001). There was no difference in appropriateness when the institutions were grouped by volume (fewer than 300 procedures annually or at least 300 procedures annually), location (upstate vs downstate), or by teaching status. CONCLUSIONS: Few PTCAs were performed for inappropriate indications in New York State. However, the large number of procedures performed for indications that were rated uncertain as to their net benefit requires further study and justification at both clinical and policy levels.