Incorporating harms into the weighting of the revised Agency for Healthcare Research and Quality Patient Safety for Selected Indicators Composite (Patient Safety Indicator 90).

OBJECTIVE: To reweight the Agency for Healthcare Research and Quality Patient Safety for Selected Indicators Composite (Patient Safety Indicator [PSI] 90) from weights based solely on the frequency of component PSIs to those that incorporate excess harm reflecting patients' preferences for outcome-related health states. DATA SOURCES: National administrative and claims data involving hospitalizations in nonfederal, nonrehabilitation, acute care hospitals. STUDY DESIGN: We estimated the average excess aggregate harm associated with the occurrence of each component PSI using a cohort sample for each indicator based on denominator-eligible records. We used propensity scores to account for potential confounding in the risk models for each PSI and weighted observations to estimate the "average treatment effect in the treated" for those with the PSI event. We fit separate regression models for each harm outcome. Final PSI weights reflected both the disutilities and the frequencies of the harms. DATA COLLECTION/EXTRACTION METHODS: We estimated PSI frequencies from the 2012 Healthcare Cost and Utilization Project State Inpatient Databases with present on admission data and excess harms using 2012-2013 Centers for Medicare & Medicaid Services Medicare Fee-for-Service data. PRINCIPAL FINDINGS: Including harms in the weighting scheme changed individual component weights from the original frequency-based weighting. In the reweighted composite, PSIs 11 ("Postoperative Respiratory Failure"), 13 ("Postoperative Sepsis"), and 12 ("Perioperative Pulmonary Embolism or Deep Vein Thrombosis") contributed the greatest harm, with weights of 29.7%, 21.1%, and 20.4%, respectively. Regarding reliability, the overall average hospital signal-to-noise ratio for the reweighted PSI 90 was 0.7015. Regarding discrimination, among hospitals with greater than median volume, 34% had significantly better PSI 90 performance, and 41% had significantly worse performance than benchmark rates (based on percentiles). CONCLUSIONS: Reformulation of PSI 90 with harm-based weights is feasible and results in satisfactory reliability and discrimination, with a more clinically meaningful distribution of component weights.

Use of the Poisson Distribution Is a Helpful Tool That Is Underused in Nursing Practice.

BACKGROUND: The Poisson distribution is used to find the probability of an event occurring over an interval of time, distance, area, or volume. PROBLEM: It is a helpful statistical tool, especially when evaluating rare events, and is underused in nursing practice. APPROACH: A single-group study design is used to demonstrate use of the Poisson distribution in determining whether a change in the number of discrete events is due to random variation or reflects a change in practice patterns and in determining the probability of seeing the number of observed events. OUTCOMES: Steps demonstrate how one can easily use the Poisson distribution to answer common questions. CONCLUSION: Use of the Poisson distribution can help nurses make better informed decisions about observed variations in care, especially when the data are not normally distributed, and can prevent undue concern when fluctuations in the number of events are associated with random fluctuations.

Risk of Distal Embolization From tPA (Tissue-Type Plasminogen Activator) Administration Prior to Endovascular Stroke Treatment.

BACKGROUND AND PURPOSE: In large artery occlusion stroke, both intravenous (IV) tPA (tissue-type plasminogen activator) and endovascular stroke treatment (EST) are standard-of-care. It is unknown how often tPA causes distal embolization, in which a procedurally accessible large artery occlusion is converted to a more distal and potentially inaccessible occlusion. METHODS: We analyzed data from a decentralized stroke telemedicine program in an integrated healthcare delivery system covering 21 hospitals, with 2 high-volume EST centers. We captured all cases sent for EST and examined the relationship between IV tPA administration and the rate of distal embolization, the rate of target recanalization (modified Treatment in Cerebral Infarction scale 2b/3), clinical improvement before EST, and short-term and long-term clinical outcomes. RESULTS: Distal embolization before EST was quite common (63/314 [20.1%]) and occurred more often after IV tPA before EST (57/229 [24.9%]) than among those not receiving IV tPA (6/85 [7.1%]; P<0.001). Distal embolization was associated with an inability to attempt EST: after distal embolization, 26/63 (41.3%) could not have attempted EST because of the new clot location, while in cases without distal embolization, only 8/249 (3.2%) were unable to have attempted EST (P<0.001). Among patients who received IV tPA, 13/242 (5.4%) had sufficient symptom improvement that a catheter angiogram was not performed; 6/342 (2.5%) had improvement to within 2 points of their baseline NIHSS. At catheter angiogram, 2/229 (0.9%) of patients who had received tPA had complete recanalization without distal embolization. Both IV tPA and EST recanalization were associated with improved long-term outcome. CONCLUSIONS: IV tPA administration before EST for large artery occlusion is associated with distal embolization, which in turn may reduce the chance that EST can be attempted and recanalization achieved. At the same time, some IV tPA-treated patients show symptomatic improvement and complete recanalization. Because IV tPA is associated with both distal embolization and improved long-term clinical outcome, there is a need for prospective clinical trials testing the net benefit or harm of IV tPA before EST.

Sex-Based Differences in Symptom Perception and Care-Seeking Behavior in Acute Stroke.

CONTEXT: Lack of early stroke recognition and delays in seeking emergency care by persons experiencing a stroke severely limit acute treatment options. Sparse and sometimes conflicting evidence suggests sex differences in care-seeking behaviors in stroke, stroke knowledge, perceptions of stroke symptoms, and the importance of physical location at the time of stroke and of having a witnessed stroke. OBJECTIVE: To explore specific sex-based differences in stroke presentation and associated care-seeking behaviors. DESIGN: Descriptive study based on a convenience sample of 60 patients with stroke admitted to an academic medical center in Northern California. MAIN OUTCOME MEASURES: Impact of the patient's sex on 1) time to presentation (early [≤ 4.5 hours] vs late [> 4.5 hours]); 2) perception of symptoms and clinical signs; 3) stroke knowledge and decision making; 4) physical location at the time of stroke; and 5) bystander presence and assistance with decision making. RESULTS: There was a discrepancy between how patients perceive symptoms and their medical findings on physical examination. Although most patients had at least one sign or symptom associated with nationally used stroke recognition acronyms, both sexes delayed care because they did not perceive their symptoms as urgent. Early-presenting men were more likely to have a higher stroke severity score at admission, receive alteplase, arrive by Emergency Medical Services, and have a witnessed stroke. Both early- and late-presenting women reported more nonfocal symptoms than did men. CONCLUSION: This study suggests that there are sex-based differences in symptom perception and care-seeking behavior in acute ischemic stroke.

How Accurate is the AHRQ Patient Safety Indicator for Hospital-Acquired Pressure Ulcer in a National Sample of Records?

In 2008, we conducted a retrospective cross-sectional study to determine the test characteristics of the Agency for Healthcare Research and Quality patient safety indicator (PSI) for hospital-acquired pressure ulcer (PU). We sampled 1,995 inpatient records that met PSI 3 criteria and 4,007 records assigned to 14 DRGs with the highest empirical rates of PSI 3, which did not meet PSI 3 criteria, from 32 U.S. academic hospitals. We estimated the positive predictive value (PPV), sensitivity, and specificity of PSI 3 using both the software version contemporary to the hospitalizations (v3.1) and an approximation of the current version (v4.4). Of records that met PSI 3 version 3.1 criteria, 572 (PPV 28.3%; 95% CI 23.6-32.9%) were true positive. PU that was present on admission (POA) accounted for 76% of the false-positive records. Estimated sensitivity was 48.2% (95% CI 41.0-55.3%) and specificity 71.4% (95% CI 68.3-74.5%). Reclassifying records based on reported POA information and PU stage to approximate version 4.4 of PSI 3 improved sensitivity (78.6%; 95% CI 62.7-94.5%) and specificity (98.0; 95% CI 97.1-98.9%). In conclusion, accounting for POA information and PU staging to approximate newer versions of the PSI software (v4.3) moderately improves validity.

Detecting postoperative hemorrhage or hematoma from administrative data: the performance of the AHRQ Patient Safety Indicator.

BACKGROUND: Patient Safety Indicator (PSI) 9, "postoperative hemorrhage or hematoma" (PHH), of the US Agency for Healthcare Research and Quality has been considered for public quality of care reporting. We sought to evaluate its performance in detecting true complications. METHODS: We conducted a retrospective, cross-sectional study of hospitalizations that met PSI 9 eligibility criteria. We sampled records flagged positive and negative by PSI 9 from a diverse set of 31 hospitals between February 2006, and June 2009. Trained abstractors reviewed medical records using standard instruments. We determined the sensitivity, specificity, and positive (PPV) and negative (NPV) predictive values of the indicator. RESULTS: Of 181 analyzable records flagged by PSI 9, 168 (93%; weighted PPV, 95% [95% confidence interval (CI), 90-98%]) involved an accurately coded event, but only 126 (70%; weighted PPV, 78% [95% CI, 58-90%]) represented true PHH. Thirty-two false positives involved only intraoperative hemorrhage. Among true positives, hypotension occurred in 28% and death attributed to the PHH in 4%. Thirty-two of 281 records flagged negative by PSI 9 (but enriched with questionably negative records) represented true PHH. The indicator's sensitivity was 42% (95% CI, 23-64%), specificity 99.9% (95% CI, 99.8-100%), and NPV 99.7% (95% CI, 99.0-99.9%). Modifying the indicator to include additional procedure codes improved both sensitivity (85% [95% CI, 67-94%]) and PPV (76% [95% CI, 60-88%]). CONCLUSION: PSI 9 holds promise in detecting serious, possibly preventable complications. The indicator might be improved by specification of the 998.11 hemorrhage code to exclude purely intraoperative events and addition of procedure codes to the indicator's numerator criteria.

Validity of the AHRQ patient safety indicator for postoperative physiologic and metabolic derangement based on a national sample of medical records.

OBJECTIVE: The Agency for Healthcare Research and Quality Patient Safety Indicator (PSI) 10, "Postoperative Physiologic and Metabolic Derangement" (PPMD), uses administrative data to detect postoperative acute kidney injury (AKI) requiring dialysis and diabetes-related complications. We sought to evaluate the indicator's criterion validity. RESEARCH DESIGN: We conducted a retrospective cross-sectional study of hospitalization records flagged positive and negative by PSI 10 from a diverse set of 35 hospitals between February 1, 2006 and June 30, 2009. Trained nurse abstractors reviewed medical records. We determined the indicator's sensitivity, specificity, and positive and negative predictive values. RESULTS: Of 94 records flagged by PSI 10 (87 for AKI, 7 for diabetic complications, 1 for both), 69 (73%) involved an accurately coded event; 60 (64%; 95% CI, 46%-79%) represented true PPMD from a clinical perspective. Two of 8 records flagged for diabetic complications were true events. Nineteen false positives involved preoperative renal failure. Three of 230 records flagged negative (enriched with questionably negative records) represented true PPMD. The indicator's sensitivity was 66% (20%-94%), specificity 99.9% (99.5%-100%), and negative predictive value 99.9% (99.4%-100%). Considering dialysis access procedures tantamount to dialysis and excluding records with lower urinary tract obstruction might increase the sensitivity and positive predictive value to 98% (87%-100%) and 72% (50%-87%), respectively. CONCLUSIONS: PSI 10 mostly concerns AKI and currently has moderate criterion validity, which might improve with increased use of "present on admission" coding, abandonment of the diabetes criteria, and adjustments to the indicator specifications regarding dialysis access and urinary tract obstruction.

Using the Agency for Healthcare Research and Quality patient safety indicators for targeting nursing quality improvement.

Quantifying the critical impact nurses have on the prevention and early recognition of potential complications and adverse events, such as those identified by the Agency for Healthcare Research and Quality (AHRQ) patient safety indicators (PSI), is becoming increasingly important. In this paper, we describe how the AHRQ PSI may be used to identify nursing-specific opportunities to improve care based on data from the national AHRQ PSI validation pilot project.

Positive predictive value of the AHRQ Patient Safety Indicator "Postoperative Sepsis": implications for practice and policy.

BACKGROUND: Patient Safety Indicator (PSI) 13, or "Postoperative Sepsis," of the Agency for Healthcare Quality and Research (AHRQ), was recently adopted as part of a composite measure of patient safety by the Centers for Medicare and Medicaid Services (CMS). We sought to examine its positive predictive value (PPV) by determining how well it identifies true cases of postoperative sepsis. STUDY DESIGN: Two retrospective cross-sectional studies of hospitalization records that met PSI 13 criteria were conducted, one within the Veterans Administration (VA) Hospitals from fiscal years (FY) 2003 to 2007, and one within community hospitals between October 1, 2005 and March 31, 2007. Trained abstractors reviewed medical records from each database using standardized abstraction instruments. We determined the PPV of the indicator and performed descriptive analyses of cases. RESULTS: Of 112 cases flagged and reviewed within the VA system, 59 were true events of postoperative sepsis, yielding a PPV of 53% (95% CI 42% to 64%). Within the community hospital sector, of 164 flagged and reviewed cases, 67 were true cases of postoperative sepsis, yielding a PPV of 41% (95% CI 28% to 54%). False positives were due to infections that were present on admission, urgent or emergent cases, no clinical diagnosis of sepsis, or other coding limitations such as nonspecific shock in postoperative patients. CONCLUSIONS: PSI 13 has relatively poor predictive ability to identify true cases of postoperative sepsis in both the VA and nonfederal sectors. The lack of information on diagnosis timing, confusion about the definition of elective admission, and coding limitations were the major reasons for false positives. As it currently stands, the use of PSI 13 as a stand-alone measure for hospital reporting appears premature.

Positive predictive value of the Agency for Healthcare Research and Quality Patient Safety Indicator for central line-related bloodstream infection ("selected infections due to medical care").

As part of the Agency for Healthcare Research and Quality Patient Safety Indicator (PSI) Validation Pilot Project, we evaluated the criterion validity of PSI 7. At the time of this study, PSI 7 was entitled "Selected Infections Due to Medical Care" and targeted catheter-related infections and inflammatory reactions. We conducted a retrospective cross-sectional study of 23 volunteer U.S. hospitals, where trained abstractors reviewed a sample of records that met PSI 7 criteria from October 1, 2005 to March 31, 2007. Of the 191 cases that met PSI 7 criteria, 104 (positive predictive value = 54%, 95% confidence interval: 40-69%) represented true infections. Of these cases, 77 (74%) were associated with central venous catheters, 15 (15%) were associated with peripheral intravenous (n=13) and or or arterial catheters (n=6), and 12 (11%) were associated with unknown catheters. Of the 87 (46%) false-positive cases, 41 (47%) did not have a qualifying infection identified by the abstractor, 38 (44%) had an infection present on admission, and 8 (9%) had an exclusionary diagnosis. PSI 7 has a low positive predictive value compared with other PSIs recently studied. Present on admission diagnoses and improved coding for infections related to central venous catheters (implemented October 2007) may improve validity.

Designing an abstraction instrument: lessons from efforts to validate the AHRQ patient safety indicators.

BACKGROUND: The U.S. Agency for Healthcare Research and Quality (AHRQ) and other organizations have developed quality indicators based on hospital administrative data. Characteristics of effective abstraction instruments were identified for determining both the positive predictive value (PPV) of Patient Safety Indicators (PSIs) and the extent to which hospitals and clinicians could have prevented adverse events. METHODS: Through an iterative process involving nurse abstractors, physicians, and nurses with quality improvement experience, and health services researchers, 25 abstraction instruments were designed for 12 AHRQ provider-level morbidity PSIs. Data were analyzed from 13 of these instruments, and data are being collected using several more. FINDINGS: Common problems in designing the instruments included avoiding uninformative questions and premature termination of the abstraction process, anticipating misinterpretation of questions, allowing an appropriate range of response options; using clear terminology, optimizing the flow of the abstraction process, balancing the utility of data against abstractor burden, and recognizing the needs of end users, such as hospitals and quality improvement professionals and researchers, for the abstracted information. CONCLUSIONS: Designing medical record abstraction instruments for quality improvement research involves several potential pitfalls. Understanding how we addressed these challenges might help both investigators and users of outcome indicators to appreciate the strengths and limitations of outcome-based quality indicators and tools designed to validate or investigate such indicators within provider organizations.

Detection of postoperative respiratory failure: how predictive is the Agency for Healthcare Research and Quality's Patient Safety Indicator?

BACKGROUND: Patient Safety Indicator (PSI) 11, or postoperative respiratory failure, was developed by the US Agency for Healthcare Research and Quality to detect incident cases of respiratory failure after elective operations through use of ICD-9-CM diagnosis and procedure codes. We sought to determine the positive predictive value (PPV) of this indicator. STUDY DESIGN: We conducted a retrospective cross-sectional study, sampling consecutive cases that met PSI 11 criteria from 18 geographically diverse academic medical centers on or before June 30, 2007. Trained abstractors from each center reviewed medical records using a standard instrument. We assessed the PPV of the indicator (with 95% CI adjusted for clustering within centers) and conducted descriptive analyses of the cases. RESULTS: Of 609 cases that met PSI 11 criteria, 551 (90.5%; 95% CI, 86.5-94.4%) satisfied the technical criteria of the indicator and 507 (83.2%; 95% CI, 77.2-89.3%) represented true cases of postoperative respiratory failure from a clinical standpoint. The most frequent reasons for being falsely positive were nonelective hospitalization, prolonged intubation for airway protection, and insufficient evidence to support a diagnosis of acute respiratory failure. Fifty percent of true-positive cases involved substantial baseline comorbidities, and 23% resulted in death. CONCLUSIONS: Although PSI 11 predicts true postoperative respiratory failure with relatively high frequency, the indicator does not limit detection to preventable cases. The PPV of PSI 11 might be increased by excluding cases with a principal diagnosis suggestive of a nonelective hospitalization and those with head or neck procedures. Removing the diagnosis code criterion from the indicator might also increase PPV, but would decrease the number of true positive cases detected by 20%.

Positive predictive value of the AHRQ accidental puncture or laceration patient safety indicator.

OBJECTIVE: Patient Safety Indicator (PSI) 15, or "Accidental Puncture or Laceration" (APL), of the US Agency for Healthcare Research and Quality was recently endorsed as a consensus standard for quality of care by the National Quality Forum. We sought to determine the positive predictive value (PPV) of this indicator. METHODS: We conducted a retrospective cross-sectional study of hospitalization records that met PSI 15 criteria. We sampled cases from 32 geographically diverse hospitals, including both teaching and nonteaching hospitals, between October 1, 2005 and March 31, 2007. Trained abstractors from each center reviewed randomly sampled medical records, using a standard instrument. We determined the PPV of the indicator and conducted descriptive analyses of the cases. RESULTS: Of the 249 cases that met PSI 15 criteria, 226 (91%; 95% CI: 88%-94%) represented true APL. Fifty-six of the true APL cases (24%) represented injuries that generally would be expected to heal without repair, yielding, from the standpoint of clinical relevance, a PPV of 68% (95% CI: 62%-74%). True positive cases that would typically warrant repair (n=170) were most likely to involve the gastrointestinal tract (30%), bladder (25%), dura (19%), or an important blood vessel (16%). In 97 of the true APL cases (43%), adhesions or other scar tissue were thought to have contributed to the complication. The 23 false-positive cases involved no apparent event other than normal operative conduct (n=7), a complication other than APL (bleeding, infection, dislodgement of a gastrostomy tube, or fracture) (7), an APL present on admission (5), or a disease-related lesion (4). CONCLUSIONS: Although PSI 15 is highly predictive of APL from a coding perspective, the indicator is less predictive of APL that could be considered clinically important. A significant proportion of cases represent relatively inconsequential injuries or injuries for which the risk may have been acceptable relative to the goals of the procedure.

Correlates of memory function in community-dwelling elderly: the importance of white matter hyperintensities.

We sought to identify magnetic resonance- (MR)-imaged structures associated with declarative memory in a community-dwelling sample of elderly Mexican-American individuals with a spectrum of cognitive decline. Measured structures were the hemispheric volumes of the hippocampus (HC), parahippocampal gyrus, and remaining temporal lobes, as well as severity of white matter signal hyperintensities (WMH). Participants were an imaged subsample from the Sacramento Area Latino Study of Aging (SALSA), N = 122. Individuals were categorized as normal, memory impaired (MI), cognitively impaired non-demented (CIND), or demented. We show that WMH was the strongest structural predictor for performance on a delayed free-recall task (episodic memory) in the entire sample. The association of WMH with delayed recall was most prominent in elderly normals and mildly cognitively impaired individuals with no dementia or impairment of daily function. However, the left HC was associated with verbal delayed recall only in people with dementia. The right HC volume predicted nonverbal semantic-memory performance. We conclude that WMH are an important pathological substrate that affects certain memory functions in normal individuals and those with mild memory loss and discuss how tasks associated with WMH may rely upon frontal lobe function.