Application of Participatory Ergonomics to the Dissemination of a Quality Improvement Program for Optimizing Blood Culture Use.

BACKGROUND: Blood cultures are overused in pediatric ICUs (PICUs), which may lead to unnecessary antibiotic use and antibiotic resistance. Using a participatory ergonomics (PE) approach, the authors disseminated a quality improvement (QI) program for optimizing blood culture use in PICUs to a national 14-hospital collaborative. The objective of this study was to evaluate the dissemination process and its impact on blood culture reduction. METHODS: The PE approach emphasized three key principles (stakeholder participation, application of human factors and ergonomics knowledge and tools, and cross-site collaboration) with a six-step dissemination process. Data on interactions between sites and the coordinating team and site experiences with the dissemination process were collected using site diaries and semiannual surveys with local QI teams, respectively, and correlated with the site-specific change in blood culture rates. RESULTS: Overall, participating sites were able to successfully implement the program and reduced their blood culture rates from 149.4 blood cultures per 1,000 patient-days/month before implementation to 100.5 blood cultures per 1,000 patient-days/month after implementation, corresponding to a 32.7% relative reduction (p < 0.001). Variations in the dissemination process, as well as in local interventions and implementation strategies, were observed across sites. Site-specific changes in blood culture rates were weakly negatively correlated with the number of preintervention interactions with the coordinating team (p = 0.057) but not correlated with their experiences with the six domains of the dissemination process or their interventions. CONCLUSIONS: The authors applied a PE approach to disseminate a QI program for optimizing PICU blood culture use to a multisite collaborative. Working with local stakeholders, participating sites tailored their interventions and implementation processes and achieved the goal of reducing blood culture use.

Association of Diagnostic Stewardship for Blood Cultures in Critically Ill Children With Culture Rates, Antibiotic Use, and Patient Outcomes: Results of the Bright STAR Collaborative.

Importance: Blood culture overuse in the pediatric intensive care unit (PICU) can lead to unnecessary antibiotic use and contribute to antibiotic resistance. Optimizing blood culture practices through diagnostic stewardship may reduce unnecessary blood cultures and antibiotics. Objective: To evaluate the association of a 14-site multidisciplinary PICU blood culture collaborative with culture rates, antibiotic use, and patient outcomes. Design, Setting, and Participants: This prospective quality improvement (QI) collaborative involved 14 PICUs across the United States from 2017 to 2020 for the Bright STAR (Testing Stewardship for Antibiotic Reduction) collaborative. Data were collected from each participating PICU and from the Children's Hospital Association Pediatric Health Information System for prespecified primary and secondary outcomes. Exposures: A local QI program focusing on blood culture practices in the PICU (facilitated by a larger QI collaborative). Main Outcomes and Measures: The primary outcome was blood culture rates (per 1000 patient-days/mo). Secondary outcomes included broad-spectrum antibiotic use (total days of therapy and new initiations of broad-spectrum antibiotics ≥3 days after PICU admission) and PICU rates of central line-associated bloodstream infection (CLABSI), Clostridioides difficile infection, mortality, readmission, length of stay, sepsis, and severe sepsis/septic shock. Results: Across the 14 PICUs, the blood culture rate was 149.4 per 1000 patient-days/mo preimplementation and 100.5 per 1000 patient-days/mo postimplementation, for a 33% relative reduction (95% CI, 26%-39%). Comparing the periods before and after implementation, the rate of broad-spectrum antibiotic use decreased from 506 days to 440 days per 1000 patient-days/mo, respectively, a 13% relative reduction (95% CI, 7%-19%). The broad-spectrum antibiotic initiation rate decreased from 58.1 to 53.6 initiations/1000 patient-days/mo, an 8% relative reduction (95% CI, 4%-11%). Rates of CLABSI decreased from 1.8 to 1.1 per 1000 central venous line days/mo, a 36% relative reduction (95% CI, 20%-49%). Mortality, length of stay, readmission, sepsis, and severe sepsis/septic shock were similar before and after implementation. Conclusions and Relevance: Multidisciplinary diagnostic stewardship interventions can reduce blood culture and antibiotic use in the PICU. Future work will determine optimal strategies for wider-scale dissemination of diagnostic stewardship in this setting while monitoring patient safety and balancing measures.

A Research Agenda for Diagnostic Excellence in Critical Care Medicine.

Diagnosing critically ill patients in the intensive care unit is difficult. As a result, diagnostic errors in the intensive care unit are common and have been shown to cause harm. Research to improve diagnosis in critical care medicine has accelerated in past years. However, much work remains to fully elucidate the diagnostic process in critical care. To achieve diagnostic excellence, interdisciplinary research is needed, adopting a balanced strategy of continued biomedical discovery while addressing the complex care delivery systems underpinning the diagnosis of critical illness.

Culture Ordering for Patients with New-onset Fever: A Survey of Pediatric Intensive Care Unit Clinician Practices.

INTRODUCTION: Accurate assessment of infection in critically ill patients is vital to their care. Both indiscretion and under-utilization of diagnostic microbiology testing can contribute to inappropriate antibiotic administration or delays in diagnosis. However, indiscretion in diagnostic microbiology cultures may also lead to unnecessary tests that, if false-positive, would incur additional costs and unhelpful evaluations. This quality improvement project objective was to assess pediatric intensive care unit (PICU) clinicians' attitudes and practices around the microbiology work-up for patients with new-onset fever. METHODS: We developed and conducted a self-administered electronic survey of PICU clinicians at a single institution. The survey included 7 common clinical vignettes of PICU patients with new-onset fever and asked participants whether they would obtain central line blood cultures, peripheral blood cultures, respiratory aspirate cultures, cerebrospinal fluid cultures, urine cultures, and/or urinalyses. RESULTS: Forty-seven of 54 clinicians (87%) completed the survey. Diagnostic specimen ordering practices were notably heterogeneous. Respondents unanimously favored a decision-support algorithm to guide culture specimen ordering practices for PICU patients with fever (100%, N = 47). A majority (91.5%, N = 43) indicated that a decision-support algorithm would be a means to align PICU and consulting care teams when ordering culture specimens for patients with fever. CONCLUSION: This survey revealed variability of diagnostic specimen ordering practices for patients with new fever, supporting an opportunity to standardize practices. Clinicians favored a decision-support tool and thought that it would help align patient management between clinical team members. The results will be used to inform future diagnostic stewardship efforts.

Early Prediction of Multiple Organ Dysfunction in the Pediatric Intensive Care Unit.

Objective: The objective of the study is to build models for early prediction of risk for developing multiple organ dysfunction (MOD) in pediatric intensive care unit (PICU) patients. Design: The design of the study is a retrospective observational cohort study. Setting: The setting of the study is at a single academic PICU at the Johns Hopkins Hospital, Baltimore, MD. Patients: The patients included in the study were <18 years of age admitted to the PICU between July 2014 and October 2015. Measurements and main results: Organ dysfunction labels were generated every minute from preceding 24-h time windows using the International Pediatric Sepsis Consensus Conference (IPSCC) and Proulx et al. MOD criteria. Early MOD prediction models were built using four machine learning methods: random forest, XGBoost, GLMBoost, and Lasso-GLM. An optimal threshold learned from training data was used to detect high-risk alert events (HRAs). The early prediction models from all methods achieved an area under the receiver operating characteristics curve ≥0.91 for both IPSCC and Proulx criteria. The best performance in terms of maximum F1-score was achieved with random forest (sensitivity: 0.72, positive predictive value: 0.70, F1-score: 0.71) and XGBoost (sensitivity: 0.8, positive predictive value: 0.81, F1-score: 0.81) for IPSCC and Proulx criteria, respectively. The median early warning time was 22.7 h for random forest and 37 h for XGBoost models for IPSCC and Proulx criteria, respectively. Applying spectral clustering on risk-score trajectories over 24 h following early warning provided a high-risk group with ≥0.93 positive predictive value. Conclusions: Early predictions from risk-based patient monitoring could provide more than 22 h of lead time for MOD onset, with ≥0.93 positive predictive value for a high-risk group identified pre-MOD.

Prediction of Impending Septic Shock in Children With Sepsis.

OBJECTIVES: Sepsis and septic shock are leading causes of in-hospital mortality. Timely treatment is crucial in improving patient outcome, yet treatment delays remain common. Early prediction of those patients with sepsis who will progress to its most severe form, septic shock, can increase the actionable window for interventions. We aim to extend a time-evolving risk score, previously developed in adult patients, to predict pediatric sepsis patients who are likely to develop septic shock before its onset, and to determine whether or not these risk scores stratify into groups with distinct temporal evolution once this prediction is made. DESIGN: Retrospective cohort study. SETTING: Academic medical center from July 1, 2016, to December 11, 2020. PATIENTS: Six-thousand one-hundred sixty-one patients under 18 admitted to the Johns Hopkins Hospital PICU. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: We trained risk models to predict impending transition into septic shock and compute time-evolving risk scores representative of a patient's probability of developing septic shock. We obtain early prediction performance of 0.90 area under the receiver operating curve, 43% overall positive predictive value, patient-specific positive predictive value as high as 62%, and an 8.9-hour median early warning time using Sepsis-3 labels based on age-adjusted Sequential Organ Failure Assessment score. Using spectral clustering, we stratified pediatric sepsis patients into two clusters differing in septic shock prevalence, mortality, and proportion of patients adequately fluid resuscitated. CONCLUSIONS: We demonstrate the applicability of our methodology for early prediction and stratification for risk of septic shock in pediatric sepsis patients. Through analyses of risk score evolution over time, we corroborate our past finding of an abrupt transition preceding onset of septic shock in children and are able to stratify pediatric sepsis patients using their risk score trajectories into low and high-risk categories.

Consensus Recommendations for Blood Culture Use in Critically Ill Children Using a Modified Delphi Approach.

OBJECTIVES: Blood cultures are fundamental in evaluating for sepsis, but excessive cultures can lead to false-positive results and unnecessary antibiotics. Our objective was to create consensus recommendations focusing on when to safely avoid blood cultures in PICU patients. DESIGN: A panel of 29 multidisciplinary experts engaged in a two-part modified Delphi process. Round 1 consisted of a literature summary and an electronic survey sent to invited participants. In the survey, participants rated a series of recommendations about when to avoid blood cultures on five-point Likert scale. Consensus was achieved for the recommendation(s) if 75% of respondents chose a score of 4 or 5, and these were included in the final recommendations. Any recommendations that did not meet these a priori criteria for consensus were discussed during the in-person expert panel review (Round 2). Round 2 was facilitated by an independent expert in consensus methodology. After a review of the survey results, comments from round 1, and group discussion, the panelists voted on these recommendations in real-time. SETTING: Experts' institutions; in-person discussion in Baltimore, MD. SUBJECTS: Experts in pediatric critical care, infectious diseases, nephrology, oncology, and laboratory medicine. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: Of the 27 original recommendations, 18 met criteria for achieving consensus in Round 1; some were modified for clarity or condensed from multiple into single recommendations during Round 2. The remaining nine recommendations were discussed and modified until consensus was achieved during Round 2, which had 26 real-time voting participants. The final document contains 19 recommendations. CONCLUSIONS: Using a modified Delphi process, we created consensus recommendations on when to avoid blood cultures and prevent overuse in the PICU. These recommendations are a critical step in disseminating diagnostic stewardship on a wider scale in critically ill children.

Diagnostic Errors in Pediatric Critical Care: A Systematic Review.

OBJECTIVES: To summarize the literature on prevalence, impact, and contributing factors related to diagnostic error in the PICU. DATA SOURCES: Search of PubMed, EMBASE, and the Cochrane Library up to December 2019. STUDY SELECTION: Studies on diagnostic error and the diagnostic process in pediatric critical care were included. Non-English studies with no translation, case reports/series, studies providing no information on diagnostic error, studies focused on non-PICU populations, and studies focused on a single condition/disease or a single diagnostic test/tool were excluded. DATA EXTRACTION: Data on research design, objectives, study sample, and results pertaining to the prevalence, impact, and factors associated with diagnostic error were abstracted from each study. DATA SYNTHESIS: Using independent tiered review, 396 abstracts were screened, and 17 studies (14 full-text, 3 abstracts) were ultimately included. Fifteen of 17 studies (88%) had an observational research design. Autopsy studies (autopsy rates were 20-47%) showed a 10-23% rate of missed major diagnoses; 5-16% of autopsy-discovered diagnostic errors had a potential adverse impact on survival and would have changed management. Retrospective record reviews reported varying rates of diagnostic error from 8% in a general PICU population to 12% among unexpected critical admissions and 21-25% of patients discussed at PICU morbidity and mortality conferences. Cardiovascular, infectious, congenital, and neurologic conditions were most commonly misdiagnosed. Systems factors (40-67%), cognitive factors (20-3%), and both systems and cognitive factors (40%) were associated with diagnostic error. Limited information was available on the impact of misdiagnosis. CONCLUSIONS: Knowledge of diagnostic errors in the PICU is limited. Future work to understand diagnostic errors should involve a balanced focus between studying the diagnosis of individual diseases and uncovering common system- and process-related determinants of diagnostic error.

Diagnostic Stewardship of Endotracheal Aspirate Cultures in a PICU.

BACKGROUND: Clinicians commonly obtain endotracheal aspirate cultures (EACs) in the evaluation of suspected ventilator-associated infections. However, bacterial growth in EACs does not distinguish bacterial colonization from infection and may lead to overtreatment with antibiotics. We describe the development and impact of a clinical decision support algorithm to standardize the use of EACs from ventilated PICU patients. METHODS: We monitored EAC use using a statistical process control chart. We compared the rate of EACs using Poisson regression and a quasi-experimental interrupted time series model and assessed clinical outcomes 1 year before and after introduction of the algorithm. RESULTS: In the preintervention year, there were 557 EACs over 5092 ventilator days; after introduction of the algorithm, there were 234 EACs over 3654 ventilator days (an incident rate of 10.9 vs 6.5 per 100 ventilator days). There was a 41% decrease in the monthly rate of EACs (incidence rate ratio [IRR]: 0.59; 95% confidence interval [CI] 0.51-0.67; P < .001). The interrupted time series model revealed a preexisting 2% decline in the monthly culture rate (IRR: 0.98; 95% CI 0.97-1.0; P = .01), immediate 44% drop (IRR: 0.56; 95% CI 0.45-0.70; P = .02), and stable rate in the postintervention year (IRR: 1.03; 95% CI 0.99-1.07; P = .09). In-hospital mortality, hospital length of stay, 7-day readmissions, and All Patients Refined Diagnosis Related Group severity and mortality scores were stable. The estimated direct cost savings was $26 000 per year. CONCLUSIONS: A clinical decision support algorithm standardizing EAC obtainment from ventilated PICU patients was associated with a sustained decline in the rate of EACs, without changes in mortality, readmissions, or length of stay.

Spectral clustering of risk score trajectories stratifies sepsis patients by clinical outcome and interventions received.

Sepsis is not a monolithic disease, but a loose collection of symptoms with diverse outcomes. Thus, stratification and subtyping of sepsis patients is of great importance. We examine the temporal evolution of patient state using our previously-published method for computing risk of transition from sepsis into septic shock. Risk trajectories diverge into four clusters following early prediction of septic shock, stratifying by outcome: the highest-risk and lowest-risk groups have a 76.5% and 10.4% prevalence of septic shock, and 43% and 18% mortality, respectively. These clusters differ also in treatments received and median time to shock onset. Analyses reveal the existence of a rapid (30-60 min) transition in risk at the time of threshold crossing. We hypothesize that this transition occurs as a result of the failure of compensatory biological systems to cope with infection, resulting in a bifurcation of low to high risk. Such a collapse, we believe, represents the true onset of septic shock. Thus, this rapid elevation in risk represents a potential new data-driven definition of septic shock.

How good is our diagnostic intuition? Clinician prediction of bacteremia in critically ill children.

BACKGROUND: Clinical intuition and nonanalytic reasoning play a major role in clinical hypothesis generation; however, clinicians' intuition about whether a critically ill child is bacteremic has not been explored. We endeavored to assess pediatric critical care clinicians' ability to predict bacteremia and to evaluate what affected the accuracy of those predictions. METHODS: We conducted a retrospective review of clinicians' responses to a sepsis screening tool ("Early Sepsis Detection Tool" or "ESDT") over 6 months. The ESDT was completed during the initial evaluation of a possible sepsis episode. If a culture was ordered, they were asked to predict if the culture would be positive or negative. Culture results were compared to predictions for each episode as well as vital signs and laboratory data from the preceding 24 h. RESULTS: From January to July 2017, 266 ESDTs were completed. Of the 135 blood culture episodes, 15% of cultures were positive. Clinicians correctly predicted patients with bacteremia in 82% of cases, but the positive predictive value was just 28% as there was a tendency to overestimate the presence of bacteremia. The negative predictive value was 96%. The presence of bandemia, thrombocytopenia, and abnormal CRP were associated with increased likelihood of correct positive prediction. CONCLUSIONS: Clinicians are accurate in predicting critically ill children whose blood cultures, obtained for symptoms of sepsis, will be negative. Clinicians frequently overestimate the presence of bacteremia. The combination of evidence-based practice guidelines and bedside judgment should be leveraged to optimize diagnosis of bacteremia.

Survey-based Work System Assessment to Facilitate Large-scale Dissemination of Healthcare Quality Improvement Programs.

INTRODUCTION: The dissemination of quality improvement (QI) interventions to a broader range of healthcare settings requires a proactive assessment of local work systems and processes. The objective of this study was to examine the feasibility of using a survey-based work system assessment (WSA) tool to facilitate the dissemination of a program for optimizing blood culture (BC) use. METHODS: Informed by findings from an onsite, interview-based WSA at 2 hospitals, a 50-item WSA survey was devised and administrated to 15 hospitals participating in a QI collaborative. WSA survey data were summarized, shared, and discussed with individual hospitals to inform the adaptation and implementation of the BC program. Physician champions leading the local QI team assessed the use of the WSA survey by completing an 8-item survey. RESULTS: A total of 347 clinicians completed the WSA survey, and physician champions at 12 hospitals evaluated the use of the WSA survey. Both the WSA survey data and the evaluation of the WSA survey showed that the survey-based WSA tool could help participating hospitals understand their current BC ordering practices and identify potential barriers to implementing the program from the perspectives of different clinicians. CONCLUSIONS: We demonstrated how a survey-based tool could be used to facilitate WSA in the dissemination of a program for improving BC use to a multisite collaborative. A survey-based WSA tool can be used to facilitate future large-scale intervention dissemination efforts.

Practices, Perceptions, and Attitudes in the Evaluation of Critically Ill Children for Bacteremia: A National Survey.

OBJECTIVES: Sending blood cultures in children at low risk of bacteremia can contribute to a cascade of unnecessary antibiotic exposure, adverse effects, and increased costs. We aimed to describe practice variation, clinician beliefs, and attitudes about blood culture testing in critically ill children. DESIGN: Cross-sectional electronic survey. SETTING: Fifteen PICUs enrolled in the Blood Culture Improvement Guidelines and Diagnostic Stewardship for Antibiotic Reduction in Critically Ill Children collaborative, an investigation of blood culture use in critically ill children in the United States. SUBJECTS: PICU clinicians (bedside nurses, resident physicians, fellow physicians, nurse practitioners, physician assistants, and attending physicians). INTERVENTIONS: None. MEASUREMENT AND MAIN RESULTS: Survey items explored typical blood culture practices, attitudes and beliefs about cultures, and potential barriers to changing culture use in a PICU setting. Fifteen of 15 sites participated, with 347 total responses, 15-45 responses per site, and an overall median response rate of 57%. We summarized median proportions and interquartile ranges of respondents who reported certain practices or beliefs: 86% (73-91%) report that cultures are ordered reflexively; 71% (61-77%) do not examine patients before ordering cultures; 90% (86-94%) obtain cultures for any new fever in PICU patients; 33% (19-61%) do not obtain peripheral cultures when an indwelling catheter is in place; and 64% (36-81%) sample multiple (vs single) lumens of central venous catheters for new fever. When asked about barriers to reducing unnecessary cultures, 80% (73-90%) noted fear of missing sepsis. Certain practices (culture source and indication) varied by clinician type. Obtaining surveillance cultures and routinely culturing all possible sources (each lumen of indwelling catheters and peripheral specimens) are positively correlated with baseline blood culture rates. CONCLUSIONS: There is variation in blood culture practices in the PICU. Fear and reflexive habits are common drivers of cultures. These practices may contribute to over-testing for bacteremia. Further investigation of how to optimize blood culture use is warranted.

Data-driven discovery of a novel sepsis pre-shock state predicts impending septic shock in the ICU.

Septic shock is a life-threatening condition in which timely treatment substantially reduces mortality. Reliable identification of patients with sepsis who are at elevated risk of developing septic shock therefore has the potential to save lives by opening an early window of intervention. We hypothesize the existence of a novel clinical state of sepsis referred to as the "pre-shock" state, and that patients with sepsis who enter this state are highly likely to develop septic shock at some future time. We apply three different machine learning techniques to the electronic health record data of 15,930 patients in the MIMIC-III database to test this hypothesis. This novel paradigm yields improved performance in identifying patients with sepsis who will progress to septic shock, as defined by Sepsis- 3 criteria, with the best method achieving a 0.93 area under the receiver operating curve, 88% sensitivity, 84% specificity, and median early warning time of 7 hours. Additionally, we introduce the notion of patient-specific positive predictive value, assigning confidence to individual predictions, and achieving values as high as 91%. This study demonstrates that early prediction of impending septic shock, and thus early intervention, is possible many hours in advance.