Challenging Cases in Statistical Process Control for Quality Improvement in Neonatal Intensive Care.

Effective quality improvement (QI) depends on rigorous analysis of time-series data through methods such as statistical process control (SPC). As use of SPC has become more prevalent in health care, QI practitioners must also be aware of situations that warrant special attention and potential modifications to common SPC charts, which include skewed continuous data, autocorrelation, small persistent changes in performance, confounders, and workload or productivity measures. This article reviews these situations and provides examples of SPC approaches for each.

A hybrid Shewhart chart for visualizing and learning from epidemic data.

OBJECTIVE: As the globe endures the coronavirus disease 2019 (COVID-19) pandemic, we developed a hybrid Shewhart chart to visualize and learn from day-to-day variation in a variety of epidemic measures over time. CONTEXT: Countries and localities have reported daily data representing the progression of COVID-19 conditions and measures, with trajectories mapping along the classic epidemiological curve. Settings have experienced different patterns over time within the epidemic: pre-exponential growth, exponential growth, plateau or descent and/ or low counts after descent. Decision-makers need a reliable method for rapidly detecting transitions in epidemic measures, informing curtailment strategies and learning from actions taken. METHODS: We designed a hybrid Shewhart chart describing four 'epochs' ((i) pre-exponential growth, (ii) exponential growth, (iii) plateau or descent and (iv) stability after descent) of the COVID-19 epidemic that emerged by incorporating a C-chart and I-chart with a log-regression slope. We developed and tested the hybrid chart using international data at the country, regional and local levels with measures including cases, hospitalizations and deaths with guidance from local subject-matter experts. RESULTS: The hybrid chart effectively and rapidly signaled the occurrence of each of the four epochs. In the UK, a signal that COVID-19 deaths moved into exponential growth occurred on 17 September, 44 days prior to the announcement of a large-scale lockdown. In California, USA, signals detecting increases in COVID-19 cases at the county level were detected in December 2020 prior to statewide stay-at-home orders, with declines detected in the weeks following. In Ireland, in December 2020, the hybrid chart detected increases in COVID-19 cases, followed by hospitalizations, intensive care unit admissions and deaths. Following national restrictions in late December, a similar sequence of reductions in the measures was detected in January and February 2021. CONCLUSIONS: The Shewhart hybrid chart is a valuable tool for rapidly generating learning from data in close to real time. When used by subject-matter experts, the chart can guide actionable policy and local decision-making earlier than when action is likely to be taken without it.

Factorial Analysis Quantifies the Effects of Pediatric Discharge Bundle on Hospital Readmission.

BACKGROUND AND OBJECTIVES: Factorial design of a natural experiment was used to quantify the benefit of individual and combined bundle elements from a 4-element discharge transition bundle (checklist, teach-back, handoff to outpatient providers, and postdischarge phone call) on 30-day readmission rates (RRs). METHODS: A 24 factorial design matrix of 4 bundle element combinations was developed by using patient data (N = 7725) collected from January 2014 to December 2017 from 4 hospitals. Patients were classified into 3 clinical risk groups (CRGs): no chronic disease (CRG1), single chronic condition (CRG2), and complex chronic condition (CRG3). Estimated main effects of each bundle element and their interactions were evaluated by using Study-It software. Because of variation in subgroup size, important effects from the factorial analysis were determined by using weighted effect estimates. RESULTS: RR in CRG1 was 3.5% (n = 4003), 4.1% in CRG2 (n = 1936), and 17.6% in CRG3 (n = 1786). Across the 3 CRGs, the number of subjects in the factorial groupings ranged from 16 to 674. The single most effective element in reducing RR was the checklist in CRG1 and CRG2 (reducing RR by 1.3% and 3.0%) and teach-back in CRG3 (by 4.7%) The combination of teach-back plus a checklist had the greatest effect on reducing RR in CRG3 by 5.3%. CONCLUSIONS: The effect of bundle elements varied across risk groups, indicating that transition needs may vary on the basis of population. The combined use of teach-back plus a checklist had the greatest impact on reducing RR for medically complex patients.

A maturity grid assessment tool for learning networks.

BACKGROUND: The vision of learning healthcare systems (LHSs) is attractive as a more effective model for health care services, but achieving the vision is complex. There is limited literature describing the processes needed to construct such multicomponent systems or to assess development. METHODS: We used the concept of a capability maturity matrix to describe the maturation of necessary infrastructure and processes to create learning networks (LNs), multisite collaborative LHSs that use an actor-oriented network organizational architecture. We developed a network maturity grid (NMG) assessment tool by incorporating information from literature review, content theory from existing networks, and expert opinion to establish domains and components. We refined the maturity grid in response to feedback from network leadership teams. We followed NMG scores over time for nine LNs and plotted scores for each domain component with respect to SD for one participating network. We sought subjective feedback on the experience of applying the NMG to individual networks. RESULTS: LN leaders evaluated the scope, depth, and applicability of the NMG to their networks. Qualitative feedback from network leaders indicated that changes in NMG scores over time aligned with leaders' reports about growth in specific domains; changes in scores were consistent with network efforts to improve in various areas. Scores over time showed differences in maturation in the individual domains of each network. Scoring patterns, and SD for domain component scores, indicated consistency among LN leaders in some but not all aspects of network maturity. A case example from a participating network highlighted the value of the NMG in prompting strategic discussions about network development and demonstrated that the process of using the tool was itself valuable. CONCLUSIONS: The capability maturity grid proposed here provides a framework to help those interested in creating Learning Health Networks plan and develop them over time.

Using control charts to understand community variation in COVID-19.

Decision-makers need signals for action as the coronavirus disease 2019 (COVID-19) pandemic progresses. Our aim was to demonstrate a novel use of statistical process control to provide timely and interpretable displays of COVID-19 data that inform local mitigation and containment strategies. Healthcare and other industries use statistical process control to study variation and disaggregate data for purposes of understanding behavior of processes and systems and intervening on them. We developed control charts at the county and city/neighborhood level within one state (California) to illustrate their potential value for decision-makers. We found that COVID-19 rates vary by region and subregion, with periods of exponential and non-exponential growth and decline. Such disaggregation provides granularity that decision-makers can use to respond to the pandemic. The annotated time series presentation connects events and policies with observed data that may help mobilize and direct the actions of residents and other stakeholders. Policy-makers and communities require access to relevant, accurate data to respond to the evolving COVID-19 pandemic. Control charts could prove valuable given their potential ease of use and interpretability in real-time decision-making and for communication about the pandemic at a meaningful level for communities.

Understanding the Differences in COVID-19 Case Fatality Rates Observed Across Alabama Counties.

To understand county-level variation in case fatality rates of COVID-19, a statewide analysis of COVID-19 incidence and fatality data was performed, using publicly available incidence and case fatality rate data of COVID-19 for all 67 Alabama counties and mapped with health disparities at the county level. A specific adaptation of the Shewhart p-chart, called a funnel chart, was used to compare case fatality rates. Important differences in case fatality rates across the counties did not appear to be reflective of differences in testing or incidence rates. Instead, a higher prevalence of comorbidities and vulnerabilities was observed in high fatality rate counties, while showing no differences in access to acute care. Funnel charts reliably identify counties with unexpected high and low COVID-19 case fatality rates. Social determinants of health are strongly associated with such differences. These data may assist in public health decisions including vaccination strategies, especially in southern states with similar demographics.

Developing Evidence for New Patient Safety Bundles Through Multihospital Collaboration.

OBJECTIVE: Multihospital collaboration for safety improvements is increasingly common, but strategies for developing bundles when effective evidence-based practices are not well described are limited. The Children's Hospitals' Solutions for Patient Safety (SPS) Network sought to further reduce patient harm by developing improvement bundles when preliminary evidence was limited. METHODS: As part of the novel Pioneer process, cohorts of volunteer SPS hospitals collaborated to identify a harm reduction bundle for carefully selected hospital-acquired harm categories where evidence-based practices were limited. For each harm type, a leadership team selected interventions (factors) for testing and guided the work throughout the Pioneer process. Using fundamental quality improvement techniques and a planned experimentation design, each participating hospital submitted outcome and process compliance data for the factor implemented. Data from all hospitals implementing that factor were analyzed together using Shewhart charts, response plots, and analysis of covariance to identify whether reliable implementation of the factor influenced outcomes. Factors were categorized based on strength of evidence and other clinical or evidentiary support. Factors with strong support were included in a final bundle and disseminated to all SPS hospitals. RESULTS: The SPS began the bundle identification process for nine harm types and three have completed the process. The analytic approach resulted in four scenarios that along with clinical input guided the inclusion or rejection of the factor in the final bundle. CONCLUSIONS: In this multihospital collaborative, quality improvement methods and planned experimentation were effective at developing evidence-based harm reduction bundles in situations where limited data for interventions exist.

Understanding variation in reported covid-19 deaths with a novel Shewhart chart application.

OBJECTIVE: Motivated by the coronavirus disease 2019 (covid-19) pandemic, we developed a novel Shewhart chart to visualize and learn from variation in reported deaths in an epidemic. CONTEXT: Without a method to understand if a day-to-day variation in outcomes may be attributed to meaningful signals of change-rather than variability we would expect-care providers, improvement leaders, policy-makers, and the public will struggle to recognize if epidemic conditions are improving. METHODS: We developed a novel hybrid C-chart and I-chart to detect within a geographic area the start and end of exponential growth in reported deaths. Reported deaths were the unit of analysis owing to erratic reporting of cases from variability in local testing strategies. We used simulation and case studies to assess chart performance and define technical parameters. This approach also applies to other critical measures related to a pandemic when high-quality data are available. CONCLUSIONS: The hybrid chart detected the start of exponential growth and identified early signals that the growth phase was ending. During a pandemic, timely reliable signals that an epidemic is waxing or waning may have mortal implications. This novel chart offers a practical tool, accessible to system leaders and frontline teams, to visualize and learn from daily reported deaths during an epidemic. Without Shewhart charts and, more broadly, a theory of variation in our epidemiological arsenal, we lack a scientific method for a real-time assessment of local conditions. Shewhart charts should become a standard method for learning from data in the context of a pandemic or epidemic.

Orchestrated Testing of Formula Type to Reduce Length of Stay in Neonatal Abstinence Syndrome.

BACKGROUND: Despite the standardization of care, formula feeding varied across sites of the Ohio Perinatal Quality Collaborative (OPQC). We used orchestrated testing (OT) to learn from this variation and improve nonpharmacologic care of infants with neonatal abstinence syndrome (NAS) requiring pharmacologic treatment in Ohio. METHODS: To test the impact of formula on length of stay (LOS), treatment failure, and weight loss among infants hospitalized with NAS, we compared caloric content (high versus standard) and lactose content (low versus standard) using a 22 factorial design. During October 2015 to June 2016, OPQC sites joined 1 of 4 OT groups. We used response plots to examine the effect of each factor and control charts to track formula use and LOS. We used the OT results to revise the nonpharmacologic bundle and implemented it during 2017. RESULTS: Forty-seven sites caring for 546 NAS infants self-selected into the 4 OT groups. Response plots revealed the benefit of high-calorie formula (HCF) on weight loss, treatment failure, and LOS. The nonpharmacologic treatment bundle was updated to recommend HCF when breastfeeding was not possible. During implementation, HCF use increased, and LOS decreased from 17.1 to 16.4 days across the OPQC. CONCLUSIONS: OT revealed that HCF was associated with shorter LOS in OPQC sites. Implementation of a revised nonpharmacologic care bundle was followed by additional LOS improvement in Ohio. Despite some challenges in the implementation of OT, our findings support its usefulness for learning in improvement networks.

It Is Time to Reconsider Factorial Designs: How Bradford Hill and R. A. Fisher Shaped the Standard of Clinical Evidence.

OBJECTIVES: Could medical research and quality improvement studies be more productive with greater use of multifactor study designs? METHODS: Drawing on new primary sources and the literature, we examine the roles of A. Bradford Hill and Ronald A. Fisher in introducing the design of experiments in medicine. RESULTS: Hill did not create the randomized controlled trial, but he popularized the idea. His choice to set aside Fisher's advanced study designs shaped the development of clinical research and helped the single-treatment trial to become a methodological standard. CONCLUSIONS: Multifactor designs are not widely used in medicine despite their potential to make improvement initiatives and health services research more efficient and effective. Quality managers, health system leaders, and directors of research institutes could increase productivity and gain important insights by promoting a broader use of factorial designs to study multiple interventions simultaneously and to learn from interactions.

Addressing Challenges of Baseline Variability in the Clinical Setting: Lessons from an Emergency Department.

To demonstrate methods of adjusting data in quality improvement projects for better learning about interventions over time. METHODS: A secondary analysis of data from a quality improvement project to improve patient wait times at an urban academic pediatric emergency department using electronic medical data from 2015 to 2018. The primary outcome was the wait times for low-acuity patients. Control charts were used to determine if the interventions were effective in reducing wait times. Two different data adjustment techniques were applied to account for changes in patient volume and seasonal effects on the outcome measure. RESULTS: We more effectively demonstrated improved patient wait times after adjusting for patient volume or seasonality. Patient wait times decreased from 75.2 to 72.9 minutes after the intervention; a 3% decrease sustained over 18 months. A strong correlation between patient volume and wait times was noted. Process stability was achieved on the control charts after data adjustment, with one centerline shift after data adjustment in contrast to 5 centerline shifts required before data adjustment. CONCLUSION: Adjusting for seasonality or patient volume created process stability and improved learning from control charts. After adjustment, we sustained decreased patient wait times more than a year out from the original intervention Adjusting by patient volume seems to be a preferred method of adjustment. Our findings support the importance of adjusting for baseline variability affected by seasonality or patient volumes, especially in flow projects, as a high yield method for process improvement.

Using Infant Mortality Data to Improve Maternal and Child Health Programs: An Application of Statistical Process Control Techniques for Rare Events.

Introduction The infant mortality rate (IMR) in the United States remains higher than most developed countries. To understand this public health issue and support state public health departments in displaying and analyzing data in ways that support learning, states participating in the Collaborative Improvement and Innovation Network to Reduce Infant Mortality (IM CoIIN) created statistical process control (SPC) charts for rare events. Methods State vital records data on live births and infant deaths was used to create U, T and G charts for Kansas and Alaska, two states participating in the IM CoIIN who sought methods to more effectively analyze IMR for subsets of their populations with infrequent number of deaths. The IMR and the number of days and number of births between infant deaths was charted for Kansas Non-Hispanic black population and six Alaska regions for the time periods 2013-2016 and 2011-2016, respectively. Established empirical patterns indicated points of special cause variation. Results The T and G charts for Kansas and G charts for Alaska depict points outside the upper control limit. These points indicate special cause variation and an increased number of days and/or births between deaths at these time periods. Discussion T and G charts offer value in examining rare events, and indicate special causes not detectable by U charts or other more traditional analytic methods. When small numbers make traditional analysis challenging, SPC has potential in the MCH field to better understand potential drivers of improvements in rare outcomes, inform decision making and take interventions to scale.

Co-Designing a Collaborative Chronic Care Network (C3N) for Inflammatory Bowel Disease: Development of Methods.

BACKGROUND: Our health care system fails to deliver necessary results, and incremental system improvements will not deliver needed change. Learning health systems (LHSs) are seen as a means to accelerate outcomes, improve care delivery, and further clinical research; yet, few such systems exist. We describe the process of codesigning, with all relevant stakeholders, an approach for creating a collaborative chronic care network (C3N), a peer-produced networked LHS. OBJECTIVE: The objective of this study was to report the methods used, with a diverse group of stakeholders, to translate the idea of a C3N to a set of actionable next steps. METHODS: The setting was ImproveCareNow, an improvement network for pediatric inflammatory bowel disease. In collaboration with patients and families, clinicians, researchers, social scientists, technologists, and designers, C3N leaders used a modified idealized design process to develop a design for a C3N. RESULTS: Over 100 people participated in the design process that resulted in (1) an overall concept design for the ImproveCareNow C3N, (2) a logic model for bringing about this system, and (3) 13 potential innovations likely to increase awareness and agency, make it easier to collect and share information, and to enhance collaboration that could be tested collectively to bring about the C3N. CONCLUSIONS: We demonstrate methods that resulted in a design that has the potential to transform the chronic care system into an LHS.

Sustaining SLUG Bug CLABSI Reduction: Does Sterile Tubing Change Technique Really Work?

OBJECTIVES: To evaluate the ability to sustain and further reduce central line-associated bloodstream infection (CLABSI) rates in NICUs participating in a multicenter CLABSI reduction collaborative and to assess the impact of the sterile tubing change (TC) technique as an important component in CLABSI reduction. METHODS: A multi-institutional quality improvement collaborative lowered CLABSI rates in level IV NICUs over a 12-month period. During the 19-month sustain phase, centers were encouraged to monitor and report compliance measures but were only required to report the primary outcome measure of the CLABSI rate. Four participating centers adopted the sterile TC technique during the sustain phase as part of a local Plan-Do-Study-Act cycle. RESULTS: The average aggregate baseline NICU CLABSI rate of 1.076 CLABSIs per 1000 line days was sustained for 19 months across 17 level IV NICUs from January 2013 to July 2014. Four centers transitioning from the clean to the sterile TC technique during the sustain phase had a 64% decrease in CLABSI rates from the baseline (1.59 CLABSIs per 1000 line days to 0.57 CLABSIs per 1000 line days). CONCLUSIONS: Sustaining low CLABSI rates in a multicenter collaborative is feasible with team engagement and ongoing collaboration. With these results, we further demonstrate the positive impact of the sterile TC technique in CLABSI reduction efforts.

Variation in the prevalence of urinary catheters: a profile of National Health Service patients in England.

INTRODUCTION: Harm from catheter-associated urinary tract infections is a common, potentially avoidable, healthcare complication. Variation in catheter prevalence may exist and provide opportunity for reducing harm, yet to date is poorly understood. This study aimed to determine variation in the prevalence of urinary catheters between patient groups, settings, specialities and over time. METHODS: A prospective study (July 2012 to April 2016) of National Health Service (NHS) patients surveyed by healthcare professionals, following a standardised protocol to determine the presence of a urinary catheter and duration of use, on 1 day per month using the NHS Safety Thermometer. RESULTS: 1314 organisations (253 NHS trusts) and 9 266 284 patients were included. Overall, 12.9% of patients were catheterised, but utilisation varied. There was higher utilisation of catheters in males (15.7% vs 10.7% p<0.001) and younger people (18-70 year 14.0% vs >70 year 12.8% p<0.001), utilisation was highest in hospital settings (18.6% p<0.001), particularly in critical care (76.6% p<0.001). Most catheters had been in situ <28 days (72.9% p<0.001). No clinically significant changes were seen over time in any setting or specialty. CONCLUSION: Catheter prevalence in patients receiving NHS-funded care varies according to gender, age, setting and specialty, being most prevalent in males, younger people, hospitals and critical care. Utilisation has changed only marginally over 46 months, and further guidance is indicated to provide clarity for clinicians on the insertion and removal of catheters to supplement the existing guidance on care.

Orchestrated Testing.

Health care quality improvement collaboratives implement care bundles to target critical parts of a complex system to improve a specific health outcome. The quantitative impact of each component of the care bundle is often unknown. Orchestrated testing (OT) is an application of planned experimentation that allows simultaneous examination of multiple practices (bundle elements) to determine which intervention or combination of interventions affects the outcome. The purpose of this article is to describe the process needed to design and implement OT methodology for improvement collaboratives. Examples from a multicenter collaborative to reduce central line-associated bloodstream infections highlight the practical application of this approach. The key components for implementation of OT are the following: (1) define current practice and evidence, (2) develop a factorial matrix and calculate power, (3) formulate structure for engagement, (4) analyze results, and (5) replicate findings.

Improving the Context Supporting Quality Improvement in a Neonatal Intensive Care Unit Quality Collaborative: An Exploratory Field Study.

Successful quality improvement (QI) requires a supportive context. The goal was to determine whether a structured curriculum could help QI teams improve the context supporting their QI work. An exploratory field study was conducted of 43 teams participating in a neonatal intensive care unit QI collaborative. Using a curriculum based on the Model for Understanding Success in Quality, teams identified gaps in their context and tested interventions to modify context. Surveys and self-reflective journals were analyzed to understand how teams developed changes to modify context. More than half (55%) targeted contextual improvements within the microsystem, focusing on motivation and culture. "Information sharing" interventions to communicate information about the project as a strategy to engage more staff were the most common interventions tested. Further study is needed to determine if efforts to modify context consistently lead to greater outcome improvements.

Appreciating the Nuance of Daily Symptom Variation to Individualize Patient Care.

INTRODUCTION: Improving symptoms for patients with chronic illness is difficult due to poor recall and imprecise assessments of therapeutic response to inform treatment decisions. Daily variation in symptoms may obscure subtle improvement or lead to erroneous associations between symptom changes and alteration in medication or dietary regimens. This may lead to mistaken impressions of treatment efficacy (or inefficacy). Mobile health technologies that collect daily patient reported outcome (PRO) data have the potential to improve care by providing more detailed information for clinical decision-making in practice and may facilitate conducting single subject (n-of-1) trials. METHODS: Interrupted time series to prototype mobile health enabled data collection for three patients. We recruited pediatric patients with established inflammatory bowel disease who had persistent symptoms. Based on their self-identified most troubling symptoms, patients were sent customized, daily-automated text messages to assess the extent of their symptoms. Standardized, PRO Measurement Information System (PROMIS) surveys were deployed weekly. Individual statistical process control charts were used to assess variation. Patients met with physicians regularly to interpret their data jointly. RESULTS: We report the experience of 3 patients with inflammatory bowel disease, each with different symptoms. Daily symptom monitoring uncovered important patterns, some of which even patients were unaware before reviewing their symptom data. Important associations were found between symptom variation and changes in medications and diet. PROMIS survey results assessed longitudinally accurately reflected changes in patient symptoms. CONCLUSIONS: We demonstrated how PROs can be implemented in practice. Monitoring and analyzing daily symptom data, using both customized and standard PROs, has the potential to detect meaningful variation in symptom patterns, which can inform clinical decision-making or can facilitate conducting formal n-of-1 trials to further improve outcomes.