A Comparison of the Quick Sequential (Sepsis-Related) Organ Failure Assessment Score and the National Early Warning Score in Non-ICU Patients With/Without Infection.

OBJECTIVES: The Sepsis-3 task force recommended the quick Sequential (Sepsis-Related) Organ Failure Assessment score for identifying patients with suspected infection who are at greater risk of poor outcomes, but many hospitals already use the National Early Warning Score to identify high-risk patients, irrespective of diagnosis. We sought to compare the performance of quick Sequential (Sepsis-Related) Organ Failure Assessment and National Early Warning Score in hospitalized, non-ICU patients with and without an infection. DESIGN: Retrospective cohort study. SETTING: Large U.K. General Hospital. PATIENTS: Adults hospitalized between January 1, 2010, and February 1, 2016. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: We applied the quick Sequential (Sepsis-Related) Organ Failure Assessment score and National Early Warning Score to 5,435,344 vital signs sets (241,996 hospital admissions). Patients were categorized as having no infection, primary infection, or secondary infection using International Classification of Diseases, 10th Edition codes. National Early Warning Score was significantly better at discriminating in-hospital mortality, irrespective of infection status (no infection, National Early Warning Score 0.831 [0.825-0.838] vs quick Sequential [Sepsis-Related] Organ Failure Assessment 0.688 [0.680-0.695]; primary infection, National Early Warning Score 0.805 [0.799-0.812] vs quick Sequential [Sepsis-Related] Organ Failure Assessment 0.677 [0.670-0.685]). Similarly, National Early Warning Score performed significantly better in all patient groups (all admissions, emergency medicine admissions, and emergency surgery admissions) for all outcomes studied. Overall, quick Sequential (Sepsis-Related) Organ Failure Assessment performed no better, and often worse, in admissions with infection than without. CONCLUSIONS: The National Early Warning Score outperforms the quick Sequential (Sepsis-Related) Organ Failure Assessment score, irrespective of infection status. These findings suggest that quick Sequential (Sepsis-Related) Organ Failure Assessment should be reevaluated as the system of choice for identifying non-ICU patients with suspected infection who are at greater risk of poor outcome.

National Early Warning Score Accurately Discriminates the Risk of Serious Adverse Events in Patients With Liver Disease.

BACKGROUND & AIMS: The National Early Warning Score (NEWS) is used to identify deteriorating adult hospital inpatients. However, it includes physiological parameters frequently altered in patients with cirrhosis. We aimed to assess the performance of the NEWS in acute and chronic liver diseases. METHODS: We collected vital signs, recorded in real time, from completed consecutive admissions of patients 16 years or older to a large acute-care hospital in Southern England, from January 1, 2010, through October 31, 2014. Using International Classification of Diseases, 10th revision, codes, we categorized patients as having primary liver disease, secondary liver disease, or none. For patients with liver disease, 2 analysis groups were developed: the first was based on clinical group (such as acute or chronic, alcohol-induced, or associated with portal hypertension), and the second was based on a summary of liver-related, hospital-level mortality indicator diagnoses. For each, we compared the abilities of the NEWS and 34 other early warning scores to discriminate 24-hour mortality, cardiac arrest, or unanticipated admission to the intensive care unit using the area under the receiver operating characteristic (AUROC) curve and early warning score efficiency curve analyses. RESULTS: The NEWS identified patients with primary, nonprimary, and no diagnoses of liver disease with AUROC values of 0.873 (95% CI, 0.860-0.886), 0.898 (95% CI, 0.891-0.905), and 0.879 (95% CI, 0.877-0.881), respectively. High AUROC values were also obtained for all clinical subgroups; the NEWS identified patients with alcohol-related liver disease with an AUROC value of 0.927 (95% CI, 0.912-0.941). The NEWS identified patients with liver diseases with higher AUROC values than other early warning scoring systems. CONCLUSIONS: The NEWS accurately discriminates patients at risk of death, admission to the intensive care unit, or cardiac arrest within a 24-hour period for a range of liver-related diagnoses. Its widespread use provides a ready-made, easy-to-use option for identifying patients with liver disease who require early assessment and intervention, without the need to modify parameters, weightings, or escalation criteria.

A Comparison of the Ability of the Physiologic Components of Medical Emergency Team Criteria and the U.K. National Early Warning Score to Discriminate Patients at Risk of a Range of Adverse Clinical Outcomes.

OBJECTIVE: To compare the ability of medical emergency team criteria and the National Early Warning Score to discriminate cardiac arrest, unanticipated ICU admission and death within 24 hours of a vital signs measurement, and to quantify the associated workload. DESIGN: Retrospective cohort study. SETTING: A large U.K. National Health Service District General Hospital. PATIENTS: Adults hospitalized from May 25, 2011, to December 31, 2013. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: We applied the National Early Warning Score and 44 sets of medical emergency team criteria to a database of 2,245,778 vital signs sets (103,998 admissions). The National Early Warning Score's performance was assessed using the area under the receiver-operating characteristic curve and compared with sensitivity/specificity for different medical emergency team criteria. Area under the receiver-operating characteristic curve (95% CI) for the National Early Warning Score for the combined outcome (i.e., death, cardiac arrest, or unanticipated ICU admission) was 0.88 (0.88-0.88). A National Early Warning Score value of 7 had sensitivity/specificity values of 44.5% and 97.4%, respectively. For the 44 sets of medical emergency team criteria studied, sensitivity ranged from 19.6% to 71.2% and specificity from 71.5% to 98.5%. For all outcomes, the position of the National Early Warning Score receiver-operating characteristic curve was above and to the left of all medical emergency team criteria points, indicating better discrimination. Similarly, the positions of all medical emergency team criteria points were above and to the left of the National Early Warning Score efficiency curve, indicating higher workloads (trigger rates). CONCLUSIONS: When medical emergency team systems are compared to a National Early Warning Score value of greater than or equal to 7, some medical emergency team systems have a higher sensitivity than National Early Warning Score values of greater than or equal to 7. However, all of these medical emergency team systems have a lower specificity and would generate greater workloads.

Early warning scores: unravelling detection and escalation.

PURPOSE: The purpose of this paper is to increase understanding of how patient deterioration is detected and how clinical care escalates when early warning score (EWS) systems are used. DESIGN/METHODOLOGY/APPROACH: The authors critically review a recent National Early Warning Score paper published in IJHCQA using personal experience and EWS-related publications, and debate the difference between detection and escalation. FINDINGS: Incorrect EWS choice or poorly understood EWS escalation may result in unnecessary workloads forward and responding staff. PRACTICAL IMPLICATIONS: EWS system implementers may need to revisit their guidance materials; medical and nurse educators may need to expand the curriculum to improve EWS system understanding and use. ORIGINALITY/VALUE: The paper raises the EWS debate and alerts EWS users that scrutiny is required.

Can binary early warning scores perform as well as standard early warning scores for discriminating a patient's risk of cardiac arrest, death or unanticipated intensive care unit admission?

INTRODUCTION: Although the weightings to be summed in an early warning score (EWS) calculation are small, calculation and other errors occur frequently, potentially impacting on hospital efficiency and patient care. Use of a simpler EWS has the potential to reduce errors. METHODS: We truncated 36 published 'standard' EWSs so that, for each component, only two scores were possible: 0 when the standard EWS scored 0 and 1 when the standard EWS scored greater than 0. Using 1564,153 vital signs observation sets from 68,576 patient care episodes, we compared the discrimination (measured using the area under the receiver operator characteristic curve--AUROC) of each standard EWS and its truncated 'binary' equivalent. RESULTS: The binary EWSs had lower AUROCs than the standard EWSs in most cases, although for some the difference was not significant. One system, the binary form of the National Early Warning System (NEWS), had significantly better discrimination than all standard EWSs, except for NEWS. Overall, Binary NEWS at a trigger value of 3 would detect as many adverse outcomes as are detected by NEWS using a trigger of 5, but would require a 15% higher triggering rate. CONCLUSIONS: The performance of Binary NEWS is only exceeded by that of standard NEWS. It may be that Binary NEWS, as a simplified system, can be used with fewer errors. However, its introduction could lead to significant increases in workload for ward and rapid response team staff. The balance between fewer errors and a potentially greater workload needs further investigation.

Are observation selection methods important when comparing early warning score performance?

INTRODUCTION: Sicker patients generally have more vital sign assessments, particularly immediately before an adverse outcome, and especially if the vital sign monitoring schedule is driven by an early warning score (EWS) value. This lack of independence could influence the measured discriminatory performance of an EWS. METHODS: We used a population of 1564,143 consecutive vital signs observation sets collected as a routine part of patients' care. We compared 35 published EWSs for their discrimination of the risk of death within 24h of an observation set using (1) all observations in our dataset, (2) one observation per patient care episode, chosen at random and (3) one observation per patient care episode, chosen as the closest to a randomly selected point in time in each episode. We compared the area under the ROC curve (AUROC) as a measure of discrimination for each of the 35 EWSs under each observation selection method and looked for changes in their rank order. RESULTS: There were no significant changes in rank order of the EWSs based on AUROC between the different observation selection methods, except for one EWS that included age among its components. Whichever method of observation selection was used, the National Early Warning Score (NEWS) showed the highest discrimination of risk of death within 24h. AUROCs were higher when only one observation set was used per episode of care (significantly higher for many EWSs, including NEWS). CONCLUSIONS: Vital sign measurements can be treated as if they are independent - multiple observations can be used from each episode of care--when comparing the performance and ranking of EWSs, provided no EWS includes age.

Aggregate National Early Warning Score (NEWS) values are more important than high scores for a single vital signs parameter for discriminating the risk of adverse outcomes.

INTRODUCTION: The Royal College of Physicians (RCPL) National Early Warning Score (NEWS) escalates care to a doctor at NEWS values of ≥5 and when the score for any single vital sign is 3. METHODS: We calculated the 24-h risk of serious clinical outcomes for vital signs observation sets with NEWS values of 3, 4 and 5, separately determining risks when the score did/did not include a single score of 3. We compared workloads generated by the RCPL's escalation protocol and for aggregate NEWS value alone. RESULTS: Aggregate NEWS values of 3 or 4 (n=142,282) formed 15.1% of all vital signs sets measured; those containing a single vital sign scoring 3 (n=36,207) constituted 3.8% of all sets. Aggregate NEWS values of either 3 or 4 with a component score of 3 have significantly lower risks (OR: 0.26 and 0.53) than an aggregate value of 5 (OR: 1.0). Escalating care to a doctor when any single component of NEWS scores 3 compared to when aggregate NEWS values ≥5, would have increased doctors' workload by 40% with only a small increase in detected adverse outcomes from 2.99 to 3.08 per day (a 3% improvement in detection). CONCLUSIONS: The recommended NEWS escalation protocol produces additional work for the bedside nurse and responding doctor, disproportionate to a modest benefit in increased detection of adverse outcomes. It may have significant ramifications for efficient staff resource allocation, distort patient safety focus and risk alarm fatigue. Our findings suggest that the RCPL escalation guidance warrants review.

Impact of introducing an electronic physiological surveillance system on hospital mortality.

BACKGROUND: Avoidable hospital mortality is often attributable to inadequate patient vital signs monitoring, and failure to recognise or respond to clinical deterioration. The processes involved with vital sign collection and charting; their integration, interpretation and analysis; and the delivery of decision support regarding subsequent clinical care are subject to potential error and/or failure. OBJECTIVE: To determine whether introducing an electronic physiological surveillance system (EPSS), specifically designed to improve the collection and clinical use of vital signs data, reduced hospital mortality. METHODS: A pragmatic, retrospective, observational study of seasonally adjusted in-hospital mortality rates in three main hospital specialties was undertaken before, during and after the sequential deployment and ongoing use of a hospital-wide EPSS in two large unconnected acute general hospitals in England. The EPSS, which uses wireless handheld computing devices, replaced a paper-based vital sign charting and clinical escalation system. RESULTS: During EPSS implementation, crude mortality fell from a baseline of 7.75% (2168/27,959) to 6.42% (1904/29,676) in one hospital (estimated 397 fewer deaths), and from 7.57% (1648/21,771) to 6.15% (1614/26,241) at the second (estimated 372 fewer deaths). At both hospitals, multiyear statistical process control analyses revealed abrupt and sustained mortality reductions, coincident with the deployment and increasing use of the system. The cumulative total of excess deaths reduced in all specialties with increasing use of the system across the hospital. CONCLUSIONS: The use of technology specifically designed to improve the accuracy, reliability and availability of patients' vital signs and early warning scores, and thereby the recognition of and response to patient deterioration, is associated with reduced mortality in this study.

Pain: a quality of care issue during patients' admission to hospital.

AIM: To determine the extent of clinically significant pain suffered by hospitalized patients during their stay and at discharge. BACKGROUND: The management of pain in hospitals continues to be problematic, despite long-standing awareness of the problem and improvements, e.g. acute pain teams and patient-controlled analgesia, epidural analgesia. Poorly managed pain, especially acute pain, often leads to adverse physical and psychological outcomes including persistent pain and disability. A systems approach may improve the management of pain in hospitals. DESIGN: A descriptive cross-sectional exploratory design. METHOD: A large electronic pain score database of vital signs and pain scores was interrogated between 1st January 2010 and 31st December 2010 to establish the proportion of hospital inpatient stays with clinically significant pain during the hospital stay and at discharge. FINDINGS: A total of 810,774 pain scores were analysed, representing 38,451 patient stays. Clinically significant pain was present in 38·4% of patient stays. Across surgical categories, 54·0% of emergency admissions experienced clinically significant pain, compared with 48·0% of elective admissions. Medical areas had a summary figure of 26·5%. For 30% patients, clinically significant pain was followed by a consecutive clinically significant pain score. Only 0·2% of pain assessments were made independently of vital signs. CONCLUSION: Reducing the risk of long-term persistent pain should be seen as integral to improving patient safety and can be achieved by harnessing organizational pain management processes with quality improvement initiatives. The assessment of pain alongside vital signs should be reviewed. Setting quality targets for pain are essential for improving the patient's experience.

Decision-tree early warning score (DTEWS) validates the design of the National Early Warning Score (NEWS).

AIM OF STUDY: To compare the performance of a human-generated, trial and error-optimised early warning score (EWS), i.e., National Early Warning Score (NEWS), with one generated entirely algorithmically using Decision Tree (DT) analysis. MATERIALS AND METHODS: We used DT analysis to construct a decision-tree EWS (DTEWS) from a database of 198,755 vital signs observation sets collected from 35,585 consecutive, completed acute medical admissions. We evaluated the ability of DTEWS to discriminate patients at risk of cardiac arrest, unanticipated intensive care unit admission or death, each within 24h of a given vital signs observation. We compared the performance of DTEWS and NEWS using the area under the receiver-operating characteristic (AUROC) curve. RESULTS: The structures of DTEWS and NEWS were very similar. The AUROC (95% CI) for DTEWS for cardiac arrest, unanticipated ICU admission, death, and any of the outcomes, all within 24h, were 0.708 (0.669-0.747), 0.862 (0.852-0.872), 0.899 (0.892-0.907), and 0.877 (0.870-0.883), respectively. Values for NEWS were 0.722 (0.685-0.759) [cardiac arrest], 0.857 (0.847-0.868) [unanticipated ICU admission}, 0.894 (0.887-0.902) [death], and 0.873 (0.866-0.879) [any outcome]. CONCLUSIONS: The decision-tree technique independently validates the composition and weightings of NEWS. The DT approach quickly provided an almost identical EWS to NEWS, although one that admittedly would benefit from fine-tuning using clinical knowledge. We believe that DT analysis could be used to quickly develop candidate models for disease-specific EWSs, which may be required in future.

Index blood tests and national early warning scores within 24 hours of emergency admission can predict the risk of in-hospital mortality: a model development and validation study.

BACKGROUND: We explored the use of routine blood tests and national early warning scores (NEWS) reported within ±24 hours of admission to predict in-hospital mortality in emergency admissions, using empirical decision Tree models because they are intuitive and may ultimately be used to support clinical decision making. METHODOLOGY: A retrospective analysis of adult emergency admissions to a large acute hospital during April 2009 to March 2010 in the West Midlands, England, with a full set of index blood tests results (albumin, creatinine, haemoglobin, potassium, sodium, urea, white cell count and an index NEWS undertaken within ±24 hours of admission). We developed a Tree model by randomly splitting the admissions into a training (50%) and validation dataset (50%) and assessed its accuracy using the concordance (c-) statistic. Emergency admissions (about 30%) did not have a full set of index blood tests and/or NEWS and so were not included in our analysis. RESULTS: There were 23248 emergency admissions with a full set of blood tests and NEWS with an in-hospital mortality of 5.69%. The Tree model identified age, NEWS, albumin, sodium, white cell count and urea as significant (p<0.001) predictors of death, which described 17 homogeneous subgroups of admissions with mortality ranging from 0.2% to 60%. The c-statistic for the training model was 0.864 (95%CI 0.852 to 0.87) and when applied to the testing data set this was 0.853 (95%CI 0.840 to 0.866). CONCLUSIONS: An easy to interpret validated risk adjustment Tree model using blood test and NEWS taken within ±24 hours of admission provides good discrimination and offers a novel approach to risk adjustment which may potentially support clinical decision making. Given the nature of the clinical data, the results are likely to be generalisable but further research is required to investigate this promising approach.

Development and validation of a decision tree early warning score based on routine laboratory test results for the discrimination of hospital mortality in emergency medical admissions.

AIM OF STUDY: To build an early warning score (EWS) based exclusively on routinely undertaken laboratory tests that might provide early discrimination of in-hospital death and could be easily implemented on paper. MATERIALS AND METHODS: Using a database of combined haematology and biochemistry results for 86,472 discharged adult patients for whom the admission specialty was Medicine, we used decision tree (DT) analysis to generate a laboratory decision tree early warning score (LDT-EWS) for each gender. LDT-EWS was developed for a single set (n=3496) (Q1) and validated in 22 other discrete sets each of three months long (Q2, Q3…Q23) (total n=82,976; range of n=3428 to 4093) by testing its ability to discriminate in-hospital death using the area under the receiver-operating characteristic (AUROC) curve. RESULTS: The data generated slightly different models for male and female patients. The ranges of AUROC values (95% CI) for LDT-EWS with in-hospital death as the outcome for the validation sets Q2-Q23 were: 0.755 (0.727-0.783) (Q16) to 0.801 (0.776-0.826) [all patients combined, n=82,976]; 0.744 (0.704-0.784, Q16) to 0.824 (0.792-0.856, Q2) [39,591 males]; and 0.742 (0.707-0.777, Q10) to 0.826 (0.796-0.856, Q12) [43,385 females]. CONCLUSIONS: This study provides evidence that the results of commonly measured laboratory tests collected soon after hospital admission can be represented in a simple, paper-based EWS (LDT-EWS) to discriminate in-hospital mortality. We hypothesise that, with appropriate modification, it might be possible to extend the use of LDT-EWS throughout the patient's hospital stay.

Patterns in the recording of vital signs and early warning scores: compliance with a clinical escalation protocol.

BACKGROUND: The recognition of patient deterioration depends largely on identifying abnormal vital signs, yet little is known about the daily pattern of vital signs measurement and charting. METHODS: We compared the pattern of vital signs and VitalPAC Early Warning Score (ViEWS) data collected from admissions to all adult inpatient areas (except high care areas, such as critical care units) of a NHS district general hospital from 1 May 2010 to 30 April 2011, to the hospital's clinical escalation protocol. Main outcome measures were hourly and daily patterns of vital signs and ViEWS value documentation; numbers of vital signs in the periods 08:00-11:59 and 20:00-23:59 with subsequent vital signs recorded in the following 6 h; and time to next observation (TTNO) for vital signs recorded in the periods 08:00-11:59 and 20:00-23:59. RESULTS: 950 043 vital sign datasets were recorded. The daily pattern of observation documentation was not uniform; there were large morning and evening peaks, and lower night-time documentation. The pattern was identical on all days. 23.84% of vital sign datasets with ViEWS ≥ 9 were measured at night compared with 10.12-19.97% for other ViEWS values. 47.42% of patients with ViEWS=7-8 and 31.22% of those with ViEWS ≥ 9 in the period 20:00-23:59 did not have vital signs recorded in the following 6 h. TTNO decreased with increasing ViEWS value, but less than expected by the monitoring protocol. CONCLUSIONS: There was only partial adherence to the vital signs monitoring protocol. Sicker patients appear more likely to have vital signs measured overnight, but even their observations were often not followed by timely repeat assessments. The observed pattern of monitoring may reflect the impact of competing clinical priorities.

The ability of the National Early Warning Score (NEWS) to discriminate patients at risk of early cardiac arrest, unanticipated intensive care unit admission, and death.

INTRODUCTION: Early warning scores (EWS) are recommended as part of the early recognition and response to patient deterioration. The Royal College of Physicians recommends the use of a National Early Warning Score (NEWS) for the routine clinical assessment of all adult patients. METHODS: We tested the ability of NEWS to discriminate patients at risk of cardiac arrest, unanticipated intensive care unit (ICU) admission or death within 24h of a NEWS value and compared its performance to that of 33 other EWSs currently in use, using the area under the receiver-operating characteristic (AUROC) curve and a large vital signs database (n=198,755 observation sets) collected from 35,585 consecutive, completed acute medical admissions. RESULTS: The AUROCs (95% CI) for NEWS for cardiac arrest, unanticipated ICU admission, death, and any of the outcomes, all within 24h, were 0.722 (0.685-0.759), 0.857 (0.847-0.868), 0.894 (0.887-0.902), and 0.873 (0.866-0.879), respectively. Similarly, the ranges of AUROCs (95% CI) for the other 33 EWSs were 0.611 (0.568-0.654) to 0.710 (0.675-0.745) (cardiac arrest); 0.570 (0.553-0.568) to 0.827 (0.814-0.840) (unanticipated ICU admission); 0.813 (0.802-0.824) to 0.858 (0.849-0.867) (death); and 0.736 (0.727-0.745) to 0.834 (0.826-0.842) (any outcome). CONCLUSIONS: NEWS has a greater ability to discriminate patients at risk of the combined outcome of cardiac arrest, unanticipated ICU admission or death within 24h of a NEWS value than 33 other EWSs.