Effect of Digital Early Warning Scores on Hospital Vital Sign Observation Protocol Adherence: Stepped-Wedge Evaluation.

BACKGROUND: Early warning scores (EWS) are routinely used in hospitals to assess a patient's risk of deterioration. EWS are traditionally recorded on paper observation charts but are increasingly recorded digitally. In either case, evidence for the clinical effectiveness of such scores is mixed, and previous studies have not considered whether EWS leads to changes in how deteriorating patients are managed. OBJECTIVE: This study aims to examine whether the introduction of a digital EWS system was associated with more frequent observation of patients with abnormal vital signs, a precursor to earlier clinical intervention. METHODS: We conducted a 2-armed stepped-wedge study from February 2015 to December 2016, over 4 hospitals in 1 UK hospital trust. In the control arm, vital signs were recorded using paper observation charts. In the intervention arm, a digital EWS system was used. The primary outcome measure was time to next observation (TTNO), defined as the time between a patient's first elevated EWS (EWS ≥3) and subsequent observations set. Secondary outcomes were time to death in the hospital, length of stay, and time to unplanned intensive care unit admission. Differences between the 2 arms were analyzed using a mixed-effects Cox model. The usability of the system was assessed using the system usability score survey. RESULTS: We included 12,802 admissions, 1084 in the paper (control) arm and 11,718 in the digital EWS (intervention) arm. The system usability score was 77.6, indicating good usability. The median TTNO in the control and intervention arms were 128 (IQR 73-218) minutes and 131 (IQR 73-223) minutes, respectively. The corresponding hazard ratio for TTNO was 0.99 (95% CI 0.91-1.07; P=.73). CONCLUSIONS: We demonstrated strong clinical engagement with the system. We found no difference in any of the predefined patient outcomes, suggesting that the introduction of a highly usable electronic system can be achieved without impacting clinical care. Our findings contrast with previous claims that digital EWS systems are associated with improvement in clinical outcomes. Future research should investigate how digital EWS systems can be integrated with new clinical pathways adjusting staff behaviors to improve patient outcomes.

Wearables alone will not eliminate failure to rescue.

Surveys suggest that anaesthesiologists believe that continuous monitoring with wearables will lead to improved patient outcomes. However, evidence suggests that several critical factors, including timely recognition of physiological problems, the presence of a trained team to respond to the alerts, and that the alerts occur far in advance of the deterioration, are required before overall improvement can occur. Wearables alone will not change patients' outcomes, they must be implemented as part of a system change that takes advantage of the higher frequency observations that continuous monitoring provides.

An impedance pneumography signal quality index: Design, assessment and application to respiratory rate monitoring.

Impedance pneumography (ImP) is widely used for respiratory rate (RR) monitoring. However, ImP-derived RRs can be imprecise. The aim of this study was to develop a signal quality index (SQI) for the ImP signal, and couple it with a RR algorithm, to improve RR monitoring. An SQI was designed which identifies candidate breaths and assesses signal quality using: the variation in detected breath durations, how well peaks and troughs are defined, and the similarity of breath morphologies. The SQI categorises 32 s signal segments as either high or low quality. Its performance was evaluated using two critical care datasets. RRs were estimated from high-quality segments using a RR algorithm, and compared with reference RRs derived from manual annotations. The SQI had a sensitivity of 77.7 %, and specificity of 82.3 %. RRs estimated from segments classified as high quality were accurate and precise, with mean absolute errors of 0.21 and 0.40 breaths per minute (bpm) on the two datasets. Clinical monitor RRs were significantly less precise. The SQI classified 34.9 % of real-world data as high quality. In conclusion, the proposed SQI accurately identifies high-quality segments, and RRs estimated from those segments are precise enough for clinical decision making. This SQI may improve RR monitoring in critical care. Further work should assess it with wearable sensor data.

Performance of universal early warning scores in different patient subgroups and clinical settings: a systematic review.

OBJECTIVE: To assess predictive performance of universal early warning scores (EWS) in disease subgroups and clinical settings. DESIGN: Systematic review. DATA SOURCES: Medline, CINAHL, Embase and Cochrane database of systematic reviews from 1997 to 2019. INCLUSION CRITERIA: Randomised trials and observational studies of internal or external validation of EWS to predict deterioration (mortality, intensive care unit (ICU) transfer and cardiac arrest) in disease subgroups or clinical settings. RESULTS: We identified 770 studies, of which 103 were included. Study designs and methods were inconsistent, with significant risk of bias (high: n=16 and unclear: n=64 and low risk: n=28). There were only two randomised trials. There was a high degree of heterogeneity in all subgroups and in national early warning score (I2=72%-99%). Predictive accuracy (mean area under the curve; 95% CI) was highest in medical (0.74; 0.74 to 0.75) and surgical (0.77; 0.75 to 0.80) settings and respiratory diseases (0.77; 0.75 to 0.80). Few studies evaluated EWS in specific diseases, for example, cardiology (n=1) and respiratory (n=7). Mortality and ICU transfer were most frequently studied outcomes, and cardiac arrest was least examined (n=8). Integration with electronic health records was uncommon (n=9). CONCLUSION: Methodology and quality of validation studies of EWS are insufficient to recommend their use in all diseases and all clinical settings despite good performance of EWS in some subgroups. There is urgent need for consistency in methods and study design, following consensus guidelines for predictive risk scores. Further research should consider specific diseases and settings, using electronic health record data, prior to large-scale implementation. PROSPERO REGISTRATION NUMBER: PROSPERO CRD42019143141.

Implementing a system for the real-time risk assessment of patients considered for intensive care.

BACKGROUND: Delay in identifying deterioration in hospitalised patients is associated with delayed admission to an intensive care unit (ICU) and poor outcomes. For the HAVEN project (HICF ref.: HICF-R9-524), we have developed a mathematical model that identifies deterioration in hospitalised patients in real time and facilitates the intervention of an ICU outreach team. This paper describes the system that has been designed to implement the model. We have used innovative technologies such as Portable Format for Analytics (PFA) and Open Services Gateway initiative (OSGi) to define the predictive statistical model and implement the system respectively for greater configurability, reliability, and availability. RESULTS: The HAVEN system has been deployed as part of a research project in the Oxford University Hospitals NHS Foundation Trust. The system has so far processed > 164,000 vital signs observations and > 68,000 laboratory results for > 12,500 patients and the algorithm generated score is being evaluated to review patients who are under consideration for transfer to ICU. No clinical decisions are being made based on output from the system. The HAVEN score has been computed using a PFA model for all these patients. The intent is that this score will be displayed on a graphical user interface for clinician review and response. CONCLUSIONS: The system uses a configurable PFA model to compute the HAVEN score which makes the system easily upgradable in terms of enhancing systems' predictive capability. Further system enhancements are planned to handle new data sources and additional management screens.

Early warning scores for detecting deterioration in adult hospital patients: systematic review and critical appraisal of methodology.

OBJECTIVE: To provide an overview and critical appraisal of early warning scores for adult hospital patients. DESIGN: Systematic review. DATA SOURCES: Medline, CINAHL, PsycInfo, and Embase until June 2019. ELIGIBILITY CRITERIA FOR STUDY SELECTION: Studies describing the development or external validation of an early warning score for adult hospital inpatients. RESULTS: 13 171 references were screened and 95 articles were included in the review. 11 studies were development only, 23 were development and external validation, and 61 were external validation only. Most early warning scores were developed for use in the United States (n=13/34, 38%) and the United Kingdom (n=10/34, 29%). Death was the most frequent prediction outcome for development studies (n=10/23, 44%) and validation studies (n=66/84, 79%), with different time horizons (the most frequent was 24 hours). The most common predictors were respiratory rate (n=30/34, 88%), heart rate (n=28/34, 83%), oxygen saturation, temperature, and systolic blood pressure (all n=24/34, 71%). Age (n=13/34, 38%) and sex (n=3/34, 9%) were less frequently included. Key details of the analysis populations were often not reported in development studies (n=12/29, 41%) or validation studies (n=33/84, 39%). Small sample sizes and insufficient numbers of event patients were common in model development and external validation studies. Missing data were often discarded, with just one study using multiple imputation. Only nine of the early warning scores that were developed were presented in sufficient detail to allow individualised risk prediction. Internal validation was carried out in 19 studies, but recommended approaches such as bootstrapping or cross validation were rarely used (n=4/19, 22%). Model performance was frequently assessed using discrimination (development n=18/22, 82%; validation n=69/84, 82%), while calibration was seldom assessed (validation n=13/84, 15%). All included studies were rated at high risk of bias. CONCLUSIONS: Early warning scores are widely used prediction models that are often mandated in daily clinical practice to identify early clinical deterioration in hospital patients. However, many early warning scores in clinical use were found to have methodological weaknesses. Early warning scores might not perform as well as expected and therefore they could have a detrimental effect on patient care. Future work should focus on following recommended approaches for developing and evaluating early warning scores, and investigating the impact and safety of using these scores in clinical practice. SYSTEMATIC REVIEW REGISTRATION: PROSPERO CRD42017053324.

Patient centred variables with univariate associations with unplanned ICU admission: a systematic review.

BACKGROUND: Multiple predictive scores using Electronic Patient Record data have been developed for hospitalised patients at risk of clinical deterioration. Methods used to select patient centred variables for inclusion in these scores varies. We performed a systematic review to describe univariate associations with unplanned Intensive Care Unit (ICU) admission with the aim of assisting model development for future scores that predict clinical deterioration. METHODS: Data sources were MEDLINE, EMBASE, CINAHL, CENTRAL and the Cochrane Database of Systematic Reviews. Included studies were published since 2000 describing an association between patient centred variables and unplanned ICU admission determined using univariate analysis. Two authors independently screened titles, abstracts and full texts against inclusion and exclusion criteria. DistillerSR (Evidence Partners, Canada, Ottawa, Ontario) software was used to manage the data and identify duplicate search results. All screening and data extraction forms were implemented within DistillerSR. Study quality was assessed using an adapted version of the Newcastle-Ottawa Scale. Variables were analysed for strength of association with unplanned ICU admission. RESULTS: The database search yielded 1520 unique studies; 1462 were removed after title and abstract review; 57 underwent full text screening; 16 studies were included. One hundred and eighty nine variables with an evaluated univariate association with unplanned ICU admission were described. DISCUSSION: Being male, increasing age, a history of congestive cardiac failure or diabetes, a diagnosis of hepatic disease or having abnormal vital signs were all strongly associated with ICU admission. CONCLUSION: These findings will assist variable selection during the development of future models predicting unplanned ICU admission. TRIAL REGISTRATION: This study is a component of a larger body of work registered in the ISRCTN registry ( ISRCTN12518261 ).

Breathing Rate Estimation From the Electrocardiogram and Photoplethysmogram: A Review.

Breathing rate (BR) is a key physiological parameter used in a range of clinical settings. Despite its diagnostic and prognostic value, it is still widely measured by counting breaths manually. A plethora of algorithms have been proposed to estimate BR from the electrocardiogram (ECG) and pulse oximetry (photoplethysmogram, PPG) signals. These BR algorithms provide opportunity for automated, electronic, and unobtrusive measurement of BR in both healthcare and fitness monitoring. This paper presents a review of the literature on BR estimation from the ECG and PPG. First, the structure of BR algorithms and the mathematical techniques used at each stage are described. Second, the experimental methodologies that have been used to assess the performance of BR algorithms are reviewed, and a methodological framework for the assessment of BR algorithms is presented. Third, we outline the most pressing directions for future research, including the steps required to use BR algorithms in wearable sensors, remote video monitoring, and clinical practice.

Early warning scores for detecting deterioration in adult hospital patients: a systematic review protocol.

INTRODUCTION: Early warning scores (EWSs) are used extensively to identify patients at risk of deterioration in hospital. Previous systematic reviews suggest that studies which develop EWSs suffer methodological shortcomings and consequently may fail to perform well. The reviews have also identified that few validation studies exist to test whether the scores work in other settings. We will aim to systematically review papers describing the development or validation of EWSs, focusing on methodology, generalisability and reporting. METHODS: We will identify studies that describe the development or validation of EWSs for adult hospital inpatients. Each study will be assessed for risk of bias using the Prediction model Risk of Bias ASsessment Tool (PROBAST). Two reviewers will independently extract information. A narrative synthesis and descriptive statistics will be used to answer the main aims of the study which are to assess and critically appraise the methodological quality of the EWS, to describe the predictors included in the EWSs and to describe the reported performance of EWSs in external validation. ETHICS AND DISSEMINATION: This systematic review will only investigate published studies and therefore will not directly involve patient data. The review will help to establish whether EWSs are fit for purpose and make recommendations to improve the quality of future research in this area. PROSPERO REGISTRATION NUMBER: CRD42017053324.

A ward-based time study of paper and electronic documentation for recording vital sign observations.

OBJECTIVE: To investigate time differences in recording observations and an early warning score using traditional paper charts and a novel e-Obs system in clinical practice. METHODS: Researchers observed the process of recording observations and early warning scores across 3 wards in 2 university teaching hospitals immediately before and after introduction of the e-Obs system. The process of recording observations included both measurement and documentation of vital signs. Interruptions were timed and subtracted from the measured process duration. Multilevel modeling was used to compensate for potential confounding factors. RESULTS: In all, 577 nurse events were observed (281 paper, 296 e-Obs). The geometric mean time to take a complete set of vital signs was 215 s (95% confidence interval [CI], 177 s-262 s) on paper, and 150 s (95% CI, 130 s-172 s) electronically. The treatment effect ratio was 0.70 (95% CI, 0.57-0.85, P < .001). The treatment effect ratio in ward 1 was 0.37 (95% CI, 0.26-0.53), in ward 2 was 0.98 (95% CI, 0.70-1.38), and in ward 3 was 0.93 (95% CI, 0.66-1.33). DISCUSSION: Introduction of an e-Obs system was associated with a statistically significant reduction in overall time to measure and document vital signs electronically compared to paper documentation. The reductions in time varied among wards and were of clinical significance on only 1 of 3 wards studied. CONCLUSION: Our results suggest that introduction of an e-Obs system could lower nursing workload as well as increase documentation quality.

Extraction of respiratory signals from the electrocardiogram and photoplethysmogram: technical and physiological determinants.

OBJECTIVE: Breathing rate (BR) can be estimated by extracting respiratory signals from the electrocardiogram (ECG) or photoplethysmogram (PPG). The extracted respiratory signals may be influenced by several technical and physiological factors. In this study, our aim was to determine how technical and physiological factors influence the quality of respiratory signals. APPROACH: Using a variety of techniques 15 respiratory signals were extracted from the ECG, and 11 from PPG signals collected from 57 healthy subjects. The quality of each respiratory signal was assessed by calculating its correlation with a reference oral-nasal pressure respiratory signal using Pearson's correlation coefficient. MAIN RESULTS: Relevant results informing device design and clinical application were obtained. The results informing device design were: (i) seven out of 11 respiratory signals were of higher quality when extracted from finger PPG compared to ear PPG; (ii) laboratory equipment did not provide higher quality of respiratory signals than a clinical monitor; (iii) the ECG provided higher quality respiratory signals than the PPG; (iv) during downsampling of the ECG and PPG significant reductions in quality were first observed at sampling frequencies of <250 Hz and <16 Hz respectively. The results informing clinical application were: (i) frequency modulation-based respiratory signals were generally of lower quality in elderly subjects compared to young subjects; (ii) the qualities of 23 out of 26 respiratory signals were reduced at elevated BRs; (iii) there were no differences associated with gender. SIGNIFICANCE: Recommendations based on the results are provided regarding device designs for BR estimation, and clinical applications. The dataset and code used in this study are publicly available.

An assessment of algorithms to estimate respiratory rate from the electrocardiogram and photoplethysmogram.

Over 100 algorithms have been proposed to estimate respiratory rate (RR) from the electrocardiogram (ECG) and photoplethysmogram (PPG). As they have never been compared systematically it is unclear which algorithm performs the best. Our primary aim was to determine how closely algorithms agreed with a gold standard RR measure when operating under ideal conditions. Secondary aims were: (i) to compare algorithm performance with IP, the clinical standard for continuous respiratory rate measurement in spontaneously breathing patients; (ii) to compare algorithm performance when using ECG and PPG; and (iii) to provide a toolbox of algorithms and data to allow future researchers to conduct reproducible comparisons of algorithms. Algorithms were divided into three stages: extraction of respiratory signals, estimation of RR, and fusion of estimates. Several interchangeable techniques were implemented for each stage. Algorithms were assembled using all possible combinations of techniques, many of which were novel. After verification on simulated data, algorithms were tested on data from healthy participants. RRs derived from ECG, PPG and IP were compared to reference RRs obtained using a nasal-oral pressure sensor using the limits of agreement (LOA) technique. 314 algorithms were assessed. Of these, 270 could operate on either ECG or PPG, and 44 on only ECG. The best algorithm had 95% LOAs of -4.7 to 4.7 bpm and a bias of 0.0 bpm when using the ECG, and -5.1 to 7.2 bpm and 1.0 bpm when using PPG. IP had 95% LOAs of -5.6 to 5.2 bpm and a bias of -0.2 bpm. Four algorithms operating on ECG performed better than IP. All high-performing algorithms consisted of novel combinations of time domain RR estimation and modulation fusion techniques. Algorithms performed better when using ECG than PPG. The toolbox of algorithms and data used in this study are publicly available.

Evaluation of the effects of implementing an electronic early warning score system: protocol for a stepped wedge study.

BACKGROUND: An Early Warning Score is a clinical risk score based upon vital signs intended to aid recognition of patients in need of urgent medical attention. The use of an escalation of care policy based upon an Early Warning Score is mandated as the standard of practice in British hospitals. Electronic systems for recording vital sign observations and Early Warning Score calculation offer theoretical benefits over paper-based systems. However, the evidence for their clinical benefit is limited. Previous studies have shown inconsistent results. The majority have employed a "before and after" study design, which may be strongly confounded by simultaneously occurring events. This study aims to examine how the implementation of an electronic early warning score system, System for Notification and Documentation (SEND), affects the recognition of clinical deterioration occurring in hospitalised adult patients. METHODS: This study is a non-randomised stepped wedge evaluation carried out across the four hospitals of the Oxford University Hospitals NHS Trust, comparing charting on paper and charting using SEND. We assume that more frequent monitoring of acutely ill patients is associated with better recognition of patient deterioration. The primary outcome measure is the time between a patient's first observations set with an Early Warning Score above the alerting threshold and their subsequent set of observations. Secondary outcome measures are in-hospital mortality, cardiac arrest and Intensive Care admission rates, hospital length of stay and system usability measured using the System Usability Scale. We will also measure Intensive Care length of stay, Intensive Care mortality, Acute Physiology and Chronic Health Evaluation (APACHE) II acute physiology score on admission, to examine whether the introduction of SEND has any effect on Intensive Care-related outcomes. DISCUSSION: The development of this protocol has been informed by guidance from the Agency for Healthcare Research and Quality (AHRQ) Health Information Technology Evaluation Toolkit and Delone and McLeans's Model of Information System Success. Our chosen trial design, a stepped wedge study, is well suited to the study of a phased roll out. The choice of primary endpoint is challenging. We have selected the time from the first triggering observation set to the subsequent observation set. This has the benefit of being easy to measure on both paper and electronic charting and having a straightforward interpretation. We have collected qualitative measures of system quality via a user questionnaire and organisational descriptors to help readers understand the context in which SEND has been implemented.

SEND: a system for electronic notification and documentation of vital sign observations.

BACKGROUND: Recognising the limitations of a paper-based approach to documenting vital sign observations and responding to national clinical guidelines, we have explored the use of an electronic solution that could improve the quality and safety of patient care. We have developed a system for recording vital sign observations at the bedside, automatically calculating an Early Warning Score, and saving data such that it is accessible to all relevant clinicians within a hospital trust. We have studied current clinical practice of using paper observation charts, and attempted to streamline the process. We describe our user-focussed design process, and present the key design decisions prior to describing the system in greater detail. RESULTS: The system has been deployed in three pilot clinical areas over a period of 9 months. During this time, vital sign observations were recorded electronically using our system. Analysis of the number of observations recorded (21,316 observations) and the number of active users (111 users) confirmed that the system is being used for routine clinical observations. Feedback from clinical end-users was collected to assess user acceptance of the system. This resulted in a System Usability Scale score of 77.8, indicating high user acceptability. CONCLUSIONS: Our system has been successfully piloted, and is in the process of full implementation throughout adult inpatient clinical areas in the Oxford University Hospitals. Whilst our results demonstrate qualitative acceptance of the system, its quantitative effect on clinical care is yet to be evaluated.

Signal-quality indices for the electrocardiogram and photoplethysmogram: derivation and applications to wireless monitoring.

The identification of invalid data in recordings obtained using wearable sensors is of particular importance since data obtained from mobile patients is, in general, noisier than data obtained from nonmobile patients. In this paper, we present a signal quality index (SQI), which is intended to assess whether reliable heart rates (HRs) can be obtained from electrocardiogram (ECG) and photoplethysmogram (PPG) signals collected using wearable sensors. The algorithms were validated on manually labeled data. Sensitivities and specificities of 94% and 97% were achieved for the ECG and 91% and 95% for the PPG. Additionally, we propose two applications of the SQI. First, we demonstrate that, by using the SQI as a trigger for a power-saving strategy, it is possible to reduce the recording time by up to 94% for the ECG and 93% for the PPG with only minimal loss of valid vital-sign data. Second, we demonstrate how an SQI can be used to reduce the error in the estimation of respiratory rate (RR) from the PPG. The performance of the two applications was assessed on data collected from a clinical study on hospital patients who were able to walk unassisted.

The digital patient.

Despite efforts, the detection of patients who are deteriorating in hospital is often later than it should be. Several technologies could provide the basis of a solution. Recording of vital signs could be improved by both automated transmission of the measured parameters to an electronic patient record and the use of unobtrusive wearable monitors that track the patient's physiology continuously. Electronic charting systems could make the recorded vital signs readily available for further processing. Software algorithms could identify such patients with greater sensitivity and specificity than the existing, paper-based track-and-trigger systems. Electronic storage of vital signs also makes intelligent alerting and remote patient surveillance possible. However, the potential of these technologies depends strongly on implementation, with poor-quality deployment likely to worsen patient care.