RESUMO
ObjectivesTo compare approaches for obtaining relative and absolute estimates of risk of 28-day COVID-19 mortality for adults in the general population of England in the context of changing levels of circulating infection. DesignThree designs were compared. (A) case-cohort which does not explicitly account for the time-changing prevalence of COVID-19 infection, (B) 28-day landmarking, a series of sequential overlapping sub-studies incorporating time-updating proxy measures of the prevalence of infection, and (C) daily landmarking. Regression models were fitted to predict 28-day COVID-19 mortality. SettingWorking on behalf of NHS England, we used clinical data from adult patients from all regions of England held in the TPP SystmOne electronic health record system, linked to Office for National Statistics (ONS) mortality data, using the OpenSAFELY platform. ParticipantsEligible participants were adults aged 18 or over, registered at a general practice using TPP software on 1st March 2020 with recorded sex, postcode and ethnicity. 11,972,947 individuals were included, and 7,999 participants experienced a COVID-19 related death. The study period lasted 100 days, ending 8th June 2020. PredictorsA range of demographic characteristics and comorbidities were used as potential predictors. Local infection prevalence was estimated with three proxies: modelled based on local prevalence and other key factors; rate of A&E COVID-19 related attendances; and rate of suspected COVID-19 cases in primary care. Main outcome measuresCOVID-19 related death. ResultsAll models discriminated well between patients who did and did not experience COVID-19 related death, with C-statistics ranging from 0.92-0.94. Accurate estimates of absolute risk required data on local infection prevalence, with modelled estimates providing the best performance. ConclusionsReliable estimates of absolute risk need to incorporate changing local prevalence of infection. Simple models can provide very good discrimination and may simplify implementation of risk prediction tools in practice.
RESUMO
ObjectivesTo consider the potential of the National Early Warning Score (NEWS2) for COVID-19 risk prediction on unplanned admission to hospital. DesignLogistic regression model development and validation study using a cohort of unplanned emergency medical admission to hospital. SettingYork Hospital (YH) as model development dataset and Scarborough Hospital (SH) as model validation dataset. ParticipantsUnplanned adult medical admissions discharged over 3 months (11 March 2020 to 13 June 2020) from two hospitals (YH for model development; SH for external model validation) based on admission NEWS2 electronically recorded within {+/-}24 hours of admission. We used logistic regression modelling to predict the risk of COVID-19 using NEWS2 (Model M0) versus enhanced cNEWS models which included age + sex (model M1) + subcomponents (including diastolic blood pressure + oxygen flow rate + oxygen scale) of NEWS2 (model M2). The ICD-10 code U071 was used to identify COVID-19 admissions. Model performance was evaluated according to discrimination (c statistic), calibration (graphically), and clinical usefulness at NEWS2 [≥]5. ResultsThe prevalence of COVID-19 was higher in SH (11.0%=277/2520) than YH (8.7%=343/3924) with higher index NEWS2 (3.2 vs 2.8) but similar in-hospital mortality (8.4% vs 8.2%). The c-statistics for predicting COVID-19 for cNEWS models (M1,M2) was substantially better than NEWS2 alone (M0) in development (M2: 0.78 (95%CI 0.75-0.80) vs M0 0.71 (95%CI 0.68-0.74)) and validation datasets (M2: 0.72 (95%CI 0.69-0.75) vs M0 0.65 (95%CI 0.61-0.68)). Model M2 had better calibration than Model M0 with improved sensitivity (M2: 57% (95%CI 51%-63%) vs M0 44% (95%CI 38%-50%)) and similar specificity (M2: 76% (95%CI 74%-78%) vs M0 75% (95%CI 73%-77%)) for validation dataset at NEWS2[≥]5. ConclusionsModel M2 is reasonably accurate for predicting the on-admission risk of COVID-19. It may be clinically useful for an early warning system at the time of admission especially to triage large numbers of unplanned hospital admissions.
RESUMO
ObjectiveTo develop and externally validate COVID-19 Estimated Risk (COVER) scores that quantify a patients risk of hospital admission (COVER-H), requiring intensive services (COVER-I), or fatality (COVER-F) in the 30-days following COVID-19 diagnosis. MethodsWe analyzed a federated network of electronic medical records and administrative claims data from 14 data sources and 6 countries. We developed and validated 3 scores using 6,869,127 patients with a general practice, emergency room, or outpatient visit with diagnosed influenza or flu-like symptoms any time prior to 2020. The scores were validated on patients with confirmed or suspected COVID-19 diagnosis across five databases from South Korea, Spain and the United States. Outcomes included i) hospitalization with pneumonia, ii) hospitalization with pneumonia requiring intensive services or death iii) death in the 30 days after index date. ResultsOverall, 44,507 COVID-19 patients were included for model validation. We identified 7 predictors (history of cancer, chronic obstructive pulmonary disease, diabetes, heart disease, hypertension, hyperlipidemia, kidney disease) which combined with age and sex discriminated which patients would experience any of our three outcomes. The models achieved high performance in influenza. When transported to COVID-19 cohorts, the AUC ranges were, COVER-H: 0.69-0.81, COVER-I: 0.73-0.91, and COVER-F: 0.72-0.90. Calibration was overall acceptable. ConclusionsA 9-predictor model performs well for COVID-19 patients for predicting hospitalization, intensive services and fatality. The models could aid in providing reassurance for low risk patients and shield high risk patients from COVID-19 during de-confinement to reduce the virus impact on morbidity and mortality.
RESUMO
ObjectiveTo review and critically appraise published and preprint reports of models that aim to predict either (i) presence of existing COVID-19 infection, (ii) future complications in individuals already diagnosed with COVID-19, or (iii) models to identify individuals at high risk for COVID-19 in the general population. DesignRapid systematic review and critical appraisal of prediction models for diagnosis or prognosis of COVID-19 infection. Data sourcesPubMed, EMBASE via Ovid, Arxiv, medRxiv and bioRxiv until 24th March 2020. Study selectionStudies that developed or validated a multivariable COVID-19 related prediction model. Two authors independently screened titles, abstracts and full text. Data extractionData from included studies were extracted independently by at least two authors based on the CHARMS checklist, and risk of bias was assessed using PROBAST. Data were extracted on various domains including the participants, predictors, outcomes, data analysis, and prediction model performance. Results2696 titles were screened. Of these, 27 studies describing 31 prediction models were included for data extraction and critical appraisal. We identified three models to predict hospital admission from pneumonia and other events (as a proxy for covid-19 pneumonia) in the general population; 18 diagnostic models to detect COVID-19 infection in symptomatic individuals (13 of which were machine learning utilising computed tomography (CT) results); and ten prognostic models for predicting mortality risk, progression to a severe state, or length of hospital stay. Only one of these studies used data on COVID-19 cases outside of China. Most reported predictors of presence of COVID-19 in suspected patients included age, body temperature, and signs and symptoms. Most reported predictors of severe prognosis in infected patients included age, sex, features derived from CT, C-reactive protein, lactic dehydrogenase, and lymphocyte count. Estimated C-index estimates for the prediction models ranged from 0.73 to 0.81 in those for the general population (reported for all 3 general population models), from 0.81 to > 0.99 in those for diagnosis (reported for 13 of the 18 diagnostic models), and from 0.85 to 0.98 in those for prognosis (reported for 6 of the 10 prognostic models). All studies were rated at high risk of bias, mostly because of non-representative selection of control patients, exclusion of patients who had not experienced the event of interest by the end of the study, and poor statistical analysis, including high risk of model overfitting. Reporting quality varied substantially between studies. A description of the study population and intended use of the models was absent in almost all reports, and calibration of predictions was rarely assessed. ConclusionCOVID-19 related prediction models are quickly entering the academic literature, to support medical decision making at a time where this is urgently needed. Our review indicates proposed models are poorly reported and at high risk of bias. Thus, their reported performance is likely optimistic and using them to support medical decision making is not advised. We call for immediate sharing of the individual participant data from COVID-19 studies to support collaborative efforts in building more rigorously developed prediction models and validating (evaluating) existing models. The aforementioned predictors identified in multiple included studies could be considered as candidate predictors for new models. We also stress the need to follow methodological guidance when developing and validating prediction models, as unreliable predictions may cause more harm than benefit when used to guide clinical decisions. Finally, studies should adhere to the TRIPOD statement to facilitate validating, appraising, advocating and clinically using the reported models. Systematic review registration protocolosf.io/ehc47/, registration: osf.io/wy245 Summary boxesO_ST_ABSWhat is already known on this topicC_ST_ABS- The sharp recent increase in COVID-19 infections has put a strain on healthcare systems worldwide, necessitating efficient early detection, diagnosis of patients suspected of the infection and prognostication of COVID-19 confirmed cases. - Viral nucleic acid testing and chest CT are standard methods for diagnosing COVID-19, but are time-consuming. - Earlier reports suggest that the elderly, patients with comorbidity (COPD, cardiovascular disease, hypertension), and patients presenting with dyspnoea are vulnerable to more severe morbidity and mortality after COVID-19 infection. What this study adds- We identified three models to predict hospital admission from pneumonia and other events (as a proxy for COVID-19 pneumonia) in the general population. - We identified 18 diagnostic models for COVID-19 detection in symptomatic patients. - 13 of these were machine learning models based on CT images. - We identified ten prognostic models for COVID-19 infected patients, of which six aimed to predict mortality risk in confirmed or suspected COVID-19 patients, two aimed to predict progression to a severe or critical state, and two aimed to predict a hospital stay of more than 10 days from admission. - Included studies were poorly reported compromising their subsequent appraisal, and recommendation for use in daily practice. All studies were appraised at high risk of bias, raising concern that the models may be flawed and perform poorly when applied in practice, such that their predictions may be unreliable.
