Discovering Social Determinants of Health from Case Reports using Natural Language Processing: Algorithmic Development and Validation

BackgroundSocial determinants of health are non-medical factors that influence health outcomes (SDOH). There is a wealth of SDOH information available via electronic health records, clinical reports, and social media, usually in free texts format, which poses a significant challenge and necessitates the use of natural language processing (NLP) techniques to extract key information. ObjectiveThe objective of this research is to advance the automatic extraction of SDOH from clinical texts. Setting and DataThe case reports of COVID-19 patients from the published literature are curated to create a corpus. A portion of the data is annotated by experts to create gold labels, and active learning is used for corpus re-annotation. MethodsA named entity recognition (NER) framework is developed and tested to extract SDOH along with a few prominent clinical entities (diseases, treatments, diagnosis) from the free texts. The proposed model consists of three deep neural networks - A Transformer-based model, a BiLSTM model and a CRF module. ResultsThe proposed NER implementation achieves an accuracy (F1-score) of 92.98% on our test set and generalizes well on benchmark data. A careful analysis of case examples demonstrates the superiority of the proposed approach in correctly classifying the named entities. ConclusionsNLP can be used to extract key information, such as SDOH from free texts. A more accurate understanding of SDOH is needed to further improve healthcare outcomes.

COVID-19 International Border Surveillance Cohort Study at Toronto's Pearson Airport

ObjectivesThe primary objective was to estimate the positivity rate of air travelers coming to Toronto, Canada in September and October, 2020, at arrival, day 7 and day 14. Secondary objectives were to estimate degree of risk based on country of origin; to assess knowledge and attitudes towards COVID-19 control measures; and subjective well-being during the quarantine period. DesignProspective cohort of arriving international travelers. SettingToronto Pearson Airport Terminal 1, Toronto, Canada. ParticipantsPassengers arriving on international flights. Inclusion criteria were those aged 18 or older who had a final destination within 100 km of the airport; spoke English or French; and provided consent. Excluded were those taking a connecting flight; who had no internet access; who exhibited symptoms of COVID-19 on arrival; or who were exempted from quarantine. Main outcome measuresPositive for SARS-CoV-2 virus on RT-PCR with self-administered nasal-oral swab, and general well-being using the WHO-5 index. ResultsOf 16,361 passengers enrolled, 248 (1{middle dot}5%, 95% CI 1.3%,1.5%) tested positive. Of these, 167 (67%) were identified on arrival, 67 (27%) on day 7, and 14 (6%) on day 14. The positivity rate increased from 1% in September to 2% in October. Average well-being score declined from 19.8 (out of a maximum of 25) to 15.5 between arrival and day 7 (p<0.001). ConclusionsA single arrival test will pick up two-thirds of individuals who will become positive, with most of the rest detected on the second test at day 7. These results support strategies identified through mathematical models that a reduced quarantine combined with testing can be as effective as a 14 day quarantine. Article SummaryO_ST_ABSStrengths and limitations of this studyC_ST_ABSO_LIDecisions regarding border restrictions have been based on trial and error and mathematical models with limited empirical data to support such decision-making. C_LIO_LIThis study assessed the prevalence of SARS-CoV-2 in a cohort of international travellers at arrival, day 7 and 14 of quarantine. C_LIO_LIIt is limited to one airport and there is the potential from bias due to non-participation and loss to follow-up. C_LIO_LISelf-collected nasal-oral swabs were used which facilitated participation but may have reduced sensitivity. C_LI

Machine Learning Forecast of Growth in COVID-19 Confirmed Infection Cases with Non-Pharmaceutical Interventions and Cultural Dimensions: Algorithm Development and Validation

BackgroundNational governments have implemented non-pharmaceutical interventions to control and mitigate against the COVID-19 pandemic. A deep understanding of these interventions is required. ObjectiveWe investigate the prediction of future daily national Confirmed Infection Growths - the percentage change in total cumulative cases across 14 days - using metrics representative of non-pharmaceutical interventions and cultural dimensions of each country. MethodsWe combine the OxCGRT dataset, Hofstedes cultural dimensions, and COVID-19 daily reported infection case numbers to train and evaluate five non-time series machine learning models in predicting Confirmed Infection Growth. We use three validation methods - in-distribution, out-of-distribution, and country-based cross-validation - for evaluation, each applicable to a different use case of the models. ResultsOur results demonstrate high R2 values between the labels and predictions for the in-distribution, out-of-distribution, and country-based cross-validation methods (0.959, 0.513, and 0.574 respectively) using random forest and AdaBoost regression. While these models may be used to predict the Confirmed Infection Growth, the differing accuracies obtained from the three tasks suggest a strong influence of the use case. ConclusionsThis work provides new considerations in using machine learning techniques with non-pharmaceutical interventions and cultural dimensions data for predicting the national growth of confirmed infections of COVID-19.

Development of a Multivariable Model for COVID-19 Risk Stratification Based on Gradient Boosting Decision Trees

ImportancePopulation stratification of the adult population in Ontario, Canada by their risk of COVID-19 complications can support rapid pandemic response, resource allocation, and decision making. ObjectiveTo develop and validate a multivariable model to predict risk of hospitalization due to COVID-19 severity from routinely collected health records of the entire adult population of Ontario, Canada. Design, Setting, and ParticipantsThis cohort study included 36,323 adult patients (age [≥] 18 years) from the province of Ontario, Canada, who tested positive for SARS-CoV-2 nucleic acid by polymerase chain reaction between February 2 and October 5, 2020, and followed up through November 5, 2020. Patients living in long-term care facilities were excluded from the analysis. Main Outcomes and MeasuresRisk of hospitalization within 30 days of COVID-19 diagnosis was estimated via Gradient Boosting Decision Trees, and risk factor importance was examined via Shapley values. ResultsThe study cohort included 36,323 patients with majority female sex (18,895 [52.02%]) and median (IQR) age of 45 (31-58) years. The cohort had a hospitalization rate of 7.11% (2,583 hospitalizations) with median (IQR) time to hospitalization of 1 (0-5) days, and a mortality rate of 2.49% (906 deaths) with median (IQR) time to death of 12 (6-27) days. In contrast to patients who were not hospitalized, those who were hospitalized had a higher median age (64 years vs 43 years, p-value < 0.001), majority male (56.25% vs 47.35%, p-value<0.001), and had a higher median [IQR] number of comorbidities (3 [2-6] vs 1 [0-3], p-value<0.001). Patients were randomly split into development (n=29,058, 80%) and held-out validation (n=7,265, 20%) cohorts. The final Gradient Boosting model was built using the XGBoost algorithm and achieved high discrimination (development cohort: mean area under the receiver operating characteristic curve across the five folds of 0.852; held-out validation cohort: 0.8475) as well as excellent calibration (R2=0.998, slope=1.01, intercept=-0.01). The patients who scored at the top 10% in the validation cohort captured 47.41% of the actual hospitalizations, whereas those scored at the top 30% captured 80.56%. Patients in the held-out validation cohort (n=7,265) with a score of at least 0.5 (n=2,149, 29.58%) had a 20.29% hospitalization rate (positive predictive value 20.29%) compared with 2.2% hospitalization rate for those with a score less than 0.5 (n=5,116, 70.42%; negative predictive value 97.8%). Aside from age, gender and number of comorbidities, the features that most contribute to model predictions were: history of abnormal blood levels of creatinine, neutrophils and leukocytes, geography and chronic kidney disease. ConclusionsA risk stratification model has been developed and validated using unique, de-identified, and linked routinely collected health administrative data available in Ontario, Canada. The final XGBoost model showed a high discrimination rate, with the potential utility to stratify patients at risk of serious COVID-19 outcomes. This model demonstrates that routinely collected health system data can be successfully leveraged as a proxy for the potential risk of severe COVID-19 complications. Specifically, past laboratory results and demographic factors provide a strong signal for identifying patients who are susceptible to complications. The model can support population risk stratification that informs patients protection most at risk for severe COVID-19 complications.

Patient characteristics, clinical care, resource use, and outcomes associated with hospitalization for COVID-19 in the Toronto area

BackgroundPatient characteristics, clinical care, resource use, and outcomes associated with hospitalization for coronavirus disease (COVID-19) in Canada are not well described. MethodsWe described all adult discharges from inpatient medical services and medical-surgical intensive care units (ICU) between November 1, 2019 and June 30, 2020 at 7 hospitals in Toronto and Mississauga, Ontario. We compared patients hospitalized with COVID-19, influenza and all other conditions using multivariable regression models controlling for patient age, sex, comorbidity, and residence in long-term-care. ResultsThere were 43,462 discharges in the study period, including 1,027 (3.0%) with COVID-19 and 783 (2.3%) with influenza. Patients with COVID-19 had similar age to patients with influenza and other conditions (median age 65 years vs. 68 years and 68 years, respectively, SD<0.1). Patients with COVID-19 were more likely to be male (59.1%) and 11.7% were long-term care residents. Patients younger than 50 years accounted for 21.2% of all admissions for COVID-19 and 24.0% of ICU admissions. Compared to influenza, patients with COVID-19 had significantly greater mortality (unadjusted 19.9% vs 6.1%, aRR: 3.47, 95%CI: 2.57, 4.67), ICU use (unadjusted 26.4% vs 18.0%, aRR 1.52, 95%CI: 1.27, 1.83) and hospital length-of-stay (unadjusted median 8.7 days vs 4.8 days, aRR: 1.40, 95%CI: 1.20, 1.64), and not significantly different 30-day readmission (unadjusted 8.6% vs 8.2%, aRR: 1.01, 95%CI: 0.72, 1.42). InterpretationAdults hospitalized with COVID-19 during the first wave of the pandemic used substantial hospital resources and suffered high mortality. COVID-19 was associated with significantly greater mortality, ICU use, and hospital length-of-stay than influenza.

COVID-19 Demographics, Acute Care Resource Use and Mortality by Age and Sex in Ontario, Canada: Population-based Retrospective Cohort Analysis

BackgroundUnderstanding resource use for COVID-19 is critical. We conducted a population-based cohort study using public health data to describe COVID-19 associated age- and sex-specific acute care use, length of stay (LOS), and mortality. MethodsWe used Ontarios Case and Contact Management (CCM) Plus database of individuals who tested positive for COVID-19 in Ontario from March 1 to September 30, 2020 to determine age- and sex-specific hospitalizations, intensive care unit (ICU) admissions, invasive mechanical ventilation (IMV) use, LOS, and mortality. We stratified analyses by month of infection to study temporal trends and conducted subgroup analyses by long-term care residency. ResultsDuring the observation period, 56,476 COVID-19 cases were reported (72% < 60 years, 52% female). The proportion of cases shifted from older populations (> 60 years) to younger populations (10-39 years) over time. Overall, 10% of individuals were hospitalized, of those 22% were admitted to ICU, and 60% of those used IMV. Mean LOS for individuals in the ward, ICU without IMV, and ICU with IMV was 12.8, 8.5, 20.5 days, respectively. Mortality for individuals receiving care in the ward, ICU without IMV, and ICU with IMV was 24%, 30%, and 45%, respectively. All outcomes varied by age and decreased over time, overall and within age groups. InterpretationThis descriptive study shows acute care use and mortality varying by age, and decreasing between March and September in Ontario. Improvements in clinical practice and changing risk distributions among those infected may contribute to fewer severe outcomes among those infected with COVID-19.