Discovering Social Determinants of Health from Case Reports using Natural Language Processing: Algorithmic Development and Validation (preprint)

ABSTRACT

Background: Social determinants of health are non-medical factors that influence health outcomes (SDOH). There is a wealth of SDOH information available via electronic health records, clinical reports, and social media, usually in free texts format, which poses a significant challenge and necessitates the use of natural language processing (NLP) techniques to extract key information. Objective: The objective of this research is to advance the automatic extraction of SDOH from clinical texts. Setting and Data The case reports of COVID-19 patients from the published literature are curated to create a corpus. A portion of the data is annotated by experts to create gold labels, and active learning is used for corpus re-annotation. Methods: A named entity recognition (NER) framework is developed and tested to extract SDOH along with a few prominent clinical entities (diseases, treatments, diagnosis) from the free texts. The proposed model consists of three deep neural networks-A Transformer-based model, a BiLSTM model and a CRF module. Results: The proposed NER implementation achieves an accuracy (F1-score) of 92.98% on our test set and generalizes well on benchmark data. A careful analysis of case examples demonstrates the superiority of the proposed approach in correctly classifying the named entities. Conclusions: NLP can be used to extract key information, such as SDOH from free texts. A more accurate understanding of SDOH is needed to further improve healthcare outcomes.