Answering List-Type Questions in Health Domain with Pretrained Large Language Model: A Case for COVID-19 Symptoms.

List-type questions, which can have a varying number of answers, are more common in the health domain where people seek for health-related information from a passage or passages. An example of this type of question answering task is to find COVID-19 symptoms from a Twitter post. However, due to the lack of annotated instances for supervised learning, automatic identification of COVID-19 symptoms from Twitter posts is challenging. We investigated detection of symptom mentions in Twitter posts using GPT-3, a pre-trained large language model, along with few-shot learning. Our results of 5-shot and 10-shot learning on a corpus of 655 annotated tweets demonstrate that few-shot learning with pre-trained large language model is a promising approach to answering list-type questions with a minimal amount of effort of annotation.

Large Language Model as Unsupervised Health Information Retriever.

Retrieving health information is a task of search for health-related information from a variety of sources. Gathering self-reported health information may help enrich the knowledge body of the disease and its symptoms. We investigated retrieving symptom mentions in COVID-19-related Twitter posts with a pretrained large language model (GPT-3) without providing any examples (zero-shot learning). We introduced a new performance measure of total match (TM) to include exact, partial and semantic matches. Our results show that the zero-shot approach is a powerful method without the need to annotate any data, and it can assist in generating instances for few-shot learning which may achieve better performance.