AlpaPICO: Extraction of PICO frames from clinical trial documents using LLMs.

In recent years, there has been a surge in the publication of clinical trial reports, making it challenging to conduct systematic reviews. Automatically extracting Population, Intervention, Comparator, and Outcome (PICO) from clinical trial studies can alleviate the traditionally time-consuming process of manually scrutinizing systematic reviews. Existing approaches of PICO frame extraction involves supervised approach that relies on the existence of manually annotated data points in the form of BIO label tagging. Recent approaches, such as In-Context Learning (ICL), which has been shown to be effective for a number of downstream NLP tasks, require the use of labeled examples. In this work, we adopt ICL strategy by employing the pretrained knowledge of Large Language Models (LLMs), gathered during the pretraining phase of an LLM, to automatically extract the PICO-related terminologies from clinical trial documents in unsupervised set up to bypass the availability of large number of annotated data instances. Additionally, to showcase the highest effectiveness of LLM in oracle scenario where large number of annotated samples are available, we adopt the instruction tuning strategy by employing Low Rank Adaptation (LORA) to conduct the training of gigantic model in low resource environment for the PICO frame extraction task. More specifically, both of the proposed frameworks utilize AlpaCare as base LLM which employs both few-shot in-context learning and instruction tuning techniques to extract PICO-related terms from the clinical trial reports. We applied these approaches to the widely used coarse-grained datasets such as EBM-NLP, EBM-COMET and fine-grained datasets such as EBM-NLPrev and EBM-NLPh. Our empirical results show that our proposed ICL-based framework produces comparable results on all the version of EBM-NLP datasets and the proposed instruction tuned version of our framework produces state-of-the-art results on all the different EBM-NLP datasets. Our project is available at https://github.com/shrimonmuke0202/AlpaPICO.git.

Outcome Prediction from Behaviour Change Intervention Evaluations using a Combination of Node and Word Embedding.

Findings from randomized controlled trials (RCTs) of behaviour change interventions encode much of our knowledge on intervention efficacy under defined conditions. Predicting outcomes of novel interventions in novel conditions can be challenging, as can predicting differences in outcomes between different interventions or different conditions. To predict outcomes from RCTs, we propose a generic framework of combining the information from two sources - i) the instances (comprised of surrounding text and their numeric values) of relevant attributes, namely the intervention, setting and population characteristics of a study, and ii) abstract representation of the categories of these attributes themselves. We demonstrate that this way of encoding both the information about an attribute and its value when used as an embedding layer within a standard deep sequence modeling setup improves the outcome prediction effectiveness.

Knowledge Extraction and Prediction from Behavior Science Randomized Controlled Trials: A Case Study in Smoking Cessation.

Due to the fast pace at which randomized controlled trials are published in the health domain, researchers, consultants and policymakers would benefit from more automatic ways to process them by both extracting relevant information and automating the meta-analysis processes. In this paper, we present a novel methodology based on natural language processing and reasoning models to 1) extract relevant information from RCTs and 2) predict potential outcome values on novel scenarios, given the extracted knowledge, in the domain of behavior change for smoking cessation.

Information Extraction of Behavior Change Intervention Descriptions.

We describe an information extraction (IE) approach for knowledge base population of behavior change scientific intervention findings. In this paper, we focus on building a system able to characterize the specific intervention techniques that are undertaken within behavior change intervention studies. We have investigated three different configurations of a general information retrieval based framework for information extraction: a) an unsupervised approach that hinges on specification of a query for each attribute to be extracted and a few parameters for rule-based post-processing; b) a semi-supervised approach, which uses a part of the ground-truth annotations as a training set to automatically learn optimal representation of the queries; and c) a supervised approach that replaces the rule-based post processing by a text classifier. To train and evaluate our system, we make use of a ground-truth data set annotated by behavior science experts. This dataset consists of a total of 226 research papers on smoking cessation.

Unsupervised Information Extraction from Behaviour Change Literature.

This paper describes our approach to construct a scalable system for unsupervised information extraction from the behaviour change intervention literature. Due to the many different types of attribute to be extracted, we adopt a passage retrieval based framework that provides the most likely value for an attribute. Our proposed method is capable of addressing variable length passage sizes and different validation criteria for the extracted values corresponding to each attribute to be found. We evaluate our approach by constructing a manually annotated ground-truth from a set of 50 research papers with reported studies on smoking cessation.