Your browser doesn't support javascript.
VALIDATION OF A MACHINE LEARNING MODEL IN A PROSPECTIVE COHORT: THE RISQ STUDY
Gut ; 71:A92, 2022.
Article in English | EMBASE | ID: covidwho-2005363
ABSTRACT
Introduction Previously our group had identified 20 features which were associated with the development of upper gastrointestinal (UGI) cancers using a machine learning approach.[1] We sought to refine this model and to validate this in an independent dataset to assess its generalisability in an interim analysis. Methods We selected patients who were recruited for the multicentre Saliva to Predict rIsk of disease using Transcriptomics and epigenetics (SPIT) study to develop our model. Patients were recruited from 2-week wait suspected UGI pathways and additionally enriched with patients with confirmed oesophageal adenocarcinoma admitted as inpatients. We used regularised logistic regression (glmnet) from the caret package in R software to create the model. 60% of the data with 10-fold cross validation was used for training, with the remaining 40% for testing. For validation, we used data from the predicting RIsk of disease uSing detailed Questionnaires (RISQ) study, an ongoing prospective multicentre study using the questionnaire based on the our previous work.1 We evaluated the model using area under the receiver operating characteristic curve (AUC). Results We included 93 cancer and 715 non-cancer patients for training and testing and 21 cancer and 203 non-cancer patients for validation. We further reduced the model to 18 features without significant detriment to model performance. In the training and testing data AUC was 0.86 (95%CI 0.81- 0.91) and 0.75 (95%CI 0.67-0.83) respectively. We set a threshold of 0.03 as a cut off based on a cost function where false negatives had a 50-time greater impact than false positive cases (figure 1). For the validation cohort we achieved an AUC of 0.95 (95%CI 0.90-1.00). This equated to a sensitivity 0.952 and a specificity of 0.897 for detecting cancer. Conclusions Initial results from our model compare favourably with the Edinburgh Dysphagia Scale, which has a sensitivity and specificity of 0.984 and 0.093 respectively.2 It also appears to have a high specificity, potentially helping to reduce unnecessary endoscopies. We aim to further increase the size of the validation cohort to ensure its robustness and generalisability. Our model could be applied to triaging and prioritising endoscopic referral backlogs as a result of COVID- 19.3.
Keywords

Full text: Available Collection: Databases of international organizations Database: EMBASE Type of study: Cohort study / Observational study / Prognostic study Language: English Journal: Gut Year: 2022 Document Type: Article

Similar

MEDLINE

...
LILACS

LIS


Full text: Available Collection: Databases of international organizations Database: EMBASE Type of study: Cohort study / Observational study / Prognostic study Language: English Journal: Gut Year: 2022 Document Type: Article