Predicting short- to medium-term care home admission risk in older adults: a systematic review of externally validated models.

INTRODUCTION: Predicting risk of care home admission could identify older adults for early intervention to support independent living but require external validation in a different dataset before clinical use. We systematically reviewed external validations of care home admission risk prediction models in older adults. METHODS: We searched Medline, Embase and Cochrane Library until 14 August 2023 for external validations of prediction models for care home admission risk in adults aged ≥65 years with up to 3 years of follow-up. We extracted and narratively synthesised data on study design, model characteristics, and model discrimination and calibration (accuracy of predictions). We assessed risk of bias and applicability using Prediction model Risk Of Bias Assessment Tool. RESULTS: Five studies reporting validations of nine unique models were included. Model applicability was fair but risk of bias was mostly high due to not reporting model calibration. Morbidities were used as predictors in four models, most commonly neurological or psychiatric diseases. Physical function was also included in four models. For 1-year prediction, three of the six models had acceptable discrimination (area under the receiver operating characteristic curve (AUC)/c statistic 0.70-0.79) and the remaining three had poor discrimination (AUC < 0.70). No model accounted for competing mortality risk. The only study examining model calibration (but ignoring competing mortality) concluded that it was excellent. CONCLUSIONS: The reporting of models was incomplete. Model discrimination was at best acceptable, and calibration was rarely examined (and ignored competing mortality risk when examined). There is a need to derive better models that account for competing mortality risk and report calibration as well as discrimination.

Multimorbidity analysis with low condition counts: a robust Bayesian approach for small but important subgroups.

BACKGROUND: Robustly examining associations between long-term conditions may be important in identifying opportunities for intervention in multimorbidity but is challenging when evidence is limited. We have developed a Bayesian inference framework that is robust to sparse data and used it to quantify morbidity associations in the oldest old, a population with limited available data. METHODS: We conducted a retrospective cross-sectional study of a representative dataset of primary care patients in Scotland as of March 2007. We included 40 long-term conditions and studied their associations in 12,009 individuals aged 90 and older, stratified by sex (3039 men, 8970 women). We analysed associations obtained with Relative Risk (RR), a standard measure in the literature, and compared them with our proposed measure, Associations Beyond Chance (ABC). To enable a broad exploration of interactions between long-term conditions, we built networks of association and assessed differences in their analysis when associations are estimated by RR or ABC. FINDINGS: Our Bayesian framework was appropriately more cautious in attributing association when evidence is lacking, particularly in uncommon conditions. This caution in reporting association was also present in reporting differences in associations between sex and affected the aggregated measures of multimorbidity and network representations. INTERPRETATION: Incorporating uncertainty into multimorbidity research is crucial to avoid misleading findings when evidence is limited, a problem that particularly affects small but important subgroups. Our proposed framework improves the reliability of estimations of associations and, more in general, of research into disease mechanisms and multimorbidity. FUNDING: National Institute for Health and Care Research.

Performance of models for predicting 1-year to 3-year mortality in older adults: a systematic review of externally validated models.

Mortality prediction models support identifying older adults with short life expectancy for whom clinical care might need modifications. We systematically reviewed external validations of mortality prediction models in older adults (ie, aged 65 years and older) with up to 3 years of follow-up. In March, 2023, we conducted a literature search resulting in 36 studies reporting 74 validations of 64 unique models. Model applicability was fair but validation risk of bias was mostly high, with 50 (68%) of 74 validations not reporting calibration. Morbidities (most commonly cardiovascular diseases) were used as predictors by 45 (70%) of 64 of models. For 1-year prediction, 31 (67%) of 46 models had acceptable discrimination, but only one had excellent performance. Models with more than 20 predictors were more likely to have acceptable discrimination (risk ratio [RR] vs <10 predictors 1·68, 95% CI 1·06-2·66), as were models including sex (RR 1·75, 95% CI 1·12-2·73) or predicting risk during comprehensive geriatric assessment (RR 1·86, 95% CI 1·12-3·07). Development and validation of better-performing mortality prediction models in older people are needed.

Analysis of the impact of COVID-19 on Scotland's care-homes from March 2020 to October 2021: national linked data cohort analysis.

BACKGROUND: The impact of the COVID-19 pandemic on long-term care residents remains of wide interest, but most analyses focus on the initial wave of infections. OBJECTIVE: To examine change over time in: (i) The size, duration, classification and pattern of care-home outbreaks of COVID-19 and associated mortality and (ii) characteristics associated with an outbreak. DESIGN: Retrospective observational cohort study using routinely-collected data. SETTING: All adult care-homes in Scotland (1,092 homes, 41,299 places). METHODS: Analysis was undertaken at care-home level, over three periods. Period (P)1 01/03/2020-31/08/2020; P2 01/09/2020-31/05/2021 and P3 01/06/2021-31/10/2021. Outcomes were the presence and characteristics of outbreaks and mortality within the care-home. Cluster analysis was used to compare the pattern of outbreaks. Logistic regression examined care-home characteristics associated with outbreaks. RESULTS: In total 296 (27.1%) care-homes had one outbreak, 220 (20.1%) had two, 91 (8.3%) had three, and 68 (6.2%) had four or more. There were 1,313 outbreaks involving residents: 431 outbreaks in P1, 559 in P2 and 323 in P3. The COVID-19 mortality rate per 1,000 beds fell from 45.8 in P1, to 29.3 in P2, and 3.5 in P3. Larger care-homes were much more likely to have an outbreak, but associations between size and outbreaks were weaker in later periods. CONCLUSIONS: COVID-19 mitigation measures appear to have been beneficial, although the impact on residents remained severe until early 2021. Care-home residents, staff, relatives and providers are critical groups for consideration and involvement in future pandemic planning.

External validation of the QLifetime cardiovascular risk prediction tool: population cohort study.

BACKGROUND: Prediction of lifetime cardiovascular disease (CVD) risk is recommended in many clinical guidelines, but lifetime risk models are rarely externally validated. The aim of this study was to externally validate the QRiskLifetime incident CVD risk prediction tool. METHODS: Independent external validation of QRiskLifetime using Clinical Practice Research Datalink data, examining discrimination and calibration in the whole population and stratified by age, and reclassification compared to QRISK3. Since lifetime CVD risk is unobservable, performance was evaluated at 10-years' follow-up, and lifetime performance inferred in terms of performance for in the different age-groups from which lifetime predictions are derived. RESULTS: One million, two hundreds sixty thousand and three hundreds twenty nine women and 1,223,265 men were included in the analysis. Discrimination was excellent in the whole population (Harrell's-C = 0.844 in women, 0.808 in men), but moderate to poor stratified by age-group (Harrell's C in people aged 30-44 0.714 for both men and women, in people aged 75-84 0.578 in women and 0.556 in men). Ten-year CVD risk was under-predicted in the whole population, and in all age-groups except women aged 45-64, with worse under-prediction in older age-groups. Compared to those at highest QRISK3 estimated 10-year risk, those with highest lifetime risk were younger (mean age: women 50.5 vs. 71.3 years; men 46.3 vs. 63.8 years) and had lower systolic blood pressure and prevalence of treated hypertension, but had more family history of premature CVD, and were more commonly minority ethnic. Over 10-years, the estimated number needed to treat (NNT) with a statin to prevent one CVD event in people with QRISK3 ≥ 10% was 34 in women and 37 in men, compared to 99 and 100 for those at highest lifetime risk. CONCLUSIONS: QRiskLifetime underpredicts 10-year CVD risk in nearly all age-groups, so is likely to also underpredict lifetime risk. Treatment based on lifetime risk has considerably lower medium-term benefit than treatment based on 10-year risk.

Development of prediction models for one-year brain tumour survival using machine learning: a comparison of accuracy and interpretability.

BACKGROUND AND OBJECTIVE: Prediction of survival in patients diagnosed with a brain tumour is challenging because of heterogeneous tumour behaviours and treatment response. Advances in machine learning have led to the development of clinical prognostic models, but due to the lack of model interpretability, integration into clinical practice is almost non-existent. In this retrospective study, we compare five classification models with varying degrees of interpretability for the prediction of brain tumour survival greater than one year following diagnosis. METHODS: 1028 patients aged ≥16 years with a brain tumour diagnosis between April 2012 and April 2020 were included in our study. Three intrinsically interpretable 'glass box' classifiers (Bayesian Rule Lists [BRL], Explainable Boosting Machine [EBM], and Logistic Regression [LR]), and two 'black box' classifiers (Random Forest [RF] and Support Vector Machine [SVM]) were trained on electronic patients records for the prediction of one-year survival. All models were evaluated using balanced accuracy (BAC), F1-score, sensitivity, specificity, and receiver operating characteristics. Black box model interpretability and misclassified predictions were quantified using SHapley Additive exPlanations (SHAP) values and model feature importance was evaluated by clinical experts. RESULTS: The RF model achieved the highest BAC of 78.9%, closely followed by SVM (77.7%), LR (77.5%) and EBM (77.1%). Across all models, age, diagnosis (tumour type), functional features, and first treatment were top contributors to the prediction of one year survival. We used EBM and SHAP to explain model misclassifications and investigated the role of feature interactions in prognosis. CONCLUSION: Interpretable models are a natural choice for the domain of predictive medicine. Intrinsically interpretable models, such as EBMs, may provide an advantage over traditional clinical assessment of brain tumour prognosis by weighting potential risk factors and their interactions that may be unknown to clinicians. An agreement between model predictions and clinical knowledge is essential for establishing trust in the models decision making process, as well as trust that the model will make accurate predictions when applied to new data.

A literature review on the analysis of symptom-based clinical pathways: Time for a different approach?

Breathlessness is a common clinical presentation, accounting for a quarter of all emergency hospital attendances. As a complex undifferentiated symptom, it may be caused by dysfunction in multiple body systems. Electronic health records are rich with activity data to inform clinical pathways from undifferentiated breathlessness to specific disease diagnoses. These data may be amenable to process mining, a computational technique that uses event logs to identify common patterns of activity. We reviewed use of process mining and related techniques to understand clinical pathways for patients with breathlessness. We searched the literature from two perspectives: studies of clinical pathways for breathlessness as a symptom, and those focussed on pathways for respiratory and cardiovascular diseases that are commonly associated with breathlessness. The primary search included PubMed, IEEE Xplore and ACM Digital Library. We included studies if breathlessness or a relevant disease was present in combination with a process mining concept. We excluded non-English publications, and those focussed on biomarkers, investigations, prognosis, or disease progression rather than symptoms. Eligible articles were screened before full-text review. Of 1,400 identified studies, 1,332 studies were excluded through screening and removal of duplicates. Following full-text review of 68 studies, 13 were included in qualitative synthesis, of which two (15%) were symptom and 11 (85%) disease focused. While studies reported highly varied methodologies, only one included true process mining, using multiple techniques to explore Emergency Department clinical pathways. Most included studies trained and internally validated within single-centre datasets, limiting evidence for wider generalisability. Our review has highlighted a lack of clinical pathway analyses for breathlessness as a symptom, compared to disease-focussed approaches. Process mining has potential application in this area, but has been under-utilised in part due to data interoperability challenges. There is an unmet research need for larger, prospective multicentre studies of patient pathways following presentation with undifferentiated breathlessness.

Care-home outbreaks of COVID-19 in Scotland March to May 2020: National linked data cohort analysis.

BACKGROUND: understanding care-home outbreaks of COVID-19 is a key public health priority in the ongoing pandemic to help protect vulnerable residents. OBJECTIVE: to describe all outbreaks of COVID-19 infection in Scottish care-homes for older people between 01/03/2020 and 31/03/2020, with follow-up to 30/06/2020. DESIGN AND SETTING: National linked data cohort analysis of Scottish care-homes for older people. METHODS: data linkage was used to identify outbreaks of COVID-19 in care-homes. Care-home characteristics associated with the presence of an outbreak were examined using logistic regression. Size of outbreaks was modelled using negative binomial regression. RESULTS: 334 (41%) Scottish care-homes for older people experienced an outbreak, with heterogeneity in outbreak size (1-63 cases; median = 6) and duration (1-94 days, median = 31.5 days). Four distinct patterns of outbreak were identified: 'typical' (38% of outbreaks, mean 11.2 cases and 48 days duration), severe (11%, mean 29.7 cases and 60 days), contained (37%, mean 3.5 cases and 13 days) and late-onset (14%, mean 5.4 cases and 17 days). Risk of a COVID-19 outbreak increased with increasing care-home size (for ≥90 beds vs <20, adjusted OR = 55.4, 95% CI 15.0-251.7) and rising community prevalence (OR = 1.2 [1.0-1.4] per 100 cases/100,000 population increase). No routinely available care-home characteristic was associated with outbreak size. CONCLUSIONS: reducing community prevalence of COVID-19 infection is essential to protect those living in care-homes. More systematic national data collection to understand care-home residents and the homes in which they live is a priority in ensuring we can respond more effectively in future.

Using Artificial Intelligence for Predicting Survival of Individual Grafts in Liver Transplantation: A Systematic Review.

The demand for liver transplantation far outstrips the supply of deceased donor organs, and so, listing and allocation decisions aim to maximize utility. Most existing methods for predicting transplant outcomes use basic methods, such as regression modeling, but newer artificial intelligence (AI) techniques have the potential to improve predictive accuracy. The aim was to perform a systematic review of studies predicting graft outcomes following deceased donor liver transplantation using AI techniques and to compare these findings to linear regression and standard predictive modeling: donor risk index (DRI), Model for End-Stage Liver Disease (MELD), and Survival Outcome Following Liver Transplantation (SOFT). After reviewing available article databases, a total of 52 articles were reviewed for inclusion. Of these articles, 9 met the inclusion criteria, which reported outcomes from 18,771 liver transplants. Artificial neural networks (ANNs) were the most commonly used methodology, being reported in 7 studies. Only 2 studies directly compared machine learning (ML) techniques to liver scoring modalities (i.e., DRI, SOFT, and balance of risk [BAR]). Both studies showed better prediction of individual organ survival with the optimal ANN model, reporting an area under the receiver operating characteristic curve (AUROC) 0.82 compared with BAR (0.62) and SOFT (0.57), and the other ANN model gave an AUC ROC of 0.84 compared with a DRI (0.68) and SOFT (0.64). AI techniques can provide high accuracy in predicting graft survival based on donors and recipient variables. When compared with the standard techniques, AI methods are dynamic and are able to be trained and validated within every population. However, the high accuracy of AI may come at a cost of losing explainability (to patients and clinicians) on how the technology works.

A Workflow-Driven Formal Methods Approach to the Generation of Structured Checklists for Intrahospital Patient Transfers.

Intrahospital transfers are a common but hazardous aspect of hospital care, with a large number of incidents posing a threat to patient safety. A growing body of work advocates the use of checklists for minimizing intrahospital transfer risk, but the majority of existing checklists are not guaranteed to be error-free and are difficult to adapt to different clinical settings or changing hospital policies. This paper details an approach that addresses these challenges through the employment of workflow technologies and formal methods for generating structured checklists. A three-phased methodology is proposed, where intrahospital transfer processes are first conceptualized, then rigorously composed into workflows that are mechanically verified, and finally, translated into a set of checklists that support hospital staff while maintaining the dependencies between different transfer tasks. A case study is presented, highlighting the feasibility of this approach, and the correctness and maintainability benefits brought by the logical underpinning of this methodology. A checklist evaluation is discussed, with promising results regarding their usefulness.