Prognostic Factors and Models to Predict Mortality Outcomes in Patients with Peripheral Arterial Disease: A Systematic Review.

OBJECTIVE: Predicting adverse outcomes in patients with peripheral arterial disease (PAD) is a complex task owing to the heterogeneity in patient and disease characteristics. This systematic review aimed to identify prognostic factors and prognostic models to predict mortality outcomes in patients with PAD Fontaine stage I - III or Rutherford category 0 - 4. DATA SOURCES: PubMed, Embase, and Cochrane Database of Systematic Reviews were searched to identify studies examining individual prognostic factors or studies aiming to develop or validate a prognostic model for mortality outcomes in patients with PAD. REVIEW METHODS: Information on study design, patient population, prognostic factors, and prognostic model characteristics was extracted, and risk of bias was evaluated. RESULTS: Sixty nine studies investigated prognostic factors for mortality outcomes in PAD. Over 80 single prognostic factors were identified, with age as a predictor of mortality in most of the studies. Other common factors included sex, diabetes, and smoking status. Six studies had low risk of bias in all domains, and the remainder had an unclear or high risk of bias in at least one domain. Eight studies developed or validated a prognostic model. All models included age in their primary model, but not sex. All studies had similar discrimination levels of > 70%. Five of the studies on prognostic models had an overall high risk of bias, whereas two studies had an overall unclear risk of bias. CONCLUSION: This systematic review shows that a large number of prognostic studies have been published, with heterogeneity in patient population, outcomes, and risk of bias. Factors such as sex, age, diabetes, hypertension, and smoking are significant in predicting mortality risk among patients with PAD Fontaine stage I - III or Rutherford category 0 - 4.

SPIN-PM: a consensus framework to evaluate the presence of spin in studies on prediction models.

OBJECTIVES: To develop a framework to identify and evaluate spin practices and its facilitators in studies on clinical prediction model regardless of the modeling technique. STUDY DESIGN AND SETTING: We followed a three-phase consensus process: (1) premeeting literature review to generate items to be included; (2) a series of structured meetings to provide comments discussed and exchanged viewpoints on items to be included with a panel of experienced researchers; and (3) postmeeting review on final list of items and examples to be included. Through this iterative consensus process, a framework was derived after all panel's researchers agreed. RESULTS: This consensus process involved a panel of eight researchers and resulted in SPIN-Prediction Models which consists of two categories of spin (misleading interpretation and misleading transportability), and within these categories, two forms of spin (spin practices and facilitators of spin). We provide criteria and examples. CONCLUSION: We proposed this guidance aiming to facilitate not only the accurate reporting but also an accurate interpretation and extrapolation of clinical prediction models which will likely improve the reporting quality of subsequent research, as well as reduce research waste.

Both clinical trial register and electronic bibliographic database searches were needed to identify randomized clinical trials for systematic reviews: an evaluation study.

OBJECTIVES: To determine whether clinical trial register (CTR) searches can accurately identify a greater number of completed randomized clinical trials (RCTs) than electronic bibliographic database (EBD) searches for systematic reviews of interventions, and to quantify the number of eligible ongoing trials. STUDY DESIGN AND SETTING: We performed an evaluation study and based our search for RCTs on the eligibility criteria of a systematic review that focused on the underrepresentation of people with chronic kidney disease in cardiovascular RCTs. We conducted a combined search of ClinicalTrials.gov and the WHO International Clinical Trials Registry Platform through the Cochrane Central Register of Controlled Trials to identify eligible RCTs registered up to June 1, 2023. We searched Cochrane Central Register of Controlled Trials, EMBASE, and MEDLINE for publications of eligible RCTs published up to June 5, 2023. Finally, we compared the search results to determine the extent to which the two sources identified the same RCTs. RESULTS: We included 92 completed RCTs. Of these, 81 had results available. Sixty-six completed RCTs with available results were identified by both sources (81% agreement [95% CI: 71-88]). We identified seven completed RCTs with results exclusively by CTR search (9% [95% CI: 4-17]) and eight exclusively by EBD search (10% [95% CI: 5-18]). Eleven RCTs were completed but lacked results (four identified by both sources (36% [95% CI: 15-65]), one exclusively by EBD search (9% [95% CI: 1-38]), and six exclusively by CTR search (55% [95% CI: 28-79])). Also, we identified 42 eligible ongoing RCTs: 16 by both sources (38% [95% CI: 25-53]) and 26 exclusively by CTR search (62% [95% CI: 47-75]). Lastly, we identified four RCTs of unknown status by both sources. CONCLUSION: CTR searches identify a greater number of completed RCTs than EBD searches. Both searches missed some included RCTs. Based on our case study, researchers (eg, information specialists, systematic reviewers) aiming to identify all available RCTs should continue to search both sources. Once the barriers to performing CTR searches alone are targeted, CTR searches may be a suitable alternative.

Risk of bias assessments in individual participant data meta-analyses of test accuracy and prediction models: a review shows improvements are needed.

OBJECTIVES: Risk of bias assessments are important in meta-analyses of both aggregate and individual participant data (IPD). There is limited evidence on whether and how risk of bias of included studies or datasets in IPD meta-analyses (IPDMAs) is assessed. We review how risk of bias is currently assessed, reported, and incorporated in IPDMAs of test accuracy and clinical prediction model studies and provide recommendations for improvement. STUDY DESIGN AND SETTING: We searched PubMed (January 2018-May 2020) to identify IPDMAs of test accuracy and prediction models, then elicited whether each IPDMA assessed risk of bias of included studies and, if so, how assessments were reported and subsequently incorporated into the IPDMAs. RESULTS: Forty-nine IPDMAs were included. Nineteen of 27 (70%) test accuracy IPDMAs assessed risk of bias, compared to 5 of 22 (23%) prediction model IPDMAs. Seventeen of 19 (89%) test accuracy IPDMAs used Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2), but no tool was used consistently among prediction model IPDMAs. Of IPDMAs assessing risk of bias, 7 (37%) test accuracy IPDMAs and 1 (20%) prediction model IPDMA provided details on the information sources (e.g., the original manuscript, IPD, primary investigators) used to inform judgments, and 4 (21%) test accuracy IPDMAs and 1 (20%) prediction model IPDMA provided information or whether assessments were done before or after obtaining the IPD of the included studies or datasets. Of all included IPDMAs, only seven test accuracy IPDMAs (26%) and one prediction model IPDMA (5%) incorporated risk of bias assessments into their meta-analyses. For future IPDMA projects, we provide guidance on how to adapt tools such as Prediction model Risk Of Bias ASsessment Tool (for prediction models) and QUADAS-2 (for test accuracy) to assess risk of bias of included primary studies and their IPD. CONCLUSION: Risk of bias assessments and their reporting need to be improved in IPDMAs of test accuracy and, especially, prediction model studies. Using recommended tools, both before and after IPD are obtained, will address this.

Increased endorsement of TRIPOD and other reporting guidelines by high impact factor journals: survey of instructions to authors.

OBJECTIVES: To assess the endorsement of reporting guidelines by high impact factor journals over the period 2017-2022, with a specific focus on the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement. STUDY DESIGN AND SETTING: We searched the online 'instructions to authors' of high impact factor medical journals in February 2017 and in January 2022 for any reference to reporting guidelines and TRIPOD in particular. RESULTS: In 2017, 205 out of 337 (61%) journals mentioned any reporting guideline in their instructions to authors and in 2022 this increased to 245 (73%) journals. A reference to TRIPOD was provided by 27 (8%) journals in 2017 and 67 (20%) in 2022. Of those journals mentioning TRIPOD in 2022, 22% provided a link to the TRIPOD website and 60% linked to TRIPOD information on the Enhancing the QUAlity and Transparency Of health Research (EQUATOR) Network website. Twenty-five percent of the journals required adherence to TRIPOD. CONCLUSION: About three-quarters of high-impact medical journals endorse the use of reporting guidelines and 20% endorse TRIPOD. Transparent reporting is important in enhancing the usefulness of health research and endorsement by journals plays a critical role in this.

Effectiveness of FeNO-guided treatment in adult asthma patients: A systematic review and meta-analysis.

OBJECTIVE: Asthma control is generally monitored by assessing symptoms and lung function. However, optimal treatment is also dependent on the type and extent of airway inflammation. Fraction of exhaled Nitric Oxide (FeNO) is a noninvasive biomarker of type 2 airway inflammation, but its effectiveness in guiding asthma treatment remains disputed. We performed a systematic review and meta-analysis to obtain summary estimates of the effectiveness of FeNO-guided asthma treatment. DESIGN: We updated a Cochrane systematic review from 2016. Cochrane Risk of Bias tool was used to assess risk of bias. Inverse-variance random-effects meta-analysis was performed. Certainty of evidence was assessed using GRADE. Subgroup analyses were performed based on asthma severity, asthma control, allergy/atopy, pregnancy and obesity. DATA SOURCES: The Cochrane Airways Group Trials Register was searched on 9 May 2023. ELIGIBILITY CRITERIA: We included randomized controlled trials (RCTs) comparing the effectiveness of a FeNO-guided treatment versus usual (symptom-guided) treatment in adult asthma patients. RESULTS: We included 12 RCTs (2,116 patients), all showing high or unclear risk of bias in at least one domain. Five RCTs reported support from a FeNO manufacturer. FeNO-guided treatment probably reduces the number of patients having ≥1 exacerbation (OR = 0.61; 95%CI 0.44 to 0.83; six RCTs; GRADE moderate certainty) and exacerbation rate (RR = 0.67; 95%CI 0.54 to 0.82; six RCTs; moderate certainty), and may slightly improve Asthma Control Questionnaire score (MD = -0.10; 95%CI -0.18 to -0.02, six RCTs; low certainty), however, this change is unlikely to be clinically important. An effect on severe exacerbations, quality of life, FEV1, treatment dosage and FeNO values could not be demonstrated. There were no indications that effectiveness is different in subgroups of patients, although evidence for subgroup analysis was limited. CONCLUSIONS: FeNO-guided asthma treatment probably results in fewer exacerbations but may not have clinically important effects on other asthma outcomes.

Diagnostic yield and safety of navigation bronchoscopy: A systematic review and meta-analysis.

BACKGROUND: Navigation bronchoscopy has seen rapid development in the past decade in terms of new navigation techniques and multi-modality approaches utilizing different techniques and tools. This systematic review analyses the diagnostic yield and safety of navigation bronchoscopy for the diagnosis of peripheral pulmonary nodules suspected of lung cancer. METHODS: An extensive search was performed in Embase, Medline and Cochrane CENTRAL in May 2022. Eligible studies used cone-beam CT-guided navigation (CBCT), electromagnetic navigation (EMN), robotic navigation (RB) or virtual bronchoscopy (VB) as the primary navigation technique. Primary outcomes were diagnostic yield and adverse events. Quality of studies was assessed using QUADAS-2. Random effects meta-analysis was performed, with subgroup analyses for different navigation techniques, newer versus older techniques, nodule size, publication year, and strictness of diagnostic yield definition. Explorative analyses of subgroups reported by studies was performed for nodule size and bronchus sign. RESULTS: A total of 95 studies (n = 10,381 patients; n = 10,682 nodules) were included. The majority (n = 63; 66.3%) had high risk of bias or applicability concerns in at least one QUADAS-2 domain. Summary diagnostic yield was 70.9% (95%-CI 68.4%-73.2%). Overall pneumothorax rate was 2.5%. Newer navigation techniques using advanced imaging and/or robotics(CBCT, RB, tomosynthesis guided EMN; n = 24 studies) had a statistically significant higher diagnostic yield compared to longer established techniques (EMN, VB; n = 82 studies): 77.5% (95%-CI 74.7%-80.1%) vs 68.8% (95%-CI 65.9%-71.6%) (p < 0.001).Explorative subgroup analyses showed that larger nodule size and bronchus sign presence were associated with a statistically significant higher diagnostic yield. Other subgroup analyses showed no significant differences. CONCLUSION: Navigation bronchoscopy is a safe procedure, with the potential for high diagnostic yield, in particular using newer techniques such as RB, CBCT and tomosynthesis-guided EMN. Studies showed a large amount of heterogeneity, making comparisons difficult. Standardized definitions for outcomes with relevant clinical context will improve future comparability.

Systematic review finds "spin" practices and poor reporting standards in studies on machine learning-based prediction models.

OBJECTIVES: We evaluated the presence and frequency of spin practices and poor reporting standards in studies that developed and/or validated clinical prediction models using supervised machine learning techniques. STUDY DESIGN AND SETTING: We systematically searched PubMed from 01/2018 to 12/2019 to identify diagnostic and prognostic prediction model studies using supervised machine learning. No restrictions were placed on data source, outcome, or clinical specialty. RESULTS: We included 152 studies: 38% reported diagnostic models and 62% prognostic models. When reported, discrimination was described without precision estimates in 53/71 abstracts (74.6% [95% CI 63.4-83.3]) and 53/81 main texts (65.4% [95% CI 54.6-74.9]). Of the 21 abstracts that recommended the model to be used in daily practice, 20 (95.2% [95% CI 77.3-99.8]) lacked any external validation of the developed models. Likewise, 74/133 (55.6% [95% CI 47.2-63.8]) studies made recommendations for clinical use in their main text without any external validation. Reporting guidelines were cited in 13/152 (8.6% [95% CI 5.1-14.1]) studies. CONCLUSION: Spin practices and poor reporting standards are also present in studies on prediction models using machine learning techniques. A tailored framework for the identification of spin will enhance the sound reporting of prediction model studies.

Overinterpretation of findings in machine learning prediction model studies in oncology: a systematic review.

OBJECTIVES: In biomedical research, spin is the overinterpretation of findings, and it is a growing concern. To date, the presence of spin has not been evaluated in prognostic model research in oncology, including studies developing and validating models for individualized risk prediction. STUDY DESIGN AND SETTING: We conducted a systematic review, searching MEDLINE and EMBASE for oncology-related studies that developed and validated a prognostic model using machine learning published between 1st January, 2019, and 5th September, 2019. We used existing spin frameworks and described areas of highly suggestive spin practices. RESULTS: We included 62 publications (including 152 developed models; 37 validated models). Reporting was inconsistent between methods and the results in 27% of studies due to additional analysis and selective reporting. Thirty-two studies (out of 36 applicable studies) reported comparisons between developed models in their discussion and predominantly used discrimination measures to support their claims (78%). Thirty-five studies (56%) used an overly strong or leading word in their title, abstract, results, discussion, or conclusion. CONCLUSION: The potential for spin needs to be considered when reading, interpreting, and using studies that developed and validated prognostic models in oncology. Researchers should carefully report their prognostic model research using words that reflect their actual results and strength of evidence.

One Heartbeat Away from a Prediction Model for Cardiovascular Diseases in Patients with Chronic Kidney Disease: A Systematic Review.

INTRODUCTION: Patients with chronic kidney disease (CKD) have a high risk of cardiovascular disease (CVD). Prediction models, combining clinical and laboratory characteristics, are commonly used to estimate an individual's CVD risk. However, these models are not specifically developed for patients with CKD and may therefore be less accurate. In this review, we aim to give an overview of CVD prognostic studies available, and their methodological quality, specifically for patients with CKD. METHODS: MEDLINE was searched for papers reporting CVD prognostic studies in patients with CKD published between 2012 and 2021. Characteristics regarding patients, study design, outcome measurement, and prediction models were compared between included studies. The risk of bias of studies reporting on prognostic factors or the development/validation of a prediction model was assessed with, respectively, the QUIPS and PROBAST tool. RESULTS: In total, 134 studies were included, of which 123 studies tested the incremental value of one or more predictors to existing models or common risk factors, while only 11 studies reported on the development or validation of a prediction model. Substantial heterogeneity in cohort and study characteristics, such as sample size, event rate, and definition of outcome measurements, was observed across studies. The most common predictors were age (87%), sex (75%), diabetes (70%), and estimated glomerular filtration rate (69%). Most of the studies on prognostic factors have methodological shortcomings, mostly due to a lack of reporting on clinical and methodological information. Of the 11 studies on prediction models, six developed and internally validated a model and four externally validated existing or developed models. Only one study on prognostic models showed a low risk of bias and high applicability. CONCLUSION: A large quantity of prognostic studies has been published, yet their usefulness remains unclear due to incomplete presentation, and lack of external validation of prognostic models. Our review can be used to select the most appropriate prognostic model depending on the patient population, outcome, and risk of bias. Future collaborative efforts should aim at improving existing models by externally validating them, evaluating the addition of new predictors, and assessment of the clinical impact. REGISTRATION: We have registered the protocol of our systematic review on PROSPERO (CRD42021228043).

Systematic review identifies the design and methodological conduct of studies on machine learning-based prediction models.

BACKGROUND AND OBJECTIVES: We sought to summarize the study design, modelling strategies, and performance measures reported in studies on clinical prediction models developed using machine learning techniques. METHODS: We search PubMed for articles published between 01/01/2018 and 31/12/2019, describing the development or the development with external validation of a multivariable prediction model using any supervised machine learning technique. No restrictions were made based on study design, data source, or predicted patient-related health outcomes. RESULTS: We included 152 studies, 58 (38.2% [95% CI 30.8-46.1]) were diagnostic and 94 (61.8% [95% CI 53.9-69.2]) prognostic studies. Most studies reported only the development of prediction models (n = 133, 87.5% [95% CI 81.3-91.8]), focused on binary outcomes (n = 131, 86.2% [95% CI 79.8-90.8), and did not report a sample size calculation (n = 125, 82.2% [95% CI 75.4-87.5]). The most common algorithms used were support vector machine (n = 86/522, 16.5% [95% CI 13.5-19.9]) and random forest (n = 73/522, 14% [95% CI 11.3-17.2]). Values for area under the Receiver Operating Characteristic curve ranged from 0.45 to 1.00. Calibration metrics were often missed (n = 494/522, 94.6% [95% CI 92.4-96.3]). CONCLUSION: Our review revealed that focus is required on handling of missing values, methods for internal validation, and reporting of calibration to improve the methodological conduct of studies on machine learning-based prediction models. SYSTEMATIC REVIEW REGISTRATION: PROSPERO, CRD42019161764.

Indicators of questionable research practices were identified in 163,129 randomized controlled trials.

OBJECTIVES: To explore indicators of the following questionable research practices (QRPs) in randomized controlled trials (RCTs): (1) risk of bias in four domains (random sequence generation, allocation concealment, blinding of participants and personnel, and blinding of outcome assessment); (2) modifications in primary outcomes that were registered in trial registration records (proxy for selective reporting bias); (3) ratio of the achieved to planned sample sizes; and (4) statistical discrepancy. STUDY DESIGN AND SETTING: Full texts of all human RCTs published in PubMed in 1996-2017 were automatically identified and information was collected automatically. Potential indicators of QRPs included author-specific, publication-specific, and journal-specific characteristics. Beta, logistic, and linear regression models were used to identify associations between these potential indicators and QRPs. RESULTS: We included 163,129 RCT publications. The median probability of bias assessed using Robot Reviewer software ranged between 43% and 63% for the four risk of bias domains. A more recent publication year, trial registration, mentioning of CONsolidated Standards Of Reporting Trials-checklist, and a higher journal impact factor were consistently associated with a lower risk of QRPs. CONCLUSION: This comprehensive analysis provides an insight into indicators of QRPs. Researchers should be aware that certain characteristics of the author team and publication are associated with a higher risk of QRPs.

How to conduct a systematic review and meta-analysis of prognostic model studies.

BACKGROUND: Prognostic models are typically developed to estimate the risk that an individual in a particular health state will develop a particular health outcome, to support (shared) decision making. Systematic reviews of prognostic model studies can help identify prognostic models that need to further be validated or are ready to be implemented in healthcare. OBJECTIVES: To provide a step-by-step guidance on how to conduct and read a systematic review of prognostic model studies and to provide an overview of methodology and guidance available for every step of the review progress. SOURCES: Published, peer-reviewed guidance articles. CONTENT: We describe the following steps for conducting a systematic review of prognosis studies: 1) Developing the review question using the Population, Index model, Comparator model, Outcome(s), Timing, Setting format, 2) Searching and selection of articles, 3) Data extraction using the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) checklist, 4) Quality and risk of bias assessment using the Prediction model Risk Of Bias ASsessment (PROBAST) tool, 5) Analysing data and undertaking quantitative meta-analysis, and 6) Presenting summary of findings, interpreting results, and drawing conclusions. Guidance for each step is described and illustrated using a case study on prognostic models for patients with COVID-19. IMPLICATIONS: Guidance for conducting a systematic review of prognosis studies is available, but the implications of these reviews for clinical practice and further research highly depend on complete reporting of primary studies.

Comparative performance of cardiovascular risk prediction models in people living with HIV.

Background: Current cardiovascular risk assessment in people living with HIV is based on general risk assessment tools; however, whether these tools can be applied in sub-Saharan African populations has been questioned. Objectives: The study aimed to assess cardiovascular risk classification of common cardiovascular disease (CVD) risk prediction models compared to the Data Collection on Adverse Events of Anti-HIV Drugs (D:A:D) 2010 and 2016 models in people living with HIV. Method: Cardiovascular disease risk was estimated by Framingham Cardiovascular and Heart Disease (FHS-CVD, FHS-CHD), Atherosclerotic Cardiovascular Disease (ASCVD) and D:A:D 2010 and 2016 risk prediction models for HIV-infected participants of the Ndlovu Cohort Study, Limpopo, rural South Africa. Participants were classified to be at low (< 10%), moderate (10% - 20%), or high-risk (> 20%) of CVD within 10 years for general CVD and five years for D:A:D models. Kappa statistics were used to determine agreement between CVD risk prediction models. Subgroup analysis was performed according to age. Results: The analysis comprised 735 HIV-infected individuals, predominantly women (56.7%), average age 43.9 (8.8) years. The median predicted CVD risk for D:A:D 2010 and FHS-CVD was 4% and for ASCVD and FHS-CHD models, 3%. For the D:A:D 2016 risk prediction model, the figure was 5%. High 10-year CVD risk was predicted for 2.9%, 0.5%, 0.7%, 3.1% and 6.6% of the study participants by FHS-CVD, FHS-CHD, ASCVD, and D:A:D 2010 and 2016. Kappa statistics ranged from 0.34 for ASCVD to 0.60 for FHS-CVD as compared to the D:A:D 2010 risk prediction model. Conclusion: Overall, predicted CVD risk is low in this population. Compared to D:A:D 2010, CVD risk estimated by the FHS-CVD model showed similar overall results for risk classification. With the exception of the D:A:D model, all other risk prediction models classified fewer people to be at high estimated CVD risk. Prospective studies are needed to develop and validate CVD risk algorithms in people living with HIV in sub-Saharan Africa.

The score after 10 years of registration of systematic review protocols.

BACKGROUND: With the exponential growth of published systematic reviews (SR), there is a high potential for overlapping and redundant duplication of work. Prospective protocol registration gives the opportunity to assess the added value of a new study or review, thereby potentially reducing research waste and simultaneously increasing transparency and research quality. The PROSPERO database for SR protocol registration was launched 10 years ago. This study aims to assess the proportion SRs of intervention studies with a protocol registration (or publication) and explore associations of SR characteristics with protocol registration status. METHODS: PubMed was searched for SRs of human intervention studies published in January 2020 and January 2021. After random-stratified sampling and eligibility screening, data extraction on publication and journal characteristics, and protocol registration status, was performed. Both descriptive and multivariable comparative statistical analyses were performed. RESULTS: A total of 357 SRs (2020: n = 163; 2021: n = 194) were included from a random sample of 1267 publications. Of the published SRs, 38% had a protocol. SRs that reported using PRISMA as a reporting guideline had higher odds of having a protocol than publications that did not report PRISMA (OR 2.71; 95% CI: 1.21 to 6.09). SRs with a higher journal impact factor had higher odds of having a protocol (OR 1.12; 95% CI 1.04 to 1.25). Publications from Asia had a lower odds of having a protocol (OR 0.43; 95% CI 0.23 to 0.80, reference category = Europe). Of the 33 SRs published in journals that endorse PROSPERO, 45% did not have a protocol. Most SR protocols were registered in PROSPERO (n = 129; 96%). CONCLUSIONS: We found that 38% of recently published SRs of interventions reported a registered or published protocol. Protocol registration was significantly associated with a higher impact factor of the journal publishing the SR and a more frequent self-reported use of the PRISMA guidelines. In some parts of the world, SR protocols are more often registered or published than others. To guide strategies to increase the uptake of SR protocol registration, further research is needed to gain understanding of the benefits and informativeness of SRs protocols among different stakeholders. SYSTEMATIC REVIEW REGISTRATION: osf.io/9kj7r/.

Risk of bias of prognostic models developed using machine learning: a systematic review in oncology.

BACKGROUND: Prognostic models are used widely in the oncology domain to guide medical decision-making. Little is known about the risk of bias of prognostic models developed using machine learning and the barriers to their clinical uptake in the oncology domain. METHODS: We conducted a systematic review and searched MEDLINE and EMBASE databases for oncology-related studies developing a prognostic model using machine learning methods published between 01/01/2019 and 05/09/2019. The primary outcome was risk of bias, judged using the Prediction model Risk Of Bias ASsessment Tool (PROBAST). We described risk of bias overall and for each domain, by development and validation analyses separately. RESULTS: We included 62 publications (48 development-only; 14 development with validation). 152 models were developed across all publications and 37 models were validated. 84% (95% CI: 77 to 89) of developed models and 51% (95% CI: 35 to 67) of validated models were at overall high risk of bias. Bias introduced in the analysis was the largest contributor to the overall risk of bias judgement for model development and validation. 123 (81%, 95% CI: 73.8 to 86.4) developed models and 19 (51%, 95% CI: 35.1 to 67.3) validated models were at high risk of bias due to their analysis, mostly due to shortcomings in the analysis including insufficient sample size and split-sample internal validation. CONCLUSIONS: The quality of machine learning based prognostic models in the oncology domain is poor and most models have a high risk of bias, contraindicating their use in clinical practice. Adherence to better standards is urgently needed, with a focus on sample size estimation and analysis methods, to improve the quality of these models.

Clinical prediction models for mortality in patients with covid-19: external validation and individual participant data meta-analysis.

OBJECTIVE: To externally validate various prognostic models and scoring rules for predicting short term mortality in patients admitted to hospital for covid-19. DESIGN: Two stage individual participant data meta-analysis. SETTING: Secondary and tertiary care. PARTICIPANTS: 46 914 patients across 18 countries, admitted to a hospital with polymerase chain reaction confirmed covid-19 from November 2019 to April 2021. DATA SOURCES: Multiple (clustered) cohorts in Brazil, Belgium, China, Czech Republic, Egypt, France, Iran, Israel, Italy, Mexico, Netherlands, Portugal, Russia, Saudi Arabia, Spain, Sweden, United Kingdom, and United States previously identified by a living systematic review of covid-19 prediction models published in The BMJ, and through PROSPERO, reference checking, and expert knowledge. MODEL SELECTION AND ELIGIBILITY CRITERIA: Prognostic models identified by the living systematic review and through contacting experts. A priori models were excluded that had a high risk of bias in the participant domain of PROBAST (prediction model study risk of bias assessment tool) or for which the applicability was deemed poor. METHODS: Eight prognostic models with diverse predictors were identified and validated. A two stage individual participant data meta-analysis was performed of the estimated model concordance (C) statistic, calibration slope, calibration-in-the-large, and observed to expected ratio (O:E) across the included clusters. MAIN OUTCOME MEASURES: 30 day mortality or in-hospital mortality. RESULTS: Datasets included 27 clusters from 18 different countries and contained data on 46 914patients. The pooled estimates ranged from 0.67 to 0.80 (C statistic), 0.22 to 1.22 (calibration slope), and 0.18 to 2.59 (O:E ratio) and were prone to substantial between study heterogeneity. The 4C Mortality Score by Knight et al (pooled C statistic 0.80, 95% confidence interval 0.75 to 0.84, 95% prediction interval 0.72 to 0.86) and clinical model by Wang et al (0.77, 0.73 to 0.80, 0.63 to 0.87) had the highest discriminative ability. On average, 29% fewer deaths were observed than predicted by the 4C Mortality Score (pooled O:E 0.71, 95% confidence interval 0.45 to 1.11, 95% prediction interval 0.21 to 2.39), 35% fewer than predicted by the Wang clinical model (0.65, 0.52 to 0.82, 0.23 to 1.89), and 4% fewer than predicted by Xie et al's model (0.96, 0.59 to 1.55, 0.21 to 4.28). CONCLUSION: The prognostic value of the included models varied greatly between the data sources. Although the Knight 4C Mortality Score and Wang clinical model appeared most promising, recalibration (intercept and slope updates) is needed before implementation in routine care.

What did we learn in 35 years of research on nutrition and supplements for age-related macular degeneration: a systematic review.

The aim of this paper is to summarize all available evidence from systematic reviews, randomized controlled trials (RCTs) and comparative nonrandomized studies (NRS) on the association between nutrition and antioxidant, vitamin, and mineral supplements and the development or progression of age-related macular degeneration (AMD). The Cochrane Database of Systematic Reviews, Cochrane register CENTRAL, MEDLINE and Embase were searched and studies published between January 2015 and May 2021 were included. The certainty of evidence was assessed according to the GRADE methodology. The main outcome measures were development of AMD, progression of AMD, and side effects. We included 7 systematic reviews, 7 RCTs, and 13 NRS. A high consumption of specific nutrients, i.e. ß-carotene, lutein and zeaxanthin, copper, folate, magnesium, vitamin A, niacin, vitamin B6, vitamin C, docosahexaenoic acid, and eicosapentaenoic acid, was associated with a lower risk of progression of early to late AMD (high certainty of evidence). Use of antioxidant supplements and adherence to a Mediterranean diet, characterized by a high consumption of vegetables, whole grains, and nuts and a low consumption of red meat, were associated with a decreased risk of progression of early to late AMD (moderate certainty of evidence). A high consumption of alcohol was associated with a higher risk of developing AMD (moderate certainty of evidence). Supplementary vitamin C, vitamin E, or ß-carotene were not associated with the development of AMD, and supplementary omega-3 fatty acids were not associated with progression to late AMD (high certainty of evidence). Research in the last 35 years included in our overview supports that a high intake of specific nutrients, the use of antioxidant supplements and adherence to a Mediterranean diet decrease the risk of progression of early to late AMD.