External validation and updating of prediction models of bleeding risk in patients with cancer receiving anticoagulants.

OBJECTIVE: Patients with cancer are at increased bleeding risk, and anticoagulants increase this risk even more. Yet, validated bleeding risk models for prediction of bleeding risk in patients with cancer are lacking. The aim of this study is to predict bleeding risk in anticoagulated patients with cancer. METHODS: We performed a study using the routine healthcare database of the Julius General Practitioners' Network. Five bleeding risk models were selected for external validation. Patients with a new cancer episode during anticoagulant treatment or those initiating anticoagulation during active cancer were included. The outcome was the composite of major bleeding and clinically relevant non-major (CRNM) bleeding. Next, we internally validated an updated bleeding risk model accounting for the competing risk of death. RESULTS: The validation cohort consisted of 1304 patients with cancer, mean age 74.0±10.9 years, 52.2% males. In total 215 (16.5%) patients developed a first major or CRNM bleeding during a mean follow-up of 1.5 years (incidence rate; 11.0 per 100 person-years (95% CI 9.6 to 12.5)). The c-statistics of all selected bleeding risk models were low, around 0.56. Internal validation of an updated model accounting for death as competing risk showed a slightly improved c-statistic of 0.61 (95% CI 0.54 to 0.70). On updating, only age and a history of bleeding appeared to contribute to the prediction of bleeding risk. CONCLUSIONS: Existing bleeding risk models cannot accurately differentiate bleeding risk between patients. Future studies may use our updated model as a starting point for further development of bleeding risk models in patients with cancer.

Age is the main determinant of COVID-19 related in-hospital mortality with minimal impact of pre-existing comorbidities, a retrospective cohort study.

BACKGROUND: Age and comorbidities increase COVID-19 related in-hospital mortality risk, but the extent by which comorbidities mediate the impact of age remains unknown. METHODS: In this multicenter retrospective cohort study with data from 45 Dutch hospitals, 4806 proven COVID-19 patients hospitalized in Dutch hospitals (between February and July 2020) from the CAPACITY-COVID registry were included (age 69[58-77]years, 64% men). The primary outcome was defined as a combination of in-hospital mortality or discharge with palliative care. Logistic regression analysis was performed to analyze the associations between sex, age, and comorbidities with the primary outcome. The effect of comorbidities on the relation of age with the primary outcome was evaluated using mediation analysis. RESULTS: In-hospital COVID-19 related mortality occurred in 1108 (23%) patients, 836 (76%) were aged ≥70 years (70+). Both age 70+ and female sex were univariably associated with outcome (odds ratio [OR]4.68, 95%confidence interval [4.02-5.45], OR0.68[0.59-0.79], respectively;both p<  0.001). All comorbidities were univariably associated with outcome (p<0.001), and all but dyslipidemia remained significant after adjustment for age70+ and sex. The impact of comorbidities was attenuated after age-spline adjustment, only leaving female sex, diabetes mellitus (DM), chronic kidney disease (CKD), and chronic pulmonary obstructive disease (COPD) significantly associated (female OR0.65[0.55-0.75], DM OR1.47[1.26-1.72], CKD OR1.61[1.32-1.97], COPD OR1.30[1.07-1.59]). Pre-existing comorbidities in older patients negligibly (<6% in all comorbidities) mediated the association between higher age and outcome. CONCLUSIONS: Age is the main determinant of COVID-19 related in-hospital mortality, with negligible mediation effect of pre-existing comorbidities. TRIAL REGISTRATION: CAPACITY-COVID ( NCT04325412 ).

Management of superficial venous thrombosis based on individual risk profiles: protocol for the development and validation of three prognostic prediction models in large primary care cohorts.

BACKGROUND: Superficial venous thrombosis (SVT) is considered a benign thrombotic condition in most patients. However, it also can cause serious complications, such as clot progression to deep venous thrombosis (DVT) and pulmonary embolism (PE). Although most SVT patients are encountered in primary healthcare, studies on SVT nearly all were focused on patients seen in the hospital setting. This paper describes the protocol of the development and external validation of three prognostic prediction models for relevant clinical outcomes in SVT patients seen in primary care: (i) prolonged (painful) symptoms within 14 days since SVT diagnosis, (ii) for clot progression to DVT or PE within 45 days and (iii) for clot recurrence within 12 months. METHODS: Data will be used from four primary care routine healthcare registries from both the Netherlands and the UK; one UK registry will be used for the development of the prediction models and the remaining three will be used as external validation cohorts. The study population will consist of patients ≥18 years with a diagnosis of SVT. Selection of SVT cases will be based on a combination of ICPC/READ/Snowmed coding and free text clinical symptoms. Predictors considered are sex, age, body mass index, clinical SVT characteristics, and co-morbidities including (history of any) cardiovascular disease, diabetes, autoimmune disease, malignancy, thrombophilia, pregnancy or puerperium and presence of varicose veins. The prediction models will be developed using multivariable logistic regression analysis techniques for models i and ii, and for model iii, a Cox proportional hazards model will be used. They will be validated by internal-external cross-validation as well as external validation. DISCUSSION: There are currently no prediction models available for predicting the risk of serious complications for SVT patients presenting in primary care settings. We aim to develop and validate new prediction models that should help identify patients at highest risk for complications and to support clinical decision making for this understudied thrombo-embolic disorder. Challenges that we anticipate to encounter are mostly related to performing research in large, routine healthcare databases, such as patient selection, endpoint classification, data harmonisation, missing data and avoiding (predictor) measurement heterogeneity.

The prognostic value of the hamstring outcome score to predict the risk of hamstring injuries.

OBJECTIVES: Hamstring injuries are common among soccer players. The hamstring outcome score (HaOS) might be useful to identify amateur players at risk of hamstring injury. Therefore the aims of this study were: To determine the association between the HaOS and prior and new hamstring injuries in amateur soccer players, and to determine the prognostic value of the HaOS for identifying players with or without previous hamstring injuries at risk of future injury. DESIGN: Cohort study. METHODS: HaOS scores and information about previous injuries were collected at baseline and new injuries were prospectively registered during a cluster-randomized controlled trial involving 400 amateur soccer players. Analysis of variance and t-tests were used to determine the association between the HaOS and previous and new hamstring injury, respectively. Logistic regression analysis indicated the prognostic value of the HaOS for predicting new hamstring injuries. RESULTS: Analysis of data of 356 players indicated that lower HaOS scores were associated with more previous hamstring injuries (F=17.4; p=0.000) and that players with lower HaOS scores sustained more new hamstring injuries (T=3.59, df=67.23, p=0.001). With a conventional HaOS score cut-off of 80%, logistic regression models yielded a probability of hamstring injuries of 11%, 18%, and 28% for players with 0,1, or 2 hamstring injuries in the previous season, respectively. CONCLUSIONS: The HaOS is associated with previous and future hamstring injury and might be a useful tool to provide players with insight into their risk of sustaining a new hamstring injury risk when used in combination with previous injuries.

Optimizing design of research to evaluate antibiotic stewardship interventions: consensus recommendations of a multinational working group.

BACKGROUND: Antimicrobial stewardship interventions and programmes aim to ensure effective treatment while minimizing antimicrobial-associated harms including resistance. Practice in this vital area is undermined by the poor quality of research addressing both what specific antimicrobial use interventions are effective and how antimicrobial use improvement strategies can be implemented into practice. In 2016 we established a working party to identify the key design features that limit translation of existing research into practice and then to make recommendations for how future studies in this field should be optimally designed. The first part of this work has been published as a systematic review. Here we present the working group's final recommendations. METHODS: An international working group for design of antimicrobial stewardship intervention evaluations was convened in response to the fourth call for leading expert network proposals by the Joint Programming Initiative on Antimicrobial Resistance (JPIAMR). The group comprised clinical and academic specialists in antimicrobial stewardship and clinical trial design from six European countries. Group members completed a structured questionnaire to establish the scope of work and key issues to develop ahead of a first face-to-face meeting that (a) identified the need for a comprehensive systematic review of study designs in the literature and (b) prioritized key areas where research design considerations restrict translation of findings into practice. The working group's initial outputs were reviewed by independent advisors and additional expertise was sought in specific clinical areas. At a second face-to-face meeting the working group developed a theoretical framework and specific recommendations to support optimal study design. These were finalized by the working group co-ordinators and agreed by all working group members. RESULTS: We propose a theoretical framework in which consideration of the intervention rationale the intervention setting, intervention features and the intervention aims inform selection and prioritization of outcome measures, whether the research sets out to determine superiority or non-inferiority of the intervention measured by its primary outcome(s), the most appropriate study design (e.g. experimental or quasi- experimental) and the detailed design features. We make 18 specific recommendation in three domains: outcomes, objectives and study design. CONCLUSIONS: Researchers, funders and practitioners will be able to draw on our recommendations to most efficiently evaluate antimicrobial stewardship interventions.

Measurement error in continuous endpoints in randomised trials: Problems and solutions.

In randomised trials, continuous endpoints are often measured with some degree of error. This study explores the impact of ignoring measurement error and proposes methods to improve statistical inference in the presence of measurement error. Three main types of measurement error in continuous endpoints are considered: classical, systematic, and differential. For each measurement error type, a corrected effect estimator is proposed. The corrected estimators and several methods for confidence interval estimation are tested in a simulation study. These methods combine information about error-prone and error-free measurements of the endpoint in individuals not included in the trial (external calibration sample). We show that, if measurement error in continuous endpoints is ignored, the treatment effect estimator is unbiased when measurement error is classical, while Type-II error is increased at a given sample size. Conversely, the estimator can be substantially biased when measurement error is systematic or differential. In those cases, bias can largely be prevented and inferences improved upon using information from an external calibration sample, of which the required sample size increases as the strength of the association between the error-prone and error-free endpoint decreases. Measurement error correction using already a small (external) calibration sample is shown to improve inferences and should be considered in trials with error-prone endpoints. Implementation of the proposed correction methods is accommodated by a new software package for R.

Impact of predictor measurement heterogeneity across settings on the performance of prediction models: A measurement error perspective.

It is widely acknowledged that the predictive performance of clinical prediction models should be studied in patients that were not part of the data in which the model was derived. Out-of-sample performance can be hampered when predictors are measured differently at derivation and external validation. This may occur, for instance, when predictors are measured using different measurement protocols or when tests are produced by different manufacturers. Although such heterogeneity in predictor measurement between derivation and validation data is common, the impact on the out-of-sample performance is not well studied. Using analytical and simulation approaches, we examined out-of-sample performance of prediction models under various scenarios of heterogeneous predictor measurement. These scenarios were defined and clarified using an established taxonomy of measurement error models. The results of our simulations indicate that predictor measurement heterogeneity can induce miscalibration of prediction and affects discrimination and overall predictive accuracy, to extents that the prediction model may no longer be considered clinically useful. The measurement error taxonomy was found to be helpful in identifying and predicting effects of heterogeneous predictor measurements between settings of prediction model derivation and validation. Our work indicates that homogeneity of measurement strategies across settings is of paramount importance in prediction research.

How variation in predictor measurement affects the discriminative ability and transportability of a prediction model.

BACKGROUND AND OBJECTIVE: Diagnostic and prognostic prediction models often perform poorly when externally validated. We investigate how differences in the measurement of predictors across settings affect the discriminative power and transportability of a prediction model. METHODS: Differences in predictor measurement between data sets can be described formally using a measurement error taxonomy. Using this taxonomy, we derive an expression relating variation in the measurement of a continuous predictor to the area under the receiver operating characteristic curve (AUC) of a logistic regression prediction model. This expression is used to demonstrate how variation in measurements across settings affects the out-of-sample discriminative ability of a prediction model. We illustrate these findings with a diagnostic prediction model using example data of patients suspected of having deep venous thrombosis. RESULTS: When a predictor, such as D-dimer, is measured with more noise in one setting compared to another, which we conceptualize as a difference in "classical" measurement error, the expected value of the AUC decreases. In contrast, constant, "structural" measurement error does not impact on the AUC of a logistic regression model, provided the magnitude of the error is the same among cases and noncases. As the differences in measurement methods between settings (and in turn differences in measurement error structures) become more complex, it becomes increasingly difficult to predict how the AUC will differ between settings. CONCLUSION: When a prediction model is applied to a different setting to the one in which it was developed, its discriminative ability can decrease or even increase if the magnitude or structure of the errors in predictor measurements differ between the two settings. This provides an important starting point for researchers to better understand how differences in measurement methods can affect the performance of a prediction model when externally validating or implementing it in practice.

[Power, you can never have enough of it]. / Power, daar kun je nooit genoeg van krijgen.

- In randomised trials on the effects of a medical treatment, the power of the trial corresponds to the conditional probability that the conclusion of the trial will be that the treatment is effective, given a certain treatment effect.- The time to consider the power of a trial is before conducting the study, to ensure that the design of the trial is such that there is a reasonable chance of demonstrating a clinically relevant effect.

Response Adjusted for Days of Antibiotic Risk (RADAR): evaluation of a novel method to compare strategies to optimize antibiotic use.

OBJECTIVES: The Response Adjusted for Days of Antibiotic Risk (RADAR) statistic was proposed to improve the efficiency of trials comparing antibiotic stewardship strategies to optimize antibiotic use. We studied the behaviour of RADAR in a non-inferiority trial in which a ß-lactam monotherapy strategy (n = 656) was non-inferior to fluoroquinolone monotherapy (n = 888) for patients with moderately severe community-acquired pneumonia. METHODS: Patients were ranked according to clinical outcome, using five or eight categories, and antibiotic use. RADAR was calculated as the probability that the ß-lactam group had a more favourable ranking than the fluoroquinolone group. To investigate the sensitivity of RADAR to detrimental clinical outcome we simulated increasing rates of 90-day mortality in the ß-lactam group and performed the RADAR and non-inferiority analysis. RESULTS: The RADAR of the ß-lactam group compared with the fluoroquinolone group was 60.3% (95% CI 57.9%-62.7%) using five and 58.4% (95% CI 56.0%-60.9%) using eight clinical outcome categories, all in favour of ß-lactam. Sample sizes for RADAR were 38% (250/653) and 89% (580/653) of the non-inferiority sample size calculation, using five or eight clinical outcome categories, respectively. With simulated mortality rates, loss of non-inferiority of the ß-lactam group occurred at a relative risk of 1.125 in the conventional analysis, whereas using RADAR the ß-lactam group lost superiority at a relative risk of mortality of 1.25 and 1.5, with eight and five clinical outcome categories, respectively. CONCLUSIONS: RADAR favoured ß-lactam over fluoroquinolone therapy for community-acquired pneumonia. Although RADAR required fewer patients than conventional non-inferiority analysis, the statistic was less sensitive to detrimental outcomes.