Estimating the rate of overdiagnosis with prostate cancer screening: evidence from the Finnish component of the European Randomized Study of Screening for Prostate Cancer.

PURPOSE: Screening for prostate cancer may have limited impact on decreasing prostate cancer-related mortality. A major disadvantage is overdiagnosis, whereby lesions are identified that would not have become evident during the man's lifetime if screening had not taken place. The present study aims to estimate the rate of overdiagnosis using Finnish data from the European randomized trial of prostate cancer screening. METHODS: We used data from 80,149 men randomized to a screening or a control group, distinguishing four birth cohorts. We used the "catch-up method" to identify when the difference in the cumulative incidence of prostate cancer between the screening and control groups had stabilized, implying that the screening has no further effect. We define the overdiagnosis rate to be the relative excess cumulative incidence in the screened group at that point. As an independent method, we also examined the diagnosis rates of T1c tumors as an indicator of early tumors detected by PSA. RESULTS: The estimates of overdiagnosis rates from the catch-up method using the full period of available follow-up ranged between cohorts from 2.3% to 15.4%, and the T1c analysis gave very similar results. CONCLUSION: Some overdiagnosis has occurred, but there is uncertainty about its extent. A long follow-up is required to demonstrate the full impact of screening. We evaluated the overdiagnosis rates at a population level, associated with being offered screening, taking account of contamination (screening among the controls). The overall evaluation of screening should incorporate mortality benefit, cost-effectiveness, and quality of life.

Relative efficiencies of alternative preference-based designs for randomised trials.

Recent work has shown that outcomes in clinical trials can be affected by which treatment the trial participants would select if they were allowed to do so, and if they do or do not actually receive that treatment. These influences are known as selection and preference effects, respectively. Unfortunately, they cannot be evaluated in conventional, parallel group trials because patient preferences remain unknown. However, several alternative designs have been proposed, to measure and take account of patient preferences. In this paper, we discuss three preference-based designs (the two-stage, fully randomised, and partially randomised designs). In conventional trials, only the treatment effect is estimable, while the preference-based designs have the potential to estimate some or all of the selection and preference effects. The relative efficiency of these designs is affected by several factors, including the proportion of participants who are undecided about treatments, or who are unable or unwilling to state a preference; the relative preference rate between the treatments being compared, among patients who do have a preference; and the ratio of patients randomised to each treatment. We also discuss the advantages and disadvantages of these designs under different scenarios.

A systematic survey of randomised trials that stopped early for reasons of futility.

BACKGROUND: Randomised trial protocols may incorporate interim analyses, with the potential to stop the study for futility if early data show insufficient promise of a treatment benefit. Previously, we have shown that this approach will theoretically lead to mis-estimation of the treatment effect. We now wished to ascertain the importance of this phenomenon in practice. METHODS: We reviewed the methods and results in a set of trials that had stopped for futility, identified through an extensive literature search. We recorded clinical areas, interventions, study design, outcomes, trial setting, sponsorship, planned and actual treatment effects, sample sizes; power; and if there was a data safety monitoring board, or a published protocol. We identified: if interim analyses were pre-specified, and how many analyses actually occurred; what pre-specified criteria might define futility; if a futility analysis formed the basis for stopping; who made the decision to stop; and the conditional power of each study, i.e. the probability of statistically significant results if the study were to continue to its complete sample size. RESULTS: We identified 52 eligible trials, covering many clinical areas. Most trials had multiple centres, tested drugs, and 40% were industry sponsored. There were 75% where at least one interim analysis was planned a priori; a majority had only one interim analysis, typically with about half the target total sample size. A majority of trials did not pre-define a stopping rule, and a variety of reasons were given for stopping. Few studies calculated and reported low conditional power to justify the early stop. When conditional power could be calculated, it was typically low, especially under the current trend hypothesis. However, under the original design hypothesis, a few studies had relatively high conditional power. Data collection often continued after the interim analysis. CONCLUSIONS: Although other factors will typically be involved, we conclude that, from the perspective of conditional power, stopping early for futility was probably reasonable in most cases, but documentation of the basis for stopping was often missing or vague. Interpretation of truncated trials would be enhanced by improved reporting of stopping protocols, and of their actual execution.

Randomised trials with provision for early stopping for benefit (or harm): The impact on the estimated treatment effect.

Stopping rules for clinical trials are primarily intended to control Type I error rates if interim analyses are planned, but less is known about the impact that potential stopping has on estimating treatment benefit. In this paper, we derive analytic expressions for (1) the over-estimation of benefit in studies that stop early, (2) the under-estimation of benefit in completed studies, and (3) the overall bias in studies with a stopping rule. We also examine the probability of stopping early and the situation in meta-analyses. Numerical evaluations show that the greatest concern is with over-estimation of benefit in stopped studies, especially if the probability of stopping early is small. The overall bias is usually less than 10% of the true benefit, and under-estimation in completed studies is also typically small. The probability of stopping depends on the true treatment effect and sample size. The magnitude of these effects depends on the particular rule adopted, but we show that the maximum overall bias is the same for all stopping rules. We also show that an essentially unbiased meta-analysis estimate of benefit can be recovered, even if some component studies have stopping rules. We illustrate these methods using data from three clinical trials. The results confirm our earlier empirical work on clinical trials. Investigators may consult our numerical results for guidance on potential mis-estimation and bias in the treatment effect if a stopping rule is adopted. Particular concern is warranted in studies that actually stop early, where interim results may be quite misleading.

Wound irrigation does not affect health-related quality of life after open fractures: results of a randomized controlled trial.

AIMS: The Fluid Lavage in Open Fracture Wounds (FLOW) trial was a multicentre, blinded, randomized controlled trial that used a 2 × 3 factorial design to evaluate the effect of irrigation solution (soap versus normal saline) and irrigation pressure (very low versus low versus high) on health-related quality of life (HRQL) in patients with open fractures. In this study, we used this dataset to ascertain whether these factors affect whether HRQL returns to pre-injury levels at 12-months post-injury. PATIENTS AND METHODS: Participants completed the Short Form-12 (SF-12) and the EuroQol-5 Dimensions (EQ-5D) at baseline (pre-injury recall), at two and six weeks, and at three, six, nine and 12-months post-fracture. We calculated the Physical Component Score (PCS) and the Mental Component Score (MCS) of the SF-12 and the EQ-5D utility score, conducted an analysis using a multi-level generalized linear model, and compared differences between the baseline and 12-month scores. RESULTS: We found no clinically important differences between irrigating solutions or pressures for the SF-12 PCS, SF-12 MCS and EQ-5D. Irrespective of treatment, participants had not returned to their pre-injury function at 12-months for any of the three outcomes (p < 0.001). CONCLUSION: Neither the composition of the irrigation solution nor irrigation pressure applied had an effect on HRQL. Irrespective of treatment, patients had not returned to their pre-injury HRQL at 12 months post-fracture. Cite this article: Bone Joint J 2018;100-B:88-94.

Quantifying the bias in the estimated treatment effect in randomized trials having interim analyses and a rule for early stopping for futility.

In this paper, we consider the potential bias in the estimated treatment effect obtained from clinical trials, the protocols of which include the possibility of interim analyses and an early termination of the study for reasons of futility. In particular, by considering the conditional power at an interim analysis, we derive analytic expressions for various parameters of interest: (i) the underestimation or overestimation of the treatment effect in studies that stop for futility; (ii) the impact of the interim analyses on the estimation of treatment effect in studies that are completed, i.e. that do not stop for futility; (iii) the overall estimation bias in the estimated treatment effect in a single study with such a stopping rule; and (iv) the probability of stopping at an interim analysis. We evaluate these general expressions numerically for typical trial scenarios. Results show that the parameters of interest depend on a number of factors, including the true underlying treatment effect, the difference that the trial is designed to detect, the study power, the number of planned interim analyses and what assumption is made about future data to be observed after an interim analysis. Because the probability of stopping early is small for many practical situations, the overall bias is often small, but a more serious issue is the potential for substantial underestimation of the treatment effect in studies that actually stop for futility. We also consider these ideas using data from an illustrative trial that did stop for futility at an interim analysis. Copyright © 2017 John Wiley & Sons, Ltd.

Beyond the treatment effect: Evaluating the effects of patient preferences in randomised trials.

The treatments under comparison in a randomised trial should ideally have equal value and acceptability - a position of equipoise - to study participants. However, it is unlikely that true equipoise exists in practice, because at least some participants may have preferences for one treatment or the other, for a variety of reasons. These preferences may be related to study outcomes, and hence affect the estimation of the treatment effect. Furthermore, the effects of preferences can sometimes be substantial, and may even be larger than the direct effect of treatment. Preference effects are of interest in their own right, but they cannot be assessed in the standard parallel group design for a randomised trial. In this paper, we describe a model to represent the impact of preferences on trial outcomes, in addition to the usual treatment effect. In particular, we describe how outcomes might differ between participants who would choose one treatment or the other, if they were free to do so. Additionally, we investigate the difference in outcomes depending on whether or not a participant receives his or her preferred treatment, which we characterise through a so-called preference effect. We then discuss several study designs that have been proposed to measure and exploit data on preferences, and which constitute alternatives to the conventional parallel group design. Based on the model framework, we determine which of the various preference effects can or cannot be estimated with each design. We also illustrate these ideas with some examples of preference designs from the literature.

Estimation of screening sensitivity and sojourn time from an organized screening program.

Regular screening with mammography is widely recommended to reduce breast cancer mortality. However, whether breast screening does more harm than good has long been debated. Since a full evaluation of the effect on mortality could take 10-15 years in order to provide a reliable estimate of the eventual benefits and harms, it is unrealistic to expect each new modification of a screening technique to be evaluated in this way. Therefore, one needs to rapidly estimate suitable measures of the screening effect. In this paper, two measures of interest, the length of the pre-clinical state and the screening false negative rate, are discussed. A procedure is proposed to model the pre-clinical disease state duration, the false negative rate of the screening exam, and the underlying incidence rate in the screened population. We applied the model to data from the Ontario Breast Screening Program in Canada. Our results suggest that the mean preclinical duration is longer than 2 years. We also find only small marginal gains by screening every two instead of three years. The most important objective of a screening program should be to encourage first-time screening attendance.

Inference on cancer screening exam accuracy using population-level administrative data.

This paper develops a model for cancer screening and cancer incidence data, accommodating the partially unobserved disease status, clustered data structures, general covariate effects, and dependence between exams. The true unobserved cancer and detection status of screening participants are treated as latent variables, and a Markov Chain Monte Carlo algorithm is used to estimate the Bayesian posterior distributions of the diagnostic error rates and disease prevalence. We show how the Bayesian approach can be used to draw inferences about screening exam properties and disease prevalence while allowing for the possibility of conditional dependence between two exams. The techniques are applied to the estimation of the diagnostic accuracy of mammography and clinical breast examination using data from the Ontario Breast Screening Program in Canada.

Longitudinal measurements of oxygen consumption in growing infants during the first weeks after birth: old data revisited.

BACKGROUND: In a study conducted in 1966-1969, longitudinal measurements were made of the metabolic rate in growing infants. Statistical methods for analyzing longitudinal data weren't readily accessible at that time. OBJECTIVES: To measure minimal rates of oxygen consumption (V·O2, ml/min) in growing infants during the first postnatal weeks and to determine the relationships between postnatal increases in V·O2, body size and postnatal age. METHODS: We studied 61 infants of any birth weight or gestational age, including 19 of very low birth weight. The infants, nursed in incubators, were clinically well and without need of oxygen supplementation or respiratory assistance. Serial measures of V·O2 using a closed-circuit method were obtained at approximately weekly intervals. V·O2 was measured under thermoneutral conditions with the infant asleep or resting quietly. Data were analyzed using mixed-effects models. RESULTS: During early postnatal growth, V·O2 rises as surface area (m(2))(1.94) (standard error, SE 0.054) or body weight (kg)(1.24) (SE 0.033). Multivariate analyses show statistically significant effects of both size and age. Reference intervals (RIs) for V·O2 for fixed values of body weight and postnatal age are presented. As V·O2 rises with increasing size and age, there is an increase in the skin-operative environmental temperature gradient (T skin-op) required for heat loss. Required T skin-op can be predicted from surface area and heat loss (heat production minus heat storage). CONCLUSIONS: Generation of RIs for minimal rates of V·O2 in growing infants from the 1960s was enabled by application of mixed-effects statistical models for analyses of longitudinal data. Results apply to the precaffeine era of neonatal care.

Optimal allocation of participants for the estimation of selection, preference and treatment effects in the two-stage randomised trial design.

Outcomes in clinical trials may be affected by the choice of treatment that participants might make, if they were indeed allowed to choose (a so-called selection effect), and by whether they actually receive their preferred treatment (a preference effect). Selection and preference effects can be important, but they cannot be estimated in the conventional trial design. An alternative approach is the two-stage randomised trial, in which participants are first randomly divided into two subgroups. In one subgroup, participants are randomly assigned to treatments, while in the other, participants are allowed to choose their own treatment. This approach yields estimates of the direct treatment effect, and of the preference and selection effects. The latter two provide insight that goes considerably beyond what is possible in the standard randomised trial. In this paper, we determine the optimal proportion of participants who should be allocated to the choice subgroup. The precision of the estimated selection, preference and treatment effects are functions of: the total sample size; the proportion of participants allocated to choose their treatment; the variances of the outcome; the proportions of participants who select each treatment in the choice group; and the selection, preference and treatment effects themselves. We develop general expressions for the optimum proportion of participants in the choice group, depending on which effects are of primary interest. We illustrate the results with trial data comparing alternative clinical management strategies for women with abnormal results on cervical screening.

Effect of dependent errors in the assessment of diagnostic or screening test accuracy when the reference standard is imperfect.

When no gold standard is available to evaluate a diagnostic or screening test, as is often the case, an imperfect reference standard test must be used instead. Furthermore, the errors of the test and its reference standard may not be independent. Some authors have opined that positively dependent errors will lead to overestimation of test performance. Although positive dependence does increase agreement between the test and the reference standard, it is not clear if test accuracy will necessarily be overestimated in this situation, and the case of negatively associated test errors is even less clear. To examine this issue in more detail, we derive the apparent sensitivity, specificity, and overall accuracy of a test relative to an imperfect reference standard and the bias in these parameters. We demonstrate that either positive or negative bias can occur if the reference standard is imperfect. The type and magnitude of bias depend on several components: the disease prevalence, the true test sensitivity and specificity, the covariance between the false-negative test errors among the true disease cases, and the covariance between the false-positive test errors among the true noncases. If, for example, sensitivity and specificity are 0.8 for both the test and reference standard and the errors have a moderate positive dependence, test sensitivity is then underestimated at low prevalence but overestimated at high prevalence, while the opposite occurs for specificity. We illustrate these ideas through general numerical calculations and an empirical example of screening for breast cancer with magnetic resonance imaging and mammography.

Local estimates of population attributable risk.

OBJECTIVE: To investigate properties of population attributable risk (PAR) estimates, when its components come from different sources. Examples include situations where one requires local estimates of PAR for a study subset (e.g., in states or counties within a national study) or if one wishes to apply the findings of an epidemiologic study to another population. STUDY DESIGN AND SETTING: A framework for estimating local PAR values is developed, and then illustrated using synthetic and empirical data. RESULTS: A general expression for the variance of a local PAR estimate is formulated. It involves three components, reflecting (1) the variance of the disease relative risk associated with exposure to a risk factor, (2) the variance of the exposure prevalence (P), and (3) their covariance. The effects of variable stratum sizes, case-control sample size ratios, and variation in exposure P are illustrated by some synthetic scenarios, and with data from an international case-control study of heart disease. CONCLUSION: The precision of local PAR estimates can be considerably improved by incorporating external data, as opposed to limiting the calculation to data only from the local population. In some cases, variation in local PAR estimates largely reflects uncertainty in the local estimate of exposure P.

Statistical issues in the design and analysis of expertise-based randomized clinical trials.

In order to avoid certain difficulties with the conventional randomized clinical trial design, the expertise-based design has been proposed as an alternative. In the expertise-based design, patients are randomized to clinicians (e.g. surgeons), who then treat all their patients with their preferred intervention. This design recognizes individual clinical preferences and so may reduce the rates of procedural crossovers compared with the conventional design. It may also facilitate recruitment of clinicians, because they are always allowed to deliver their therapy of choice, a feature that may also be attractive to patients.The expertise-based design avoids the possibility of so-called differential expertise bias. If a standard treatment is generally more familiar to clinicians than a new experimental treatment, then in the conventional design, more patients randomized to the standard treatment will have an expert clinician, compared with patients randomized to the experimental treatment. If expertise affects the study outcome, then a biased comparison of the treatment groups will occur.We examined the relative efficiency of estimating the treatment effect in the expertise-based and conventional designs. We recognize that expected patient outcomes may be better in the expertise-based design, which in turn may modify the estimated treatment effect. In particular, a larger treatment effect in the expertise-based design can sometimes offset a higher standard error arising from the confounding of clinician effects with treatments.These concepts are illustrated with data taken from a randomized trial of two alternative surgical techniques for tibial fractures.

A geometric confidence ellipse approach to the estimation of the ratio of two variables.

One is often interested in the ratio of two variables, for example in genetics, assessing drug effectiveness, and in health economics. In this paper, we derive an explicit geometric solution to the general problem of identifying the two tangents from an arbitrary external point to an ellipse. This solution permits numerical integration of a bivariate normal distribution over a wedge-shaped region bounded by the tangents, which yields an evaluation of the tangent slopes as confidence limits on the ratio of the component variables. After suitable adjustment of the confidence coverage of the ellipse, these confidence limits are shown to be equivalent to those from Fieller's method. However, the geometric approach allows additional interpretation of the data through identification of the points of tangency, the ellipse itself, and expressions for the coverage probability of the confidence interval. Numerical evaluations using the theoretical expressions for the geometric confidence intervals (but ignoring sample variation in the underlying parameters) suggested that they perform well overall and are slightly conservative. Simulations that do take account of sample variation in the underlying parameters again suggested that the intervals perform well overall, although here they are slightly anti-conservative. Coverage probabilities for the confidence intervals were only weakly dependent on the distance and correlation of the ellipse, but there were asymmetries in the failure rates of the upper and lower confidence limits in some configurations. The probability of no real solution existing was also evaluated. These ideas are illustrated by a practical example.

An opportunity cost approach to sample size calculation in cost-effectiveness analysis.

The inclusion of economic evaluations as part of clinical trials has led to concerns about the adequacy of trial sample size to support such analysis. The analytical tool of cost-effectiveness analysis is the incremental cost-effectiveness ratio (ICER), which is compared with a threshold value (lambda) as a method to determine the efficiency of a health-care intervention. Accordingly, many of the methods suggested to calculating the sample size requirements for the economic component of clinical trials are based on the properties of the ICER. However, use of the ICER and a threshold value as a basis for determining efficiency has been shown to be inconsistent with the economic concept of opportunity cost. As a result, the validity of the ICER-based approaches to sample size calculations can be challenged. Alternative methods for determining improvements in efficiency have been presented in the literature that does not depend upon ICER values. In this paper, we develop an opportunity cost approach to calculating sample size for economic evaluations alongside clinical trials, and illustrate the approach using a numerical example. We compare the sample size requirement of the opportunity cost method with the ICER threshold method. In general, either method may yield the larger required sample size. However, the opportunity cost approach, although simple to use, has additional data requirements. We believe that the additional data requirements represent a small price to pay for being able to perform an analysis consistent with both concept of opportunity cost and the problem faced by decision makers.

Developing common metrics of mechanical exposures across aetiological studies of low back pain in working populations for use in meta-analysis.

OBJECTIVES: One of the challenges of conducting meta-analyses on the relationship between workplace mechanical exposures and low back pain is that mechanical exposures are reported in a wide variety of ways. We aimed to develop common metrics to apply in the translation of literature-based workplace mechanical exposures for use in meta-analyses, and to test the metrics' measurement properties. METHODS: We developed a set of 7-point scales to capture the intensity of important aspects of mechanical exposures that may be related to the development of low back pain in workers. The scales represented three dimensions of mechanical exposures at work: (1) trunk posture, (2) weight lifted or force exerted and (3) spinal loading, and estimated both peak and cumulative loads. Measurement properties of the scales were tested through a survey of experts in biomechanics and ergonomics who were asked to rate literature-based workplace exposure definitions using the scales and provide estimates of their confidence in their ratings. RESULTS: For each dimension the ratings for peak loads tended to be higher than the cumulative load ratings. The inter-rater reliability for the scales ranged from 0.3 to 0.5; we would need to average the ratings of at least four expert raters to have an acceptable level of reliability (>0.7). Inter-expert reliability was positively related to the experts' level of confidence in their ratings. In most cases the ranking of intensity ratings from the experts matched the ranking of exposure intensity from the original articles. CONCLUSIONS: This study provides insight into estimating the intensity of literature-based mechanical exposure metrics using a common set of scales which can be applied across epidemiologic studies. These metrics may be useful to quantify the relationship between workplace mechanical exposure and low back pain in a systematic review and meta-analysis.