Istore: a project on innovative statistical methodologies to improve rare diseases clinical trials in limited populations.

BACKGROUND: The conduct of rare disease clinical trials is still hampered by methodological problems. The number of patients suffering from a rare condition is variable, but may be very small and unfortunately statistical problems for small and finite populations have received less consideration. This paper describes the outline of the iSTORE project, its ambitions, and its methodological approaches. METHODS: In very small populations, methodological challenges exacerbate. iSTORE's ambition is to develop a comprehensive perspective on natural history course modelling through multiple endpoint methodologies, subgroup similarity identification, and improving level of evidence. RESULTS: The methodological approaches cover methods for sound scientific modeling of natural history course data, showing similarity between subgroups, defining, and analyzing multiple endpoints and quantifying the level of evidence in multiple endpoint trials that are often hampered by bias. CONCLUSION: Through its expected results, iSTORE will contribute to the rare diseases research field by providing an approach to better inform about and thus being able to plan a clinical trial. The methodological derivations can be synchronized and transferability will be outlined.

Similarity of competing risks models with constant intensities in an application to clinical healthcare pathways involving prostate cancer surgery.

The recent availability of routine medical data, especially in a university-clinical context, may enable the discovery of typical healthcare pathways, that is, typical temporal sequences of clinical interventions or hospital readmissions. However, such pathways are heterogeneous in a large provider such as a university hospital, and it is important to identify similar care pathways that can still be considered typical pathways. We understand the pathway as a temporal process with possible transitions from a single initial treatment state to hospital readmission of different types, which constitutes a competing risks setting. In this article, we propose a multi-state model-based approach to uncover pathway similarity between two groups of individuals. We describe a new bootstrap procedure for testing the similarity of constant transition intensities from two competing risk models. In a large simulation study, we investigate the performance of our similarity approach with respect to different sample sizes and different similarity thresholds. The studies are motivated by an application from urological clinical routine and we show how the results can be transferred to the application example.

Data Mining in Urology: Understanding Real-world Treatment Pathways for Lower Urinary Tract Systems via Exploration of Big Data.

With an increasing number of novel therapeutic options for lower urinary tract symptoms (LUTS), the spectrum of potential treatment pathways resulting from different combinations of treatment decisions is expanding and evolving. Treatment decisions are frequently made with little or no evidence from randomized controlled trials (RCTs) and thus require evidence from other data sources. Clinical routine data reflect real-world treatment pathways. However, evidence for LUTS from routine data means that heterogeneous pathways need to be simultaneously analyzed for compiling evidence in the absence of RCTs. Statistical multi-state model approaches can provide a powerful framework for achieving this goal. More extensive statistical and methodological efforts in the area of similarity of small data are needed to enable the valid pooling of pathways towards joining evidence. PATIENT SUMMARY: Treatment decisions should rely primarily on evidence from clinical trials. When treatment for which there is limited trial evidence needs to be provided, analysis of results from routine clinical practice can represent valuable complementary evidence, but this requires integration of data from heterogeneous treatment pathways.

Efficient model-based bioequivalence testing.

The classical approach to analyze pharmacokinetic (PK) data in bioequivalence studies aiming to compare two different formulations is to perform noncompartmental analysis (NCA) followed by two one-sided tests (TOST). In this regard, the PK parameters area under the curve (AUC) and $C_{\max}$ are obtained for both treatment groups and their geometric mean ratios are considered. According to current guidelines by the U.S. Food and Drug Administration and the European Medicines Agency, the formulations are declared to be sufficiently similar if the $90\%$ confidence interval for these ratios falls between $0.8$ and $1.25 $. As NCA is not a reliable approach in case of sparse designs, a model-based alternative has already been proposed for the estimation of $\rm AUC$ and $C_{\max}$ using nonlinear mixed effects models. Here we propose another, more powerful test than the TOST and demonstrate its superiority through a simulation study both for NCA and model-based approaches. For products with high variability on PK parameters, this method appears to have closer type I errors to the conventionally accepted significance level of $0.05$, suggesting its potential use in situations where conventional bioequivalence analysis is not applicable.

Testing for similarity of binary efficacy-toxicity responses.

Clinical trials often aim to compare two groups of patients for efficacy and/or toxicity depending on covariates such as dose. Examples include the comparison of populations from different geographic regions or age classes or, alternatively, of different treatment groups. Similarity of these groups can be claimed if the difference in average outcome is below a certain margin over the entire covariate range. In this article, we consider the problem of testing for similarity in the case that efficacy and toxicity are measured as binary outcome variables. We develop a new test for the assessment of similarity of two groups for a single binary endpoint. Our approach is based on estimating the maximal deviation between the curves describing the responses of the two groups, followed by a parametric bootstrap test. Further, using a two-dimensional Gumbel-type model we develop methodology to establish similarity for (correlated) binary efficacy-toxicity outcomes. We investigate the operating characteristics of the proposed methodology by means of a simulation study and present a case study as an illustration.

New Model-Based Bioequivalence Statistical Approaches for Pharmacokinetic Studies with Sparse Sampling.

In traditional pharmacokinetic (PK) bioequivalence analysis, two one-sided tests (TOST) are conducted on the area under the concentration-time curve and the maximal concentration derived using a non-compartmental approach. When rich sampling is unfeasible, a model-based (MB) approach, using nonlinear mixed effect models (NLMEM) is possible. However, MB-TOST using asymptotic standard errors (SE) presents increased type I error when asymptotic conditions do not hold. In this work, we propose three alternative calculations of the SE based on (i) an adaptation to NLMEM of the correction proposed by Gallant, (ii) the a posteriori distribution of the treatment coefficient using the Hamiltonian Monte Carlo algorithm, and (iii) parametric random effects and residual errors bootstrap. We evaluate these approaches by simulations, for two-arms parallel and two-period, two-sequence cross-over design with rich (n = 10) and sparse (n = 3) sampling under the null and the alternative hypotheses, with MB-TOST. All new approaches correct for the inflation of MB-TOST type I error in PK studies with sparse designs. The approach based on the a posteriori distribution appears to be the best compromise between controlled type I errors and computing times. MB-TOST using non-asymptotic SE controls type I error rate better than when using asymptotic SE estimates for bioequivalence on PK studies with sparse sampling.

Equivalence of regression curves sharing common parameters.

In clinical trials, the comparison of two different populations is a common problem. Nonlinear (parametric) regression models are commonly used to describe the relationship between covariates, such as concentration or dose, and a response variable in the two groups. In some situations, it is reasonable to assume some model parameters to be the same, for instance, the placebo effect or the maximum treatment effect. In this paper, we develop a (parametric) bootstrap test to establish the similarity of two regression curves sharing some common parameters. We show by theoretical arguments and by means of a simulation study that the new test controls its significance level and achieves a reasonable power. Moreover, it is demonstrated that under the assumption of common parameters, a considerably more powerful test can be constructed compared with the test that does not use this assumption. Finally, we illustrate the potential applications of the new methodology by a clinical trial example.

Regulatory assessment of drug dissolution profiles comparability via maximum deviation.

In drug development, comparability of dissolution profiles of 2 different formulations is usually assessed using the similarity factor f2 . In practice, the drug dissolution profiles are deemed similar if the f2 exceeds 50, which occurs when a 10% maximum difference in the mean percentage of the dissolved drug at each time point between test and reference formulation is obtained. According to the Guideline on the Investigation of Bioequivalence (CPMP/EWP/QWP/1401/98 Rev. 1/ Corr **) use of the f2 is however restricted by a set of validity conditions. If some of these conditions are not satisfied, the f2 is not considered suitable, and alternative statistical methods are needed. In this article, we propose an inferential framework based on the maximum deviation between curves to test the comparability of drug dissolution profiles. The new methodology is applicable regardless whether the validity criteria of the f2 are met or not. Contrary to the f2 , this approach also integrates the variability of the measurements over time and not only their average. To benchmark our method, we performed simulations informed by 3 real case studies provided by the European Medicines Agency and extracted from dossiers submitted to the Centralised Procedure for Marketing Authorisation Application. In the scenarios of the simulation study, the new method controlled its type I error rate when the maximum deviation was greater than the similarity acceptance limit of 10%. The power exceeded 80% for small values of the maximum deviation, while the test was more conservative for intermediate ones. Our results were also very robust to sampling variations. Based on these positive findings, we encourage applicants to consider the new maximum deviation-based method as a valid alternative to the f2 , especially when the validity criteria of the latter are not met.

Lessons learned from IDeAl - 33 recommendations from the IDeAl-net about design and analysis of small population clinical trials.

BACKGROUND: IDeAl (Integrated designs and analysis of small population clinical trials) is an EU funded project developing new statistical design and analysis methodologies for clinical trials in small population groups. Here we provide an overview of IDeAl findings and give recommendations to applied researchers. METHOD: The description of the findings is broken down by the nine scientific IDeAl work packages and summarizes results from the project's more than 60 publications to date in peer reviewed journals. In addition, we applied text mining to evaluate the publications and the IDeAl work packages' output in relation to the design and analysis terms derived from in the IRDiRC task force report on small population clinical trials. RESULTS: The results are summarized, describing the developments from an applied viewpoint. The main result presented here are 33 practical recommendations drawn from the work, giving researchers a comprehensive guidance to the improved methodology. In particular, the findings will help design and analyse efficient clinical trials in rare diseases with limited number of patients available. We developed a network representation relating the hot topics developed by the IRDiRC task force on small population clinical trials to IDeAl's work as well as relating important methodologies by IDeAl's definition necessary to consider in design and analysis of small-population clinical trials. These network representation establish a new perspective on design and analysis of small-population clinical trials. CONCLUSION: IDeAl has provided a huge number of options to refine the statistical methodology for small-population clinical trials from various perspectives. A total of 33 recommendations developed and related to the work packages help the researcher to design small population clinical trial. The route to improvements is displayed in IDeAl-network representing important statistical methodological skills necessary to design and analysis of small-population clinical trials. The methods are ready for use.

Assessing the similarity of dose response and target doses in two non-overlapping subgroups.

We consider 2 problems of increasing importance in clinical dose finding studies. First, we assess the similarity of 2 non-linear regression models for 2 non-overlapping subgroups of patients over a restricted covariate space. To this end, we derive a confidence interval for the maximum difference between the 2 given models. If this confidence interval excludes the pre-specified equivalence margin, similarity of dose response can be claimed. Second, we address the problem of demonstrating the similarity of 2 target doses for 2 non-overlapping subgroups, using again an approach based on a confidence interval. We illustrate the proposed methods with a real case study and investigate their operating characteristics (coverage probabilities, Type I error rates, power) via simulation.

Optimal designs in regression with correlated errors.

This paper discusses the problem of determining optimal designs for regression models, when the observations are dependent and taken on an interval. A complete solution of this challenging optimal design problem is given for a broad class of regression models and covariance kernels. We propose a class of estimators which are only slightly more complicated than the ordinary least-squares estimators. We then demonstrate that we can design the experiments, such that asymptotically the new estimators achieve the same precision as the best linear unbiased estimator computed for the whole trajectory of the process. As a by-product we derive explicit expressions for the BLUE in the continuous time model and analytic expressions for the optimal designs in a wide class of regression models. We also demonstrate that for a finite number of observations the precision of the proposed procedure, which includes the estimator and design, is very close to the best achievable. The results are illustrated on a few numerical examples.

Optimal designs for comparing curves.

We consider the optimal design problem for a comparison of two regression curves, which is used to establish the similarity between the dose response relationships of two groups. An optimal pair of designs minimizes the width of the confidence band for the difference between the two regression functions. Optimal design theory (equivalence theorems, efficiency bounds) is developed for this non standard design problem and for some commonly used dose response models optimal designs are found explicitly. The results are illustrated in several examples modeling dose response relationships. It is demonstrated that the optimal pair of designs for the comparison of the regression curves is not the pair of the optimal designs for the individual models. In particular it is shown that the use of the optimal designs proposed in this paper instead of commonly used "non-optimal" designs yields a reduction of the width of the confidence band by more than 50%.

Model selection versus model averaging in dose finding studies.

A key objective of Phase II dose finding studies in clinical drug development is to adequately characterize the dose response relationship of a new drug. An important decision is then on the choice of a suitable dose response function to support dose selection for the subsequent Phase III studies. In this paper, we compare different approaches for model selection and model averaging using mathematical properties as well as simulations. We review and illustrate asymptotic properties of model selection criteria and investigate their behavior when changing the sample size but keeping the effect size constant. In a simulation study, we investigate how the various approaches perform in realistically chosen settings. Finally, the different methods are illustrated with a recently conducted Phase II dose finding study in patients with chronic obstructive pulmonary disease. Copyright © 2016 John Wiley & Sons, Ltd.

Dose response signal detection under model uncertainty.

We investigate likelihood ratio contrast tests for dose response signal detection under model uncertainty, when several competing regression models are available to describe the dose response relationship. The proposed approach uses the complete structure of the regression models, but does not require knowledge of the parameters of the competing models. Standard likelihood ratio test theory is applicable in linear models as well as in nonlinear regression models with identifiable parameters. However, for many commonly used nonlinear dose response models the regression parameters are not identifiable under the null hypothesis of no dose response and standard arguments cannot be used to obtain critical values. We thus derive the asymptotic distribution of likelihood ratio contrast tests in regression models with a lack of identifiability and use this result to simulate the quantiles based on Gaussian processes. The new method is illustrated with a real data example and compared to existing procedures using theoretical investigations as well as simulations.

Bayesian T-optimal discriminating designs.

The problem of constructing Bayesian optimal discriminating designs for a class of regression models with respect to the T-optimality criterion introduced by Atkinson and Fedorov (1975a) is considered. It is demonstrated that the discretization of the integral with respect to the prior distribution leads to locally T-optimal discriminating design problems with a large number of model comparisons. Current methodology for the numerical construction of discrimination designs can only deal with a few comparisons, but the discretization of the Bayesian prior easily yields to discrimination design problems for more than 100 competing models. A new efficient method is developed to deal with problems of this type. It combines some features of the classical exchange type algorithm with the gradient methods. Convergence is proved and it is demonstrated that the new method can find Bayesian optimal discriminating designs in situations where all currently available procedures fail.

Designing dose finding studies with an active control for exponential families.

In a recent paper Dette et al. (2014) introduced optimal design problems for dose finding studies with an active control. These authors concentrated on regression models with normal distributed errors (with known variance) and the problem of determining optimal designs for estimating the smallest dose, which achieves the same treatment effect as the active control. This paper discusses the problem of designing active-controlled dose finding studies from a broader perspective. In particular, we consider a general class of optimality criteria and models arising from an exponential family, which are frequently used analyzing count data. We investigate under which circumstances optimal designs for dose finding studies including a placebo can be used to obtain optimal designs for studies with an active control. Optimal designs are constructed for several situations and the differences arising from different distributional assumptions are investigated in detail. In particular, our results are applicable for constructing optimal experimental designs to analyze active-controlled dose finding studies with discrete data, and we illustrate the efficiency of the new optimal designs with two recent examples from our consulting projects.

On the efficiency of two-stage response-adaptive designs.

In this paper, we investigate the efficiency of response-adaptive locally optimum designs. We focus on two-stage adaptive designs, where after the first stage the accrued data are used to determine a locally optimum design for the second stage. On the basis of an explicit expansion of the information matrix, we compare the variance of the maximum likelihood estimates obtained from a two-stage adaptive design and a fixed design without adaptation. For several one-parameter models, we provide explicit expressions for the relative efficiency of these two designs, which is seen to depend sensitively on the statistical problem under investigation. In particular, we show that in non-linear regression models with moderate or large variances the first-stage sample size of an adaptive design should be chosen sufficiently large in order to address variability in the interim parameter estimates. These findings support the results of recent simulation studies conducted to compare adaptive designs in more complex situations. We finally present an application to a real clinical dose-finding trial aiming at the estimation of the smallest dose achieving a certain percentage of the maximum treatment effect by using a three-parameter Emax model.

Optimal designs for composed models in pharmacokinetic-pharmacodynamic experiments.

We consider two frequently used PK/PD models and provide closed form descriptions of locally optimal designs for estimating individual parameters. In a novel way, we use these optimal designs and construct locally standardized maximin optimal designs for estimating any subset of the model parameters of interest. We do this by maximizing the minimal efficiency of the estimates across all relevant parameters so that these optimal designs are less dependent on the individual parameter or parameters of interest. Additionally, robust designs are proposed to further reduce the dependence on the nominal values of the parameters. We compare efficiencies of our proposed optimal designs with locally optimal designs and designs used in four real studies from the literature and show that our proposed designs provide advantages over those used in practice.