Alternatives to the P value: connotations of significance.

The statistical significance of a clinical trial analysis result is determined by a mathematical calculation and probability based on null hypothesis significance testing. However, statistical significance does not always align with meaningful clinical effects; thus, assigning clinical relevance to statistical significance is unreasonable. A statistical result incorporating a clinically meaningful difference is a better approach to present statistical significance. Thus, the minimal clinically important difference (MCID), which requires integrating minimum clinically relevant changes from the early stages of research design, has been introduced. As a follow-up to the previous statistical round article on P values, confidence intervals, and effect sizes, in this article, we present hands-on examples of MCID and various effect sizes and discuss the terms statistical significance and clinical relevance, including cautions regarding their use.

Use of biomarkers in clinical trials and future developments that will help identify novel biomarkers.

Engineering new solutions for therapeutic benefit in Amyotrophic Lateral Sclerosis (ALS) has proved a difficult task to accomplish. This is largely the reflection of complexities at multiple levels, that require solutions to improve cost-effectiveness and outcomes. The main obstacle related to the condition's clinical heterogeneity, chiefly the broad difference in survival observed among ALS patients, imposes large populations studies and long follow-up to evaluate any efficacy. The emerging solution is composite clinical and biological parameters enabling prognostic stratification into homogeneous phenotypes for more affordable studies. From a therapeutic development perspective, the choice of a medicinal product requires the availability of treatment-specific biomarkers of target engagement to identify off-target effects based on the compound's putative modality of action. More importantly, there are no established biomarkers of treatment response that can complement clinical outcome measures and support futility and end of treatment analyses of efficacy. Ultimately the onus rests on the development of biomarkers encompassing the unmet needs of clinical trial design, from inclusion to efficacy. These readouts of the pathological process may be used in combination with clinical and paraclinical outcome measured, significantly reducing the time and financial burden of clinical studies. Progress towards a biomarker-driven clinical trial design in ALS has been possible thanks to the accurate detection of neurofilaments and of other immunological mediators in biological fluids with the disease progression, a step change enabling the testing of novel therapeutic agents in a new clinical trial setting. However, further progress remains to be made to find treatment specific target engagement biomarkers along with readouts of treatment response that can be reliably applied to all emerging therapies and clinical studies. Here we will cover the basic notions of biomarker development in ALS clinical trials, the most crucial unanswered questions and the unmet needs in the ALS biomarkers space.

Pharmacy Workload in Clinical Trial Management: A Preliminary Complexity Assessment Tool for Sponsored Oncology and Haematology Trials.

Investigational drug services need to be organised in a structured approach, especially for sites with a large number of ongoing clinical trials. The aim of this study was to develop a tool to assess the complexity of pharmacy involvement in a sponsored oncology clinical trial. Categorisation into ordinal complexity categories was used to assess the complexity of the clinical trials for consistent pharmacy grant applications. The 15 items of the tool were divided into three sections, and individual item scores were agreed upon among four pharmacists with experience in the conduct of clinical trials at two different centres. A final version of the tool, named Pharm-CAT, was approved. The pharmacists were instructed to use Pharm-CAT to assign a score to each new sponsored trial. To determine the cut-offs for the complexity categories, the scores were sorted in ascending order and the cut-offs corresponding to the first and third tertiles of the score distribution were selected. To verify the reproducibility of the results, Pharm-CAT was applied by two pharmacists independently for each trial. Pharm-CAT proved to be user-friendly. Sixty clinical trials were evaluated and a total of 120 scores were recorded. Low-complexity scores ranged from 0 to 19, medium-complexity scores ranged from 20 to 25, and high-complexity scores were 26 or higher. The average score recorded was 22.88 points. Prospective multicentre validation of Pharm-CAT is needed to confirm its applicability.

Categorization of the amyotrophic lateral sclerosis population via the clinical determinant of post-onset ΔFS for study design and medical practice.

The amyotrophic lateral sclerosis (ALS) functional rating scale-revised (ALSFRS-R) has become the most widely utilized measure of disease severity in patients with ALS, with change in ALSFRS-R from baseline being a trusted primary outcome measure in ALS clinical trials. This is despite the scale having several established limitations, and although alternative scales have been proposed, it is unlikely that these will displace ALSFRS-R in the foreseeable future. Here, we discuss the merits of delta FS (ΔFS), the slope or rate of ALSFRS-R decline over time, as a relevant tool for innovative ALS study design, with an as yet untapped potential for optimization of drug effectiveness and patient management. In our view, categorization of the ALS population via the clinical determinant of post-onset ΔFS is an important study design consideration. It serves not only as a critical stratification factor and basis for patient enrichment but also as a tool to explore differences in treatment response across the overall population; thereby, facilitating identification of responder subgroups. Moreover, because post-onset ΔFS is derived from information routinely collected as part of standard patient care and monitoring, it provides a suitable patient selection tool for treating physicians. Overall, post-onset ΔFS is a very attractive enrichment tool that is, can and should be regularly incorporated into ALS trial design.

Assessing readiness to use electronic health record data for outcome ascertainment in clinical trials - A case study.

BACKGROUND: Variable data quality poses a challenge to using electronic health record (EHR) data to ascertain acute clinical outcomes in multi-site clinical trials. Differing EHR platforms and data comprehensiveness across clinical trial sites, especially if patients received care outside of the clinical site's network, can also affect validity of results. Overcoming these challenges requires a structured approach. METHODS: We propose a framework and create a checklist to assess the readiness of clinical sites to contribute EHR data to a clinical trial for the purpose of outcome ascertainment, based on our experience with the Strategies to Reduce Injuries and Develop Confidence in Elders (STRIDE) study, which enrolled 5451 participants in 86 primary care practices across 10 healthcare systems (sites). RESULTS: The site readiness checklist includes assessment of the infrastructure (i.e., size and structure of the site's healthcare system or clinical network), data procurement (i.e., quality of the data), and cost of obtaining study data. The checklist emphasizes the importance of understanding how data are captured and integrated across a site's catchment area and having a protocol in place for data procurement to ensure consistent and uniform extraction across each site. CONCLUSIONS: We suggest rigorous, prospective vetting of the data quality and infrastructure of each clinical site before launching a multi-site trial dependent on EHR data. The proposed checklist serves as a guiding tool to help investigators ensure robust and unbiased data capture for their clinical trials. ORIGINAL TRIAL REGISTRATION NUMBER: NCT02475850.

Developing an international consensus Reporting guideline for intervention Fidelity in Non-Drug, non-surgical trials: The ReFiND protocol.

BACKGROUND: Inadequate reporting of fidelity to interventions in trials limits the transparency and interpretation of trial findings. Despite this, most trials of non-drug, non-surgical interventions lack comprehensive reporting of fidelity. If fidelity is poorly reported, it is unclear which intervention components were tested or implemented within the trial, which also hinders research reproducibility. This protocol describes the development process of a reporting guideline for fidelity of non-drug, non-surgical interventions (ReFiND) in the context of trials. METHODS: The ReFiND guideline will be developed in six stages. Stage one: a guideline development group has been formed to oversee the guideline methodology. Stage two: a scoping review will be conducted to identify and summarize existing guidance documents on the fidelity of non-drug, non-surgical interventions. Stage three: a Delphi study will be conducted to reach consensus on reporting items. Stage four: a consensus meeting will be held to consolidate the reporting items and discuss the wording and structure of the guideline. Stage five: a guidance statement, an elaboration and explanation document, and a reporting checklist will be developed. Stage six: different strategies will be used to disseminate and implement the ReFiND guideline. DISCUSSION: The ReFiND guideline will provide a set of items developed through international consensus to improve the reporting of intervention fidelity in trials of non-drug, non-surgical interventions. This reporting guideline will enhance transparency and reproducibility in future non-drug, non-surgical intervention research.

"Reality is frequently inaccurate" A case study examining the whens and whys of post-live database changes in a UK clinical trials unit *Douglas Adams.

INTRODUCTION: Accurately estimating the costs of clinical trials is challenging. There is currently no reference class data to allow researchers to understand the potential costs associated with database change management in clinical trials. METHODS: We used a case-based approach, summarising post-live changes in eleven clinical trial databases managed by Sheffield Clinical Trials Research Unit. We reviewed the database specifications for each trial and summarised the number of changes, change type, change category, and timing of changes. We pooled our experiences and made observations in relation to key themes. RESULTS: Median total number of changes across the eleven trials was 71 (range 40-155) and median number of changes per study week was 0.48 (range 0.32-1.34). The most common change type was modification (median 39, range 20-90), followed by additions (median 32, range 18-55), then deletions (median 7, range 1-12). In our sample, changes were more common in the first half of the trial's lifespan, regardless of its overall duration. Trials which saw continuous changes seemed more likely to be external pilots or trials in areas where the trial team was either less experienced overall or within the particular therapeutic area. CONCLUSIONS: Researchers should plan trials with the expectation that clinical trial databases will require changes within the life of the trial, particularly in the early stages or with a less experienced trial team. More research is required to understand potential differences between clinical trial units and database types.

Considerations for biomarker strategies in clinical trials investigating tau-targeting therapeutics for Alzheimer's disease.

The use of biomarker-led clinical trial designs has been transformative for investigating amyloid-targeting therapies for Alzheimer's disease (AD). The designs have ensured the correct selection of patients on these trials, supported target engagement and have been used to support claims of disease modification and clinical efficacy. Ultimately, this has recently led to approval of disease-modifying, amyloid-targeting therapies for AD; something that should be noted for clinical trials investigating tau-targeting therapies for AD. There is a clear overlap of the purpose of biomarker use at each stage of clinical development between amyloid-targeting and tau-targeting clinical trials. However, there are differences within the potential context of use and interpretation for some biomarkers in particular measurements of amyloid and utility of soluble, phosphorylated tau biomarkers. Given the complexities of tau in health and disease, it is paramount that therapies target disease-relevant tau and, in parallel, appropriate assays of target engagement are developed. Tau positron emission tomography, fluid biomarkers reflecting tau pathology and downstream measures of neurodegeneration will be important both for participant recruitment and for monitoring disease-modification in tau-targeting clinical trials. Bespoke design of biomarker strategies and interpretations for different modalities and tau-based targets should also be considered.

How successful are AI-discovered drugs in clinical trials? A first analysis and emerging lessons.

AI techniques are making inroads into the field of drug discovery. As a result, a growing number of drugs and vaccines have been discovered using AI. However, questions remain about the success of these molecules in clinical trials. To address these questions, we conducted a first analysis of the clinical pipelines of AI-native Biotech companies. In Phase I we find AI-discovered molecules have an 80-90% success rate, substantially higher than historic industry averages. This suggests, we argue, that AI is highly capable of designing or identifying molecules with drug-like properties. In Phase II the success rate is ∼40%, albeit on a limited sample size, comparable to historic industry averages. Our findings highlight early signs of the clinical potential of AI-discovered molecules.

Application of the Estimand Framework to Anesthesia Trials.

SUMMARY: Events occurring after randomization, such as use of rescue medication, treatment discontinuation, or death, are common in randomized trials. These events can change either the existence or interpretation of the outcome of interest. However, appropriate handling of these intercurrent events is often unclear. The International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use (ICH) E9(R1) addendum introduced the estimand framework, which aligns trial objectives with the design, conduct, statistical analysis, and interpretation of results. This article describes how the estimand framework can be used in anesthesia trials to precisely define the treatment effect to be estimated, key attributes of an estimand, common intercurrent events in anesthesia trials with strategies for handling them, and use of the estimand framework in a hypothetical anesthesia trial on postoperative delirium. When planning anesthesia trials, clearly defining the estimand is vital to ensure that what is being estimated is clearly understood, is clinically relevant, and helps answer the clinical questions of interest.

Integrating qualitative research within a clinical trials unit: developing strategies and understanding their implementation in contexts.

BACKGROUND/AIMS: The value of using qualitative methods within clinical trials is widely recognised. How qualitative research is integrated within trials units to achieve this is less clear. This paper describes the process through which qualitative research has been integrated within Cardiff University's Centre for Trials Research (CTR) in Wales, UK. We highlight facilitators of, and challenges to, integration. METHODS: We held group discussions on the work of the Qualitative Research Group (QRG) within CTR. The content of these discussions, materials for a presentation in CTR, and documents relating to the development of the QRG were interpreted at a workshop attended by group members. Normalisation Process Theory (NPT) was used to structure analysis. A writing group prepared a document for input from members of CTR, forming the basis of this paper. RESULTS: Actions to integrate qualitative research comprised: its inclusion in Centre strategies; formation of a QRG with dedicated funding/roles; embedding of qualitative research within operating systems; capacity building/training; monitoring opportunities to include qualitative methods in studies; maximising the quality of qualitative research and developing methodological innovation. Facilitators of these actions included: the influence of the broader methodological landscape within trial/study design and its promotion of the value of qualitative research; and close physical proximity of CTR qualitative staff/students allowing sharing of methodological approaches. Introduction of innovative qualitative methods generated interest among other staff groups. Challenges included: pressure to under-resource qualitative components of research, preference for a statistical stance historically in some research areas and funding structures, and difficulties faced by qualitative researchers carving out individual academic profiles when working across trials/studies. CONCLUSIONS: Given that CTUs are pivotal to the design and conduct of RCTs and related study types across multiple disciplines, integrating qualitative research into trials units is crucial if its contribution is to be fully realised. We have made explicit one trials unit's experience of embedding qualitative research and present this to open dialogue on ways to operationalise and optimise qualitative research in trials. NPT provides a valuable framework with which to theorise these processes, including the importance of sense-making and legitimisation when introducing new practices within organisations.

Approaches and experiences implementing remote, electronic consent at the Leeds Clinical Trials Research Unit.

BACKGROUND: Use of electronic methods to support informed consent ('eConsent') is increasingly popular in clinical research. This commentary reports the approach taken to implement electronic consent methods and subsequent experiences from a range of studies at the Leeds Clinical Trials Research Unit (CTRU), a large clinical trials unit in the UK. MAIN TEXT: We implemented a remote eConsent process using the REDCap platform. The process can be used in trials of investigational medicinal products and other intervention types or research designs. Our standard eConsent system focuses on documenting informed consent, with other aspects of consent (e.g. providing information to potential participants and a recruiter discussing the study with each potential participant) occurring outside the system, though trial teams can use electronic methods for these activities where they have ethical approval. Our overall process includes a verbal consent step prior to confidential information being entered onto REDCap and an identity verification step in line with regulator guidance. We considered the regulatory requirements around the system's generation of source documents, how to ensure data protection standards were upheld and how to monitor informed consent within the system. We present four eConsent case studies from the CTRU: two randomised clinical trials and two other health research studies. These illustrate the ways eConsent can be implemented, and lessons learned, including about differences in uptake. CONCLUSIONS: We successfully implemented a remote eConsent process at the CTRU across multiple studies. Our case studies highlight benefits of study participants being able to give consent without having to be present at the study site. This may better align with patient preferences and trial site needs and therefore improve recruitment and resilience against external shocks (such as pandemics). Variation in uptake of eConsent may be influenced more by site-level factors than patient preferences, which may not align well with the aspiration towards patient-centred research. Our current process has some limitations, including the provision of all consent-related text in more than one language, and scalability of implementing more than one consent form version at a time. We consider how enhancements in CTRU processes, or external developments, might affect our approach.

Using Bayesian statistics in confirmatory clinical trials in the regulatory setting: a tutorial review.

Bayesian statistics plays a pivotal role in advancing medical science by enabling healthcare companies, regulators, and stakeholders to assess the safety and efficacy of new treatments, interventions, and medical procedures. The Bayesian framework offers a unique advantage over the classical framework, especially when incorporating prior information into a new trial with quality external data, such as historical data or another source of co-data. In recent years, there has been a significant increase in regulatory submissions using Bayesian statistics due to its flexibility and ability to provide valuable insights for decision-making, addressing the modern complexity of clinical trials where frequentist trials are inadequate. For regulatory submissions, companies often need to consider the frequentist operating characteristics of the Bayesian analysis strategy, regardless of the design complexity. In particular, the focus is on the frequentist type I error rate and power for all realistic alternatives. This tutorial review aims to provide a comprehensive overview of the use of Bayesian statistics in sample size determination, control of type I error rate, multiplicity adjustments, external data borrowing, etc., in the regulatory environment of clinical trials. Fundamental concepts of Bayesian sample size determination and illustrative examples are provided to serve as a valuable resource for researchers, clinicians, and statisticians seeking to develop more complex and innovative designs.

Upstrapping to determine futility: predicting future outcomes nonparametrically from past data.

BACKGROUND: Clinical trials often involve some form of interim monitoring to determine futility before planned trial completion. While many options for interim monitoring exist (e.g., alpha-spending, conditional power), nonparametric based interim monitoring methods are also needed to account for more complex trial designs and analyses. The upstrap is one recently proposed nonparametric method that may be applied for interim monitoring. METHODS: Upstrapping is motivated by the case resampling bootstrap and involves repeatedly sampling with replacement from the interim data to simulate thousands of fully enrolled trials. The p-value is calculated for each upstrapped trial and the proportion of upstrapped trials for which the p-value criteria are met is compared with a pre-specified decision threshold. To evaluate the potential utility for upstrapping as a form of interim futility monitoring, we conducted a simulation study considering different sample sizes with several different proposed calibration strategies for the upstrap. We first compared trial rejection rates across a selection of threshold combinations to validate the upstrapping method. Then, we applied upstrapping methods to simulated clinical trial data, directly comparing their performance with more traditional alpha-spending and conditional power interim monitoring methods for futility. RESULTS: The method validation demonstrated that upstrapping is much more likely to find evidence of futility in the null scenario than the alternative across a variety of simulations settings. Our three proposed approaches for calibration of the upstrap had different strengths depending on the stopping rules used. Compared to O'Brien-Fleming group sequential methods, upstrapped approaches had type I error rates that differed by at most 1.7% and expected sample size was 2-22% lower in the null scenario, while in the alternative scenario power fluctuated between 15.7% lower and 0.2% higher and expected sample size was 0-15% lower. CONCLUSIONS: In this proof-of-concept simulation study, we evaluated the potential for upstrapping as a resampling-based method for futility monitoring in clinical trials. The trade-offs in expected sample size, power, and type I error rate control indicate that the upstrap can be calibrated to implement futility monitoring with varying degrees of aggressiveness and that performance similarities can be identified relative to considered alpha-spending and conditional power futility monitoring methods.

Improvement of assessment in surrogate endpoint and safety outcome of single-arm trials for anticancer drugs.

INTRODUCTION: Single-arm trials (SATs) and surrogate endpoints were adopted as pivotal evidence for accelerated approval of anticancer drugs for more than 30 years. However, concerns regarding clinical evidence quality in trials, particularly in the SATs of anticancer drugs have increasingly been raised. SAT may not always provide strong evidence due to the lack of control and endpoint of overall survival that is typically present in randomized controlled trials. AREAS COVERED: Clinical trial endpoint adjudication is a crucial factor in surrogate outcome measurement to ensure the data quality of the clinical trial of anticancer drugs. In this review, we systematically discuss the characteristics of adjudications in assessments in surrogate endpoint and safety outcome respectively, which are essential for ensuring reliable and transparent outcomes. Endpoint adjudication effectively reduces potential bias and mitigates variance that may be introduced by investigators when analyzing the medical records for the surrogate endpoints. We analyze the advantages and disadvantages of each type of adjudicator and provide a summary of the roles of adjudicators. EXPERT OPINION: By suggestion of improving data reliability and transparency in pivotal trials, this review aims to supply a strategy for better clinical investigation for anticancer drugs, ultimately leading to better patient outcomes.

The role of quality of life data as an endpoint for collecting real-world evidence within geroscience clinical trials.

With geroscience research evolving at a fast pace, the need arises for human randomized controlled trials to assess the efficacy of geroprotective interventions to prevent age-related adverse outcomes, disease, and mortality in normative aging cohorts. However, to confirm efficacy requires a long-term and costly approach as time to the event of morbidity and mortality can be decades. While this could be circumvented using sensitive biomarkers of aging, current molecular, physiological, and digital endpoints require further validation. In this review, we discuss how collecting real-world evidence (RWE) by obtaining health data that is amenable for collection from large heterogeneous populations in a real-world setting can help speed up validation of geroprotective interventions. Further, we propose inclusion of quality of life (QoL) data as a biomarker of aging and candidate endpoint for geroscience clinical trials to aid in distinguishing healthy from unhealthy aging. We highlight how QoL assays can aid in accelerating data collection in studies gathering RWE on the geroprotective effects of repurposed drugs to support utilization within healthy longevity medicine. Finally, we summarize key metrics to consider when implementing QoL assays in studies, and present the short-form 36 (SF-36) as the most well-suited candidate endpoint.

Reporting of interventional clinical trial results in an academic center: a survey of completed studies.

BACKGROUND: The dissemination of clinical trial results is an important scientific and ethical endeavour. This survey of completed interventional studies in a French academic center describes their reporting status. METHODS: We explored all interventional studies sponsored by Rennes University Hospital identified on the French Open Science Monitor which tracks trials registered on EUCTR or clinicaltrials.gov, and provides an automatic assessment of the reporting of results. For each study, we ascertained the actual reporting of results using systematic searches on the hospital internal database, bibliographic databases (Google Scholar, PubMed), and by contacting all principal investigators (PIs). We describe several features (including total budget and numbers of trial participants) of the studies that did not report any results. RESULTS: The French Open Science Monitor identified 93 interventional studies, among which 10 (11%) reported results. In contrast, our survey identified 36 studies (39%) reporting primary analysis results and an additional 18 (19%) reporting results for secondary analyses (without results for their primary analysis). The overall budget for studies that did not report any results was estimated to be €5,051,253 for a total of 6,735 trial participants. The most frequent reasons for the absence of results reported by PIs were lack of time for 18 (42%), and logistic difficulties (e.g. delay in obtaining results or another blocking factor) for 12 (28%). An association was found between non-publication and negative results (adjusted Odds Ratio = 4.70, 95% Confidence Interval [1.67;14.11]). CONCLUSIONS: Even allowing for the fact that automatic searches underestimate the number of studies with published results, the level of reporting was disappointingly low. This amounts to a waste of trial participants' implication and money. Corrective actions are needed. TRIAL REGISTRATION: https://osf.io/q5hcs.