Pilot and feasibility studies for pragmatic trials have unique considerations and areas of uncertainty.

BACKGROUND AND OBJECTIVE: Feasibility studies are increasingly being used to support the development of, and investigate uncertainties around, future large-scale trials. The future trial can be designed with either a pragmatic or explanatory mindset. Whereas pragmatic trials aim to inform the choice between different care options and thus, are designed to resemble conditions outside of a clinical trial environment, explanatory trials examine the benefit of a treatment under more controlled conditions. There is existing guidance for designing feasibility studies, but none that explicitly considers the goals of pragmatic designs. We aimed to identify unique areas of uncertainty that are relevant to planning a pragmatic trial. RESULTS: We identified ten relevant domains, partly based on the pragmatic-explanatory continuum indicator summary-2 (PRECIS-2) framework, and describe potential questions of uncertainty within each: intervention development, research ethics, participant identification and eligibility, recruitment of individuals, setting, organization, flexibility of delivery, flexibility of adherence, follow-up, and importance of primary outcome to patients and decision-makers. We present examples to illustrate how uncertainty in these domains might be addressed within a feasibility study. CONCLUSION: Researchers planning a feasibility study in advance of a pragmatic trial should consider feasibility objectives specifically relevant to areas of uncertainty for pragmatic trials.

Transparency of informed consent in pilot and feasibility studies is inadequate: a single-center quality assurance study.

BACKGROUND: Pilot and feasibility studies (PAFS) often have complex objectives aimed at assessing feasibility of conducting a larger study. These may not be clear to participants in pilot studies. METHODS: Here, we aimed to assess the transparency of informed consent in PAFS by investigating whether researchers communicate, through patient information leaflets and consent forms, key features of the studies. We collected this data from original versions of these documents submitted for ethics approval and the final approved documents for PAFS submitted to the Hamilton Integrated Research Ethics Board, Canada. RESULTS: One hundred eighty-four PAFS, submitted for ethics approval from 2004 to 2020, were included, and we found that of the approved consent documents which were provided to participants, 83.2% (153) stated the terms "pilot" or "feasibility" in their title, 12% (22) stated the definition of a pilot/feasibility study, 42.4% (78) of the studies stated their intent to assess feasibility, 19.6% (36) stated the specific feasibility objectives, 1.6% (3) stated the criteria for success of the pilot study, and 0.5% (1) stated all five of these criteria. After ethics review, a small increase in transparency occurred, ranging from 1.6 to 2.8% depending on the criteria. By extracting data from the protocols of the PAFS, we found that 73.9% (136) stated intent to assess feasibility, 71.2% (131) stated specific feasibility objectives, and 33.7% (62) stated criteria for success of the study to lead to a larger study. CONCLUSION: The transparency of informed consent in PAFS is inadequate and needs to be specifically addressed by research ethics guidelines. Research ethics boards and researchers ought to be made aware and mindful of best practices of informed consent in the context of PAFS.

Validation and development of models using clinical, biochemical and ultrasound markers for predicting pre-eclampsia: an individual participant data meta-analysis.

BACKGROUND: Pre-eclampsia is a leading cause of maternal and perinatal mortality and morbidity. Early identification of women at risk is needed to plan management. OBJECTIVES: To assess the performance of existing pre-eclampsia prediction models and to develop and validate models for pre-eclampsia using individual participant data meta-analysis. We also estimated the prognostic value of individual markers. DESIGN: This was an individual participant data meta-analysis of cohort studies. SETTING: Source data from secondary and tertiary care. PREDICTORS: We identified predictors from systematic reviews, and prioritised for importance in an international survey. PRIMARY OUTCOMES: Early-onset (delivery at < 34 weeks' gestation), late-onset (delivery at ≥ 34 weeks' gestation) and any-onset pre-eclampsia. ANALYSIS: We externally validated existing prediction models in UK cohorts and reported their performance in terms of discrimination and calibration. We developed and validated 12 new models based on clinical characteristics, clinical characteristics and biochemical markers, and clinical characteristics and ultrasound markers in the first and second trimesters. We summarised the data set-specific performance of each model using a random-effects meta-analysis. Discrimination was considered promising for C-statistics of ≥ 0.7, and calibration was considered good if the slope was near 1 and calibration-in-the-large was near 0. Heterogeneity was quantified using I2 and τ2. A decision curve analysis was undertaken to determine the clinical utility (net benefit) of the models. We reported the unadjusted prognostic value of individual predictors for pre-eclampsia as odds ratios with 95% confidence and prediction intervals. RESULTS: The International Prediction of Pregnancy Complications network comprised 78 studies (3,570,993 singleton pregnancies) identified from systematic reviews of tests to predict pre-eclampsia. Twenty-four of the 131 published prediction models could be validated in 11 UK cohorts. Summary C-statistics were between 0.6 and 0.7 for most models, and calibration was generally poor owing to large between-study heterogeneity, suggesting model overfitting. The clinical utility of the models varied between showing net harm to showing minimal or no net benefit. The average discrimination for IPPIC models ranged between 0.68 and 0.83. This was highest for the second-trimester clinical characteristics and biochemical markers model to predict early-onset pre-eclampsia, and lowest for the first-trimester clinical characteristics models to predict any pre-eclampsia. Calibration performance was heterogeneous across studies. Net benefit was observed for International Prediction of Pregnancy Complications first and second-trimester clinical characteristics and clinical characteristics and biochemical markers models predicting any pre-eclampsia, when validated in singleton nulliparous women managed in the UK NHS. History of hypertension, parity, smoking, mode of conception, placental growth factor and uterine artery pulsatility index had the strongest unadjusted associations with pre-eclampsia. LIMITATIONS: Variations in study population characteristics, type of predictors reported, too few events in some validation cohorts and the type of measurements contributed to heterogeneity in performance of the International Prediction of Pregnancy Complications models. Some published models were not validated because model predictors were unavailable in the individual participant data. CONCLUSION: For models that could be validated, predictive performance was generally poor across data sets. Although the International Prediction of Pregnancy Complications models show good predictive performance on average, and in the singleton nulliparous population, heterogeneity in calibration performance is likely across settings. FUTURE WORK: Recalibration of model parameters within populations may improve calibration performance. Additional strong predictors need to be identified to improve model performance and consistency. Validation, including examination of calibration heterogeneity, is required for the models we could not validate. STUDY REGISTRATION: This study is registered as PROSPERO CRD42015029349. FUNDING: This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 24, No. 72. See the NIHR Journals Library website for further project information.

WHAT IS THE PROBLEM?: Pre-eclampsia, a condition in pregnancy that results in raised blood pressure and protein in the urine, is a major cause of complications for the mother and baby. WHAT IS NEEDED?: A way of accurately identifying women at high risk of pre-eclampsia to allow clinicians to start preventative interventions such as administering aspirin or frequently monitoring women during pregnancy. WHERE ARE THE RESEARCH GAPS?: Although over 100 tools (models) have been reported worldwide to predict pre-eclampsia, to date their performance in women managed in the UK NHS is unknown. WHAT DID WE PLAN TO DO?: We planned to comprehensively identify all published models that predict the risk of pre-eclampsia occurring at any time during pregnancy and to assess if this prediction is accurate in the UK population. If the existing models did not perform satisfactorily, we aimed to develop new prediction models. WHAT DID WE FIND?: We formed the International Prediction of Pregnancy Complications network, which provided data from a large number of studies (78 studies, 25 countries, 125 researchers, 3,570,993 singleton pregnancies). We were able to assess the performance of 24 out of the 131 models published to predict pre-eclampsia in 11 UK data sets. The models did not accurately predict the risk of pre-eclampsia across all UK data sets, and their performance varied within individual data sets. We developed new prediction models that showed promising performance on average across all data sets, but their ability to correctly identify women who develop pre-eclampsia varied between populations. The models were more clinically useful when used in the care of first-time mothers pregnant with one child, compared to a strategy of treating them all as if they were at high-risk of pre-eclampsia. WHAT DOES THIS MEAN?: Before using the International Prediction of Pregnancy Complications models in various populations, they need to be adjusted for characteristics of the particular population and the setting of application.

Tailored, psychological intervention for anxiety and/or depression in people with chronic obstructive pulmonary disease (COPD), TANDEM (Tailored intervention for ANxiety and DEpression Management in COPD): statistical analysis plan for a randomised controlled trial.

BACKGROUND: The aim of the TANDEM trial is to evaluate whether a tailored, psychological cognitive behavioural approach intervention, which links into, and optimises the effects of routine pulmonary rehabilitation (PR), leads to a reduction in mild/moderate anxiety and/or depression in people with moderate, severe or very severe chronic obstructive pulmonary disease. METHODS AND DESIGN: TANDEM is a multi-centre, two-arm, parallel group, pragmatic, individually randomised controlled, superiority trial including an internal pilot. Participants are randomised to receive either the intervention (a tailored psychological intervention plus usual care including referral to PR) or the control (usual care including referral to PR). The designed randomisation ratio is 1.25:1 in favour of the intervention. The multiple-primary outcomes are participant depression and anxiety at 6 months, measured using the Hospital Anxiety and Depression Scale (HADS) depression and anxiety subscales. RESULTS: This article describes the statistical analysis plan (SAP) for the TANDEM trial. In particular, we describe the general analysis principles, how we will handle missing data, the primary and secondary outcomes and how these will be analysed, sensitivity analyses for the multiple-primary outcomes, and any other analyses and data summaries. The SAP was developed and published prior to completion of follow-up of the last participant. TRIAL REGISTRATION: ISRCTN registry ISRCTN59537391. Registered on 20 March 2017.

Assessing the transparency of informed consent in feasibility and pilot studies: a single-centre quality assurance study protocol.

INTRODUCTION: Pilot/feasibility studies assess the feasibility of conducting a larger study. Although researchers ought to communicate the feasibility objectives to their participants, many research ethics guidelines do not comment on how informed consent applies to pilot studies. It is unclear whether researchers and research ethics boards clearly communicate the purpose of pilot studies to participants consenting.The primary objective of this study is to assess whether pilot/feasibility studies submitted for ethics approval to a research ethics board transparently communicate the purpose of the study to participants through their informed consent practice. A highly transparent consent practice entails the consent documents communicate: (1) the term 'pilot' or 'feasibility' in the title; (2) the definition of a pilot/feasibility study; (3) the primary objectives of the study are to assess feasibility; (4) the specific feasibility objectives; and (5) the criteria for the study to successfully lead to the main study. The secondary objectives are to assess whether there is a difference between submitted and revised versions of the consent documents (revisions are made to obtain research ethics approval), to determine factors associated with transparent consent practices and to assess the consistency with which pilot and feasibility studies assess feasibility outcomes as their primary objectives. METHODS AND ANALYSIS: This is a retrospective review of informed consent information for pilot/feasibility studies submitted to the Hamilton integrated Research Ethics Board, Canada. We will look at submitted and revised consent documents for pilot/feasibility studies submitted over a 14-year period. We will use descriptive statistics to summarise data, reporting results as percentages with 95% CIs, and conduct logistic regression to determine characteristics associated with transparent consent practices. ETHICS AND DISSEMINATION: The study protocol was approved by the Hamilton integrated Research Ethics Board, and the results of this study will be submitted for publication in a peer-reviewed journal.

Quality of reporting of pilot and feasibility cluster randomised trials: a systematic review.

OBJECTIVES: To systematically review the quality of reporting of pilot and feasibility of cluster randomised trials (CRTs). In particular, to assess (1) the number of pilot CRTs conducted between 1 January 2011 and 31 December 2014, (2) whether objectives and methods are appropriate and (3) reporting quality. METHODS: We searched PubMed (2011-2014) for CRTs with 'pilot' or 'feasibility' in the title or abstract; that were assessing some element of feasibility and showing evidence the study was in preparation for a main effectiveness/efficacy trial. Quality assessment criteria were based on the Consolidated Standards of Reporting Trials (CONSORT) extensions for pilot trials and CRTs. RESULTS: Eighteen pilot CRTs were identified. Forty-four per cent did not have feasibility as their primary objective, and many (50%) performed formal hypothesis testing for effectiveness/efficacy despite being underpowered. Most (83%) included 'pilot' or 'feasibility' in the title, and discussed implications for progression from the pilot to the future definitive trial (89%), but fewer reported reasons for the randomised pilot trial (39%), sample size rationale (44%) or progression criteria (17%). Most defined the cluster (100%), and number of clusters randomised (94%), but few reported how the cluster design affected sample size (17%), whether consent was sought from clusters (11%), or who enrolled clusters (17%). CONCLUSIONS: That only 18 pilot CRTs were identified necessitates increased awareness of the importance of conducting and publishing pilot CRTs and improved reporting. Pilot CRTs should primarily be assessing feasibility, avoiding formal hypothesis testing for effectiveness/efficacy and reporting reasons for the pilot, sample size rationale and progression criteria, as well as enrolment of clusters, and how the cluster design affects design aspects. We recommend adherence to the CONSORT extensions for pilot trials and CRTs.

CONSORT 2010 statement: extension to randomised pilot and feasibility trials.

The Consolidated Standards of Reporting Trials (CONSORT) statement is a guideline designed to improve the transparency and quality of the reporting of randomised controlled trials (RCTs). In this article we present an extension to that statement for randomised pilot and feasibility trials conducted in advance of a future definitive RCT. The checklist applies to any randomised study in which a future definitive RCT, or part of it, is conducted on a smaller scale, regardless of its design (eg, cluster, factorial, crossover) or the terms used by authors to describe the study (eg, pilot, feasibility, trial, study). The extension does not directly apply to internal pilot studies built into the design of a main trial, non-randomised pilot and feasibility studies, or phase II studies, but these studies all have some similarities to randomised pilot and feasibility studies and so many of the principles might also apply. The development of the extension was motivated by the growing number of studies described as feasibility or pilot studies and by research that has identified weaknesses in their reporting and conduct. We followed recommended good practice to develop the extension, including carrying out a Delphi survey, holding a consensus meeting and research team meetings, and piloting the checklist. The aims and objectives of pilot and feasibility randomised studies differ from those of other randomised trials. Consequently, although much of the information to be reported in these trials is similar to those in randomised controlled trials (RCTs) assessing effectiveness and efficacy, there are some key differences in the type of information and in the appropriate interpretation of standard CONSORT reporting items. We have retained some of the original CONSORT statement items, but most have been adapted, some removed, and new items added. The new items cover how participants were identified and consent obtained; if applicable, the prespecified criteria used to judge whether or how to proceed with a future definitive RCT; if relevant, other important unintended consequences; implications for progression from pilot to future definitive RCT, including any proposed amendments; and ethical approval or approval by a research review committee confirmed with a reference number. This article includes the 26 item checklist, a separate checklist for the abstract, a template for a CONSORT flowchart for these studies, and an explanation of the changes made and supporting examples. We believe that routine use of this proposed extension to the CONSORT statement will result in improvements in the reporting of pilot trials. Editor's note: In order to encourage its wide dissemination this article is freely accessible on the BMJ and Pilot and Feasibility Studies journal websites.