Raising Public Awareness of Clinical Trials: Development of Messages for a National Health Communication Campaign.

Clinical trials are essential for developing new and effective treatments and improving patient quality of life; however, many trials cannot answer their primary research questions because they fall short of their recruitment goals. This article reports the results of formative research conducted in two populations, the public and primary care physicians, to identify messages that may raise awareness and increase interest in clinical trials and be used in a national communication campaign. Results suggested that participants were primarily motivated to participate in clinical trials out of a self-interest to help themselves first. Messages illustrated that current treatments were tested via clinical trials, helped normalize trials as routine practices, and reduced concerns over trying something new first. Participants wanted messages that portray trials as state-of-the-art choices that offer some hope, show people like themselves, and are described in a clear, concise manner with actionable steps for them to take. The study revealed some differences in message salience, with healthy audiences exhibiting lower levels of interest. Our results suggest that targeted messages are needed, and that communication with primary health-care providers is an important and necessary component in raising patient awareness of the importance of clinical trials.

A Novel Approach to Measuring Efficiency of Scientific Research Projects: Data Envelopment Analysis.

PURPOSE: Measuring the efficiency of resource allocation for the conduct of scientific projects in medical research is difficult due to, among other factors, the heterogeneity of resources supplied (e.g., dollars or FTEs) and outcomes expected (e.g., grants, publications). While this is an issue in medical science, it has been approached successfully in other fields by using data envelopment analysis (DEA). DEA has a number of advantages over other techniques as it simultaneously uses multiple heterogeneous inputs and outputs to determine which projects are performing most efficiently, referred to as being at the efficiency frontier, when compared to others in the data set. METHOD: This research uses DEA for the evaluation of supported translational science projects by the Oregon Clinical and Translational Research Institute (OCTRI), a NCATS Clinical & Translational Science Award (CTSA) recipient. RESULTS: These results suggest that the primary determinate of overall project efficiency at OCTRI is the amount of funding, with smaller amounts of funding providing more efficiency than larger funding amounts. CONCLUSION: These results, and the use of DEA, highlight both the success of using this technique in helping determine medical research efficiency and those factors to consider when distributing funds for new projects at CTSAs.

The National Cancer Institute-American Society of Clinical Oncology Cancer Trial Accrual Symposium: summary and recommendations.

INTRODUCTION: Many challenges to clinical trial accrual exist, resulting in studies with inadequate enrollment and potentially delaying answers to important scientific and clinical questions. METHODS: The National Cancer Institute (NCI) and the American Society of Clinical Oncology (ASCO) cosponsored the Cancer Trial Accrual Symposium: Science and Solutions on April 29-30, 2010 to examine the state of accrual science related to patient/community, physician/provider, and site/organizational influences, and identify new interventions to facilitate clinical trial enrollment. The symposium featured breakout sessions, plenary sessions, and a poster session including 100 abstracts. Among the 358 attendees were clinical investigators, researchers of accrual strategies, research administrators, nurses, research coordinators, patient advocates, and educators. A bibliography of the accrual literature in these three major areas was provided to participants in advance of the meeting. After the symposium, the literature in these areas was revisited to determine if the symposium recommendations remained relevant within the context of the current literature. RESULTS: Few rigorously conducted studies have tested interventions to address challenges to clinical trials accrual. Attendees developed recommendations for improving accrual and identified priority areas for future accrual research at the patient/community, physician/provider, and site/organizational levels. Current literature continues to support the symposium recommendations. CONCLUSIONS: A combination of approaches addressing both the multifactorial nature of accrual challenges and the characteristics of the target population may be needed to improve accrual to cancer clinical trials. Recommendations for best practices and for future research developed from the symposium are provided.

Estimating return on investment in translational research: methods and protocols.

Assessing the value of clinical and translational research funding on accelerating the translation of scientific knowledge is a fundamental issue faced by the National Institutes of Health (NIH) and its Clinical and Translational Awards (CTSAs). To address this issue, the authors propose a model for measuring the return on investment (ROI) of one key CTSA program, the clinical research unit (CRU). By estimating the economic and social inputs and outputs of this program, this model produces multiple levels of ROI: investigator, program, and institutional estimates. A methodology, or evaluation protocol, is proposed to assess the value of this CTSA function, with specific objectives, methods, descriptions of the data to be collected, and how data are to be filtered, analyzed, and evaluated. This article provides an approach CTSAs could use to assess the economic and social returns on NIH and institutional investments in these critical activities.

A "three-plus-one" evaluation model for clinical research management.

Clinical research management (CRM) is a critical resource for the management of clinical trials and it requires proper evaluation. This article advances a model of evaluation that has three local levels, plus one global level, for evaluating the value of CRM. The primary level for evaluation is that of the study or processes level. The managerial or aggregate level concerns management of the portfolio of trials under the control of the CRM office. The third, often overlooked level of evaluation, is the strategic level, whose goal is encapsulated in the phrase, "doing the right trials, while doing trials right." The global ("plus one") evaluation level concerns the need to evaluate the ever-increasing number of multi-institutional and multinational studies. As there are host of evaluation metrics, this article provides representative examples of metrics at each level and provides methods that can aid in the selecting appropriate metrics for an organization.

Characteristics of oncology clinical trials: insights from a systematic analysis of ClinicalTrials.gov.

IMPORTANCE: Clinical trials are essential to cancer care, and data about the current state of research in oncology are needed to develop benchmarks and set the stage for improvement. OBJECTIVE: To perform a comprehensive analysis of the national oncology clinical research portfolio. DESIGN: All interventional clinical studies registered on ClinicalTrials.gov between October 2007 and September 2010 were identified using Medical Subject Heading terms and submitted conditions. They were reviewed to validate classification, subcategorized by cancer type, and stratified by design characteristics to facilitate comparison across cancer types and with other specialties. RESULTS: Of 40 970 interventional studies registered between October 2007 and September 2010, a total of 8942 (21.8%) focused on oncology. Compared with other specialties, oncology trials were more likely to be single arm (62.3% vs 23.8%; P < .001), open label (87.8% vs 47.3%; P < .001), and nonrandomized (63.9% vs 22.7%; P < .001). There was moderate but significant correlation between number of trials conducted by cancer type and associated incidence and mortality (Spearman rank correlation coefficient, 0.56 [P = .04] and 0.77 [P = .001], respectively). More than one-third of all oncology trials were conducted solely outside North America. CONCLUSIONS AND RELEVANCE: There are significant variations between clinical trials in oncology and other diseases, as well as among trials within oncology. The differences must be better understood to improve both the impact of cancer research on clinical practice and the use of constrained resources.

A virtual national laboratory for reengineering clinical translational science.

Clinical research is burdened by inefficiencies and complexities, with a poor record of trial completion, none of which is desirable. The Clinical and Translational Science Award (CTSA) Consortium, including more than 60 clinical research institutions, supports a unified national effort to become, in effect, a virtual national laboratory designed to identify, implement, evaluate, and extend process improvements across all parts of clinical research, from conception to completion. If adequately supported by academic health centers, industry, and funding agencies, the Consortium could become a test bed for improvements that can dramatically reduce wasteful complexity, thus increasing the likelihood of clinical trial completion.

The importance of doing trials right while doing the right trials.

Effort is being expended in investigating efficiency measures (i.e., doing trials right) through achievement of accrual and endpoint goals for clinical trials. It is time to assess the impact of such trials on meeting the critical needs of cancer patients by establishing effectiveness measures (i.e., doing the right trials).

The prevalence and economic impact of low-enrolling clinical studies at an academic medical center.

PURPOSE: The authors assessed the prevalence and associated economic impact of low-enrolling clinical studies at a single academic medical center. METHOD: The authors examined all clinical studies receiving institutional review board (IRB) review between FY2006 and FY2009 at Oregon Health & Science University (OHSU) for recruitment performance and analyzed them by type of IRB review (full-board, exempt, expedited), funding mechanism, and academic unit. A low-enrolling study included those with zero or one participant at the time of study termination. The authors calculated the costs associated with IRB review, financial setup, contract negotiation, and department study start-up activities and the total economic impact on OHSU of low-enrolling studies for FY2009. RESULTS: A total of 837 clinical studies were terminated during the study period, 260 (31.1%) of which were low-enrolling. A greater proportion of low-enrolling studies were government funded than industry funded (P=.006). The authors found significant differences among the various academic units with respect to percentages of low-enrolling studies (from 10% to 67%). The uncompensated economic impact of low-enrolling studies was conservatively estimated to be nearly $1 million for FY2009. CONCLUSIONS: A substantial proportion of clinical studies incurred high institutional and departmental expense but resulted in little scientific benefit. Although a certain percentage of low-enrolling studies can be expected in any research organization, the overall number of such studies must be managed to reduce the aggregate costs of conducting research and to maximize research opportunities. Effective, proactive interventions are needed to address the prevalence and impact of low enrollment.

Predicting accrual achievement: monitoring accrual milestones of NCI-CTEP-sponsored clinical trials.

PURPOSE: The need to increase the number oncology clinical trials with sufficient enrollments is a well-known issue, particularly for trials targeting therapeutic applications. It is critical to identify early predictors of eventual study accrual achievement. EXPERIMENTAL DESIGN: All nonpediatric phase I, I/II, II, and III therapeutic studies supported by the National Cancer Institute Cancer Therapy Evaluation Program (NCI-CTEP) between 2000 and 2007 (n = 764) were analyzed for accrual performance. Accrual achievement is defined as those enrolling 100% or more of the stated minimum accrual goal at the time of trial closure. Two accrual milestones were analyzed per trial: time to first patient enrollment and expected time to accrual goal. Multivariate logistic regression analysis was used to calculate the OR with respect to the likelihood of clinical trial accrual achievement. RESULTS: A total of 81.5% (n = 623) of the trials did not achieve the projected accrual goals within the anticipated accruing period. Furthermore, 37.2% (n = 284) of trials failed to achieve the minimum projected accrual at study closure regardless of time the trial was open. Trials that accrue the first enrollment beyond 2 months (n = 379, 49.6%) are significantly less likely to achieve the accrual performance than those trials that enroll patients under 2 months (OR: 0.637, 95% CI: 0.464-0.875, P = 0.005). Of the studies that are open beyond the anticipated enrollment period (n = 603), those do not achieve at least 60.0% of the projected minimum accrual (n = 391, 64.8%) have a significantly less likelihood of achieving final accruals by study closure (OR: 0.190, 95% CI: 0.055-0.652, P = 0.008). CONCLUSIONS: The time to first patient enrollment and expected time to accrual goal are shown to be valid measures to evaluate the likelihood of achieving the minimum projected accrual.

AccrualNet: Addressing Low Accrual Via a Knowledge-Based, Community of Practice Platform.

PURPOSE: Present the design and initial evaluation of a unique, Web-enabled platform for the development of a community of practice around issues of oncology clinical trial accrual. METHODS: The National Cancer Institute (NCI) conducted research with oncology professionals to identify unmet clinical trial accrual needs in the field. In response, a comprehensive platform for accrual resources, AccrualNet, was created by using an agile development process, storyboarding, and user testing. Literature and resource searches identified relevant content to populate the site. Descriptive statistics were tracked for resource and site usage. Use cases were defined to support implementation. RESULTS: ACCRUALNET HAS FIVE LEVELS: (1) clinical trial macrostages (prestudy, active study, and poststudy); (2) substages (developing a protocol, selecting a trial, preparing to open, enrolling patients, managing the trial, retaining participants, and lessons learned); (3) strategies for each substage; (4) multiple activities for each strategy; and (5) multiple resources for each activity. Since its launch, AccrualNet has had more than 45,000 page views, with the Tools & Resources, Conversations, and Training sections being the most viewed. Total resources have increased 69%, to 496 items. Analysis of articles in the site reveals that 22% are from two journals and 46% of the journals supplied a single article. To date, there are 29 conversations with 43 posts. Four use cases are discussed. CONCLUSION: AccrualNet represents a unique, centralized comprehensive-solution platform to systematically capture accrual knowledge for all stages of a clinical trial. It is designed to foster a community of practice by encouraging users to share additional strategies, resources, and ideas.

Phase III clinical trial development: a process of chutes and ladders.

PURPOSE: The Institute of Medicine report on cooperative groups and the National Cancer Institute (NCI) report from the Operational Efficiency Working Group both recommend changes to the processes for opening a clinical trial. This article provides evidence for the need for such changes by completing the first comprehensive review of all the time and steps required to open a phase III oncology clinical trial and discusses the effect of time to protocol activation on subject accrual. METHODS: The Dilts and Sandler method was used at four cancer centers, two cooperative groups, and the NCI Cancer Therapy Evaluation Program. Accrual data were also collected. RESULTS: Opening a phase III cooperative group therapeutic trial requires 769 steps, 36 approvals, and a median of approximately 2.5 years from formal concept review to study opening. Time to activation at one group ranged from 435 to 1,604 days, and time to open at one cancer center ranged from 21 to 836 days. At centers, group trials are significantly more likely to have zero accruals (38.8%) than nongroup trials (20.6%; P < 0.0001). Of the closed NCI Cancer Therapy Evaluation Program-approved phase III clinical trials from 2000 to 2007, 39.1% resulted in <21 accruals. CONCLUSIONS: The length, variability, and low accrual results demonstrate the need for the NCI clinical trials system to be reengineered. Improvements will be of only limited effectiveness if done in isolation; there is a need to return to the collaborative spirit with all parties creating an efficient and effective system. Recommendations put forth by the Institute of Medicine and Operational Efficiency Working Group reports, if implemented, will aid this renewal.

A sense of urgency: Evaluating the link between clinical trial development time and the accrual performance of cancer therapy evaluation program (NCI-CTEP) sponsored studies.

PURPOSE: Postactivation barriers to oncology clinical trial accruals are well documented; however, potential barriers prior to trial opening are not. We investigate one such barrier: trial development time. EXPERIMENTAL DESIGN: National Cancer Institute Cancer Therapy Evaluation Program (CTEP)-sponsored trials for all therapeutic, nonpediatric phase I, I/II, II, and III studies activated between 2000 and 2004 were investigated for an 8-year period (n = 419). Successful trials were those achieving 100% of minimum accrual goal. Time to open a study was the calendar time from initial CTEP submission to trial activation. Multivariate logistic regression analysis was used to calculate unadjusted and adjusted odds ratios (OR), controlling for study phase and size of expected accruals. RESULTS: Among the CTEP-approved oncology trials, 37.9% (n = 221) failed to attain the minimum accrual goals, with 70.8% (n = 14) of phase III trials resulting in poor accrual. A total of 16,474 patients (42.5% of accruals) accrued to those studies were unable to achieve the projected minimum accrual goal. Trials requiring less than 12 months of development were significantly more likely to achieve accrual goals (OR, 2.15; 95% confidence interval, 1.29-3.57, P = 0.003) than trials with the median development times of 12 to 18 months. Trials requiring a development time of greater than 24 months were significantly less likely to achieve accrual goals (OR, 0.40; 95% confidence interval, 0.20-0.78; P = 0.011) than trials with the median development time. CONCLUSIONS: A large percentage of oncology clinical trials do not achieve minimum projected accruals. Trial development time appears to be one important predictor of the likelihood of successfully achieving the minimum accrual goals.

Openness of patients' reporting with use of electronic records: psychiatric clinicians' views.

OBJECTIVES: Improvements in electronic health record (EHR) system development will require an understanding of psychiatric clinicians' views on EHR system acceptability, including effects on psychotherapy communications, data-recording behaviors, data accessibility versus security and privacy, data quality and clarity, communications with medical colleagues, and stigma. DESIGN: Multidisciplinary development of a survey instrument targeting psychiatric clinicians who recently switched to EHR system use, focus group testing, data analysis, and data reliability testing. MEASUREMENTS: Survey of 120 university-based, outpatient mental health clinicians, with 56 (47%) responding, conducted 18 months after transition from a paper to an EHR system. RESULTS: Factor analysis gave nine item groupings that overlapped strongly with five a priori domains. Respondents both praised and criticized the EHR system. A strong majority (81%) felt that open therapeutic communications were preserved. Regarding data quality, content, and privacy, clinicians (63%) were less willing to record highly confidential information and disagreed (83%) with including their own psychiatric records among routinely accessed EHR systems. LIMITATIONS: single time point; single academic medical center clinic setting; modest sample size; lack of prior instrument validation; survey conducted in 2005. CONCLUSIONS: In an academic medical center clinic, the presence of electronic records was not seen as a dramatic impediment to therapeutic communications. Concerns regarding privacy and data security were significant, and may contribute to reluctances to adopt electronic records in other settings. Further study of clinicians' views and use patterns may be helpful in guiding development and deployment of electronic records systems.

Steps and time to process clinical trials at the Cancer Therapy Evaluation Program.

PURPOSE: To examine the processes and document the calendar time required for the National Cancer Institute's Cancer Therapy Evaluation Program (CTEP) and Central Institutional Review Board (CIRB) to evaluate and approve phase III clinical trials. METHODS: Process steps were documented by (1) interviewing CTEP and CIRB staff regarding the steps required to activate a trial from initial concept submission to trial activation by a cooperative group, (2) reviewing standard operating procedures, and (3) inspecting trial records and documents for selected trials to identify any additional steps. Calendar time was collected from initial concept submission to activation using retrospective data from the CTEP Protocol and Information Office. RESULTS: At least 296 distinct processes are required for phase III trial activation: at least 239 working steps, 52 major decision points, 20 processing loops, and 11 stopping points. Of the 195 trials activated during the January 1, 2000, to December 31, 2007, study period, a sample of 167 (85.6%) was used for gathering timing data. Median calendar days from initial formal concept submission to CTEP to trial activation by a cooperative group was 602 days (interquartile range, 454 to 861 days). This time has not significantly changed over the past 8 years. There is a high variation in the time required to activate a clinical trial. CONCLUSION: Because of their complexity, the overall development time for phase III clinical trials is lengthy, process laden, and highly variable. To streamline the process, a solution must be sought that includes all parties involved in developing trials.

Development of clinical trials in a cooperative group setting: the eastern cooperative oncology group.

PURPOSE: We examine the processes and document the calendar time required to activate phase II and III clinical trials by an oncology group: the Eastern Cooperative Oncology Group (ECOG). METHODS: Setup steps were documented by (a) interviewing ECOG headquarters and statistical center staff, and committee chairs, (b) reviewing standard operating procedure manuals, and (c) inspecting study records, documents, and e-mails to identify additional steps. Calendar time was collected for each major process for each study in this set. RESULTS: Twenty-eight phase III studies were activated by ECOG during the January 2000 to July 2006 study period. We examined a sample from 16 of those studies in detail. More than 481 distinct processes were required for study activation: 420 working steps, 61 major decision points, 26 processing loops, and 13 stopping points. Median calendar days to activate a trial in the phase III subset was 783 days (range, 285-1,542 days) from executive approval and 808 days (range, 435-1,604 days) from initial conception of the study. Data were collected for all phase II and phase III trials activated and completed during this time period (n = 52) for which development time represented 43.9% and 54.1% of the total trial time, respectively. CONCLUSION: The steps required to develop and activate a clinical trial may require as much or more time than the actual completion of a trial. The data shows that to improve the activation process, research should to be directed toward streamlining both internal and external groups and processes.

Invisible barriers to clinical trials: the impact of structural, infrastructural, and procedural barriers to opening oncology clinical trials.

PURPOSE: To investigate the administrative barriers that impact the opening of clinical trials at the Vanderbilt-Ingram Cancer Center (VICC) and at VICC Affiliate Network (VICCAN) sites. METHODS: VICC, a National Cancer Institute-designated comprehensive cancer center, and three VICCAN community practice sites were studied. Methodology used was identification and mapping of existing processes and analysis of historical timing data. RESULTS: At course granularity, the process steps required at VICC and VICCAN main office plus local sites are 20 v 17 to 30 steps, respectively; this gap widens with finer granularity, with more than 110 v less than 60 steps, respectively. Approximately 50% of the steps are nonvalue added. For example, in the institutional review board (IRB) process, less than one third of the steps add value to the final protocol. The numbers of groups involved in the approval processes are 27 (VICC) and 6 to 14 (VICCAN home office and local sites). The median times to open a trial are 171 days (95% CI, 158 to 182 days) for VICC and 191 days (95% CI, 119 to 269 days) for the VICCAN sites. Contrary to expectations, the time for IRB review and approval (median, 47 days) is the fastest process compared with the scientific review committee review and approval (median, 70 days) and contracts and grants review (median, 78.5 days). Opening a cooperative group clinical trial is significantly (P = .05) more rapid because they require fewer review steps. CONCLUSION: There are numerous opportunities to remove nonvalue-added steps and save time in opening clinical trials. With increasing numbers of new agents, fewer domestic principal investigators, and more companies off-shoring clinical trials, overcoming such barriers is of critical importance for maintenance of core oncology research capabilities in the United States.