User Satisfaction Evaluation of the EHR4CR Query Builder: A Multisite Patient Count Cohort System.

The Electronic Health Records for Clinical Research (EHR4CR) project aims to develop services and technology for the leverage reuse of Electronic Health Records with the purpose of improving the efficiency of clinical research processes. A pilot program was implemented to generate evidence of the value of using the EHR4CR platform. The user acceptance of the platform is a key success factor in driving the adoption of the EHR4CR platform; thus, it was decided to evaluate the user satisfaction. In this paper, we present the results of a user satisfaction evaluation for the EHR4CR multisite patient count cohort system. This study examined the ability of testers (n = 22 and n = 16 from 5 countries) to perform three main tasks (around 20 minutes per task), after a 30-minute period of self-training. The System Usability Scale score obtained was 55.83 (SD: 15.37), indicating a moderate user satisfaction. The responses to an additional satisfaction questionnaire were positive about the design of the interface and the required procedure to design a query. Nevertheless, the most complex of the three tasks proposed in this test was rated as difficult, indicating a need to improve the system regarding complicated queries.

Web-based multi-site feasibility questionnaire tool.

The design of clinical trial (CT) study protocols, currently supported by clinicians, is often a slow and cumbersome process. The Electronic Health Records for Clinical Research (EHR4CR) project supports the design of study protocols through a multi-site patient count cohort system. However, there is still a need to improve the process step in which the clinicians are involved. This research aims to enhance the EHR4CR platform with a tool to support the contact of CT sponsors with clinical investigators to obtain their input regarding feasibility data for the CT protocol design. From a list of requirements, a technical architecture that responds to the needs of feasibility assessments was modelled. With this architecture as a basis, a system that allows users to generate, send, fill out and visualise results of feasibility questionnaires across clinical sites was developed and integrated within the EHR4CR platform. The resulting system improves the current methods by providing direct contact to clinical investigators, facilitating the creation and answer of feasibility questionnaires for CTs.

Efficiency and effectiveness evaluation of an automated multi-country patient count cohort system.

BACKGROUND: With the increase of clinical trial costs during the last decades, the design of feasibility studies has become an essential process to reduce avoidable and costly protocol amendments. This design includes timelines, targeted sites and budget, together with a list of eligibility criteria that potential participants need to match. The present work was designed to assess the value of obtaining potential study participant counts using an automated patient count cohort system for large multi-country and multi-site trials: the Electronic Health Records for Clinical Research (EHR4CR) system. METHODS: The evaluation focuses on the accuracy of the patient counts and the time invested to obtain these using the EHR4CR platform compared to the current questionnaire based process. This evaluation will assess the patient counts from ten clinical trials at two different sites. In order to assess the accuracy of the results, the numbers obtained following the two processes need to be compared to a baseline number, the "alloyed" gold standard, which was produced by a manual check of patient records. RESULTS: The patient counts obtained using the EHR4CR system were in three evaluated trials more accurate than the ones obtained following the current process whereas in six other trials the current process counts were more accurate. In two of the trials both of the processes had counts within the gold standard's confidence interval. In terms of efficiency the EHR4CR protocol feasibility system proved to save approximately seven calendar days in the process of obtaining patient counts compared to the current manual process. CONCLUSIONS: At the current stage, electronic health record data sources need to be enhanced with better structured data so that these can be re-used for research purposes. With this kind of data, systems such as the EHR4CR are able to provide accurate objective patient counts in a more efficient way than the current methods. Additional research using both structured and unstructured data search technology is needed to assess the value of unstructured data and to compare the amount of efforts needed for data preparation.

Feasibility platform for stroke studies: an online tool to improve eligibility criteria for clinical trials.

BACKGROUND AND PURPOSE: Eligibility criteria are a key factor for the feasibility and validity of clinical trials. We aimed to develop an online tool to assess the potential effect of inclusion and exclusion criteria on the proportion of patients eligible for an acute stroke trial. METHODS: We identified relevant inclusion and exclusion criteria of acute stroke trials. Based on these criteria and using a cohort of 1537 consecutive patients with acute ischemic stroke from 3 stroke centers, we developed a web portal feasibility platform for stroke studies (FePASS) to estimate proportions of eligible patients for acute stroke trials. We applied the FePASS resource to calculate the proportion of patients eligible for 4 recent stroke studies. RESULTS: Sixty-one eligibility criteria were derived from 30 trials on acute ischemic stroke. FePASS, publicly available at http://fepass.uni-muenster.de, displays the proportion of patients in percent to assess the effect of varying values of relevant eligibility criteria, for example, age, symptom onset time, National Institutes of Health Stroke Scale, and prestroke modified Rankin Scale, on this proportion. The proportion of eligible patients for 4 recent stroke studies ranged from 2.1% to 11.3%. Slight variations of the inclusion criteria could substantially increase the proportion of eligible patients. CONCLUSIONS: FePASS is an open access online resource to assess the effect of inclusion and exclusion criteria on the proportion of eligible patients for a stroke trial. FePASS can help to design stroke studies, optimize eligibility criteria, and to estimate the potential recruitment rate.

Comparison of electronic health record system functionalities to support the patient recruitment process in clinical trials.

OBJECTIVES: Reusing data from electronic health records for clinical and translational research and especially for patient recruitment has been tackled in a broader manner since about a decade. Most projects found in the literature however focus on standalone systems and proprietary implementations at one particular institution often for only one singular trial and no generic evaluation of EHR systems for their applicability to support the patient recruitment process does yet exist. Thus we sought to assess whether the current generation of EHR systems in Germany provides modules/tools, which can readily be applied for IT-supported patient recruitment scenarios. METHODS: We first analysed the EHR portfolio implemented at German University Hospitals and then selected 5 sites with five different EHR implementations covering all major commercial systems applied in German University Hospitals. Further, major functionalities required for patient recruitment support have been defined and the five sample EHRs and their standard tools have been compared to the major functionalities. RESULTS: In our analysis of the site's hospital information system environments (with four commercial EHR systems and one self-developed system) we found that - even though no dedicated module for patient recruitment has been provided - most EHR products comprise generic tools such as workflow engines, querying capabilities, report generators and direct SQL-based database access which can be applied as query modules, screening lists and notification components for patient recruitment support. A major limitation of all current EHR products however is that they provide no dedicated data structures and functionalities for implementing and maintaining a local trial registry. CONCLUSIONS: At the five sites with standard EHR tools the typical functionalities of the patient recruitment process could be mostly implemented. However, no EHR component is yet directly dedicated to support research requirements such as patient recruitment. We recommend for future developments that EHR customers and vendors focus much more on the provision of dedicated patient recruitment modules.

Towards a trial-ready mobile patient questionnaire system.

Gathering patient-reported data enables physicians to know a patient and his/her health-related needs in a more comprehensive way. One method to collect these data is to let the patient fill out electronic questionnaires on a mobile device, e.g. a tablet. Although having successfully implemented a prototype, the mobile questionnaire system at this site has been stretched to its limits within both routine care and clinical trials. By collecting user feedback and analysing the capabilities and limits of the underlying framework, we identified key issues of the prototype. A new implementation addresses these obstacles while keeping the overall application logic and usability. This leads to a trial-ready mobile patient questionnaire system.

Protocol feasibility workflow using an automated multi-country patient cohort system.

The Electronic Health Record for Clinical Research (EHR4CR) project aims to improve the current process of clinical trials, providing a technological platform that supports the design and execution of trials. For the protocol feasibility scenario, the system currently allows the user to create a set of in-/exclusion criteria to find patients matching these criteria across sites located in several countries. The automated multi-country patient cohort system developed in EHR4CR implies substantial changes on the current protocol feasibility process, which will be reflected in this study.

Patient recruitment workflow with and without a patient recruitment system.

In clinical trials (CTs), the process of patient recruitment (PR) is one of the main risk factors, as almost half of all trial delays are caused by problems in PR. To our knowledge, no publication in this field describes the process of PR. Therefore, weak spots and potential benefits cannot be identified. By interviewing six domain experts and modeling the workflow in a standardized way, we describe the actors, tasks and tools within PR. We compare the current workflow with Patient Recruitment System (PRS)-supported PR. The identification of eligible participants is the most complex part, but adding a PRS simplifies it by automating repetitive tasks and taking work off the Investigators' hands. This work contributes to a common understanding of the PR process.

Evaluation of data completeness in the electronic health record for the purpose of patient recruitment into clinical trials: a retrospective analysis of element presence.

BACKGROUND: Computerized clinical trial recruitment support is one promising field for the application of routine care data for clinical research. The primary task here is to compare the eligibility criteria defined in trial protocols with patient data contained in the electronic health record (EHR). To avoid the implementation of different patient definitions in multi-site trials, all participating research sites should use similar patient data from the EHR. Knowledge of the EHR data elements which are commonly available from most EHRs is required to be able to define a common set of criteria. The objective of this research is to determine for five tertiary care providers the extent of available data compared with the eligibility criteria of randomly selected clinical trials. METHODS: Each participating study site selected three clinical trials at random. All eligibility criteria sentences were broken up into independent patient characteristics, which were then assigned to one of the 27 semantic categories for eligibility criteria developed by Luo et al. We report on the fraction of patient characteristics with corresponding structured data elements in the EHR and on the fraction of patients with available data for these elements. The completeness of EHR data for the purpose of patient recruitment is calculated for each semantic group. RESULTS: 351 eligibility criteria from 15 clinical trials contained 706 patient characteristics. In average, 55% of these characteristics could be documented in the EHR. Clinical data was available for 64% of all patients, if corresponding data elements were available. The total completeness of EHR data for recruitment purposes is 35%. The best performing semantic groups were 'age' (89%), 'gender' (89%), 'addictive behaviour' (74%), 'disease, symptom and sign' (64%) and 'organ or tissue status' (61%). No data was available for 6 semantic groups. CONCLUSIONS: There exists a significant gap in structure and content between data documented during patient care and data required for patient eligibility assessment. Nevertheless, EHR data on age and gender of the patient, as well as selected information on his disease can be complete enough to allow for an effective support of the manual screening process with an intelligent preselection of patients and patient data.