From Clinical Trial Efficacy to Real-Life Effectiveness: Why Conventional Metrics do not Work.

BACKGROUND: Randomised, double-blind, clinical trial methodology minimises bias in the measurement of treatment efficacy. However, most phase III trials in non-orphan diseases do not include individuals from the population to whom efficacy findings will be applied in the real world. Thus, a translation process must be used to infer effectiveness for these populations. Current conventional translation processes are not formalised and do not have a clear theoretical or practical base. There is a growing need for accurate translation, both for public health considerations and for supporting the shift towards personalised medicine. OBJECTIVE: Our objective was to assess the results of translation of efficacy data to population efficacy from two simulated clinical trials for two drugs in three populations, using conventional methods. METHODS: We simulated three populations, two drugs with different efficacies and two trials with different sampling protocols. RESULTS: With few exceptions, current translation methods do not result in accurate population effectiveness predictions. The reason for this failure is the non-linearity of the translation method. One of the consequences of this inaccuracy is that pharmacoeconomic and postmarketing surveillance studies based on direct use of clinical trial efficacy metrics are flawed. CONCLUSION: There is a clear need to develop and validate functional and relevant translation approaches for the translation of clinical trial efficacy to the real-world setting.

"Artificial intelligence": Which services, which applications, which results and which development today in clinical research? Which impact on the quality of care? Which recommendations?

Artificial intelligence (AI), beyond the concrete applications that have already become part of our daily lives, makes it possible to process numerous and heterogeneous data and knowledge, and to understand potentially complex and abstract rules in a manner human intelligence can but without human intervention. AI combines two properties, self-learning by the successive and repetitive processing of data as well as the capacity to adapt, that is to say the possibility for a scripted program to deal with multiple situations likely to vary over time. Roundtable experts confirmed the potential contribution and theoretical benefit of AI in clinical research and in improving the efficiency of patient care. Experts also measured, as is the case for any new process that people need to get accustomed to, its impact on practices and mindset. To maximize the benefits of AI, four critical points have been identified. The careful consideration of these four points conditions the technical integration and the appropriation by all actors of the life science spectrum: researchers, regulators, drug developers, care establishments, medical practitioners and, above all, patients and the civil society. 1st critical point: produce tangible demonstrations of the contributions of AI in clinical research by quantifying its benefits. 2nd critical point: build trust to foster dissemination and acceptability of AI in healthcare thanks to an adapted regulatory framework. 3rd critical point: ensure the availability of technical skills, which implies an investment in training, the attractiveness of the health sector relative to tech-heavy sectors and the development of ergonomic data collection tools for all health operators. 4th critical point: organize a system of governance for a distributed and secure model at the national level to aggregate the information and services existing at the local level. Thirty-seven concrete recommendations have been formulated which should pave the way for a widespread adoption of AI in clinical research. In this context, the French "Health data hub" initiative constitutes an ideal opportunity.

Comparison of an effect-model-law-based method versus traditional clinical practice guidelines for optimal treatment decision-making: application to statin treatment in the French population.

Healthcare authorities make difficult decisions about how to spend limited budgets for interventions that guarantee the best cost-efficacy ratio. We propose a novel approach for treatment decision-making, OMES-in French: Objectif thérapeutique Modèle Effet Seuil (in English: Therapeutic Objective-Threshold-Effect Model; TOTEM). This approach takes into consideration results from clinical trials, adjusted for the patients' characteristics in treatment decision-making. We compared OMES with the French clinical practice guidelines (CPGs) for the management of dyslipidemia with statin in a computer-generated realistic virtual population, representing the adult French population, in terms of the number of all-cause deaths avoided (number of avoided events: NAEs) under treatment and the individual absolute benefit. The total budget was fixed at the annual amount reimbursed by the French social security for statins. With the CPGs, the NAEs was 292 for an annual cost of 122.54 M€ compared with 443 with OMES. For a fixed NAEs, OMES reduced costs by 50% (60.53 M€ yr(-1)). The results demonstrate that OMES is at least as good as, and even better than, the standard CPGs when applied to the same population. Hence the OMES approach is a practical, useful alternative which will help to overcome the limitations of treatment decision-making based uniquely on CPGs.

Effect model law: an approach for the implementation of personalized medicine.

The effect model law states that a natural relationship exists between the frequency (observation) or the probability (prediction) of a morbid event without any treatment and the frequency or probability of the same event with a treatment. This relationship is called the effect model. It applies to a single individual, individuals within a population, or groups. In the latter case, frequencies or probabilities are averages of the group. The relationship is specific to a therapy, a disease or an event, and a period of observation. If one single disease is expressed through several distinct events, a treatment will be characterized by as many effect models. Empirical evidence, simulations with models of diseases and therapies and virtual populations, as well as theoretical derivation support the existence of the law. The effect model could be estimated through statistical fitting or mathematical modelling. It enables the prediction of the (absolute) benefit of a treatment for a given patient. It thus constitutes the theoretical basis for the design of practical tools for personalized medicine.

Towards personalized medicine: exploring the consequences of the effect model-based approach.

Although personalized medicine has been a subject of research and debate in recent years, it has been underused in medical practice, except in some cancers. We believe that the main reason for the gap between the potential of personalized medicine and its use in daily medical practice can be explained by the lack of an appropriate tool to facilitate the use of biomarker values in a doctor's decision-making process. We propose that the effect model could form the basis of such a tool.

Level of evidence and therapeutic evaluation: need for more thoughts.

The practice of evidence-based medicine requires a tool to assess and discriminate available data based on objective grounds, thus facilitating access to reliable information. The level of evidence, conceptually and practically embedded in scientific activity, allows comparing the results from multiple studies testing an identical hypothesis along the lines of at least two dimensions. The first dimension deals with the design of the study, i.e. the extent to which bias is avoided or managed, the second with the quality of incorporated data. A third dimension specific to therapeutic evaluation focuses on the clinical relevance of the tested hypothesis. The concern of the final user of the information is thus put to the fore. Indeed, a general practitioner will be interested in the benefit for its patients whereas the concern of a biologist might significantly diverge from the former matter. The bulk of existing scales of level of evidence concentrate on methodology. Some may include the second dimension but none embrace the three of them. Seldom considered are matters regarding reproducibility and procedure. This is all the more unfortunate as reproducibility is a cornerstone of scientific progress. Moreover, scales used for overviews fail to take into account the methodology designed to produce the synthesis. Inconsistent existing scales prevent the emergence of a generally agreed standard. Therefore, there is a need to further specify the concept of level of evidence in therapy evaluation and design scales encompassing the three above-mentioned dimensions: methodology of experiment, quality of data, and clinical relevance of the primary criterion.