Which factors may determine the necessary and feasible type of effectiveness evidence? A mixed methods approach to develop an instrument to help coverage decision-makers.

OBJECTIVES: Reimbursement decisions require evidence of effectiveness and, in general, a blinded randomised controlled trial (RCT) is the preferred study design to provide it. However, there are situations where a cohort study, or even patient series, can be deemed acceptable. The aim of this study was to develop an instrument that first examines which study characteristics of a blinded RCT are necessary, and then, if particular characteristics are considered necessary, examines whether these characteristics are feasible. DESIGN: We retrospectively studied 22 interventions from 20 reimbursement reports concerning medical specialist care made by the Dutch National Health Care Institute (ZIN) to identify any factors that influenced the necessity and feasibility of blinded RCTs, and their constituent study characteristics, that is, blinding, randomisation and a control group. A literature review was performed to identify additional factors. Additional expertise was included by interviewing eight experts in epidemiology, medicine and ethics. The resulting instrument was called the FIT instrument (Feasible Information Trajectory), and was prospectively validated using three consecutive reimbursement reports. RESULTS: (Blinded) RCT evidence was lacking in 5 of 11 positive reimbursement decisions and 3 of 11 negative decisions. In the reimbursement reports, we found no empirical evidence supporting situations where a blinded RCT is unnecessary. The literature also revealed few arguments against the necessity of a blinded RCT. In contrast, many factors influencing the feasibility of randomisation, a control group and blinding, were found in the reimbursement reports and the literature; for example, when a patient population is too small or when an intervention is common practice, randomisation will be hindered. CONCLUSIONS: Policy regarding the necessity and feasibility of different types of evidence of effectiveness would benefit from systematic guidance. The FIT instrument has the potential to support transparent, reproducible and well-founded decisions on appropriate evidence of effectiveness in medical specialist care.

['Fitting' evidence preferable when evaluating effectiveness of interventions]. / Passend bewijs bij het bepalen van effectiviteit van interventies.

The Dutch Health Care Insurance Board (CVZ) and the Institute for Medical Technology Assessment have developed a questionnaire to help assessors to formulate an answer to the question of which evidence fits best when evaluating the effectiveness of interventions by medical specialists. The Feasible Information Trajectory (FIT) questionnaire is based on the idea that the clinical setting defines the attainable study characteristics and thus possibly available evidence. The FIT questionnaire focuses on study characteristics (randomization, blinding and control groups) and not on study type (e.g. RCT, cohort) as is common within evidence-hierarchy grading systems. Together these attainable characteristics define the most -fitting evidence. The questionnaire should be filled in prior to the literature assessment. Subsequently the most-fitting evidence profile can be modified on the basis of a literature assessment and possibly arguments put forward by healthcare professionals. Filling out the FIT questionnaire is therefore not a static exercise but is part of the search for all relevant arguments in the assessment of effectiveness.