Toward a Framework for Benefit-Risk Assessment in Diagnostic Imaging: Identifying Scenario-specific Criteria.

RATIONALE AND OBJECTIVES: Diagnostic imaging has many effects and there is no common definition of value in diagnostic radiology. As benefit-risk trade-offs are rarely made explicit, it is not clear which framework is used in clinical guideline development. We describe initial steps toward the creation of a benefit-risk framework for diagnostic radiology. MATERIALS AND METHODS: We performed a literature search and an online survey of physicians to identify and collect benefit-risk criteria (BRC) relevant to diagnostic imaging tests. We operationalized a process for selection of BRC with the use of four clinical use case scenarios that vary by diagnostic alternatives and clinical indication. Respondent BRC selections were compared across clinical scenarios and between radiologists and nonradiologists. RESULTS: Thirty-six BRC were identified and organized into three domains: (1) those that account for differences attributable only to the test or device (n = 17); (2) those that account for clinical management and provider experiences (n = 12); and (3) those that capture patient experience (n = 7). Forty-eight survey participants selected 22 criteria from the initial list in the survey (9-11 per case). Engaging ordering physicians increased the number of criteria selected in each of the four clinical scenarios presented. We developed a process for standardizing selection of BRC in guideline development. CONCLUSION: These results suggest that a process relying on elements of comparative effectiveness and the use of standardized BRC may ensure consistent examination of differences among alternatives by way of making explicit implicit trade-offs that otherwise enter the decision-making space and detract from consistency and transparency. These findings also highlight the need for multidisciplinary teams that include input from ordering physicians.

Using the Analytic Hierarchy Process for Prioritizing Imaging Tests in Diagnosis of Suspected Appendicitis.

RATIONALE AND OBJECTIVES: In clinical guideline or criteria development processes, such as those used in developing American College of Radiology Appropriateness Criteria (ACR AC), experts subjectively evaluate benefits and risks associated with imaging tests and make complex decisions about imaging recommendations. The analytic hierarchy process (AHP) decomposes complex decisions into structured smaller decisions, incorporates quantitative evidence and qualitative expert opinion, and promotes structured consensus building. AHP may supplement and/or improve the transparency of expert opinion contributions to developing guidelines or criteria. MATERIALS AND METHODS: To conduct an empirical test using health services research tools, we convened a mock ACR AC panel of emergency department radiology and nonradiology physicians to evaluate by multicriteria decision analysis, the relative appropriateness of imaging tests for diagnosing suspected appendicitis. Panel members selected benefit-risk criteria via an online survey and assessed contrast-enhanced computed tomography, magnetic resonance imaging, and ultrasound using an AHP-based software. Participants were asked whether the process was manageable, transparent, and improved shared understanding. Priority scores were converted to rankings and compared to the rank order of ACR AC ratings. RESULTS: When compared to magnetic resonance and ultrasound imaging, participants agreed with the ACR AC that contrast-enhanced computed tomography is the most appropriate test. Contrary to the ACR AC ratings, study results suggest that magnetic resonance is preferable to ultrasound. When compared to nonradiologists, radiologists' priority scores reflect a stronger preference for computed tomography. CONCLUSIONS: Study participants addressed decision-making challenges using a relatively efficient data collection mechanism, suggesting that AHP may benefit the ACR AC guideline development process in identifying the relative appropriateness of imaging tests. With additional development, AHP may improve transparency when expert opinion is used in clinical guideline or appropriateness criteria development.

A proposed approach for quantitative benefit-risk assessment in diagnostic radiology guideline development: the American College of Radiology Appropriateness Criteria Example.

The American College of Radiology develops evidence-based practice guidelines to aid appropriate utilization of radiological procedures. Panel members use expert opinion to weight trade-offs and consensus methods to rate appropriateness of imaging tests. These ratings include an equivocal range, assigned when there is disagreement about a technology's appropriateness and the evidence base is weak or for special circumstances. It is not clear how expert consensus merges with the evidence base to arrive at an equivocal rating. Quantitative benefit-risk assessment (QBRA) methods may assist decision makers in this capacity. However, many methods exist and it is not clear which methods are best suited for this application. We perform a critical appraisal of QBRA methods and propose several steps that may aid in making transparent areas of weak evidence and barriers to consensus in guideline development. We identify QBRA methods with potential to facilitate decision making in guideline development and build a decision aid for selecting among these methods. This study identified 2 families of QBRA methods suited to guideline development when expert opinion is expected to contribute substantially to decision making. Key steps to deciding among QBRA methods involve identifying specific benefit-risk criteria and developing a state-of-evidence matrix. For equivocal ratings assigned for reasons other than disagreement or weak evidence base, QBRA may not be needed. In the presence of disagreement but the absence of a weak evidence base, multicriteria decision analysis approaches are recommended; and in the presence of weak evidence base and the absence of disagreement, incremental net health benefit alone or combined with multicriteria decision analysis is recommended. Our critical appraisal further extends investigation of the strengths and limitations of select QBRA methods in facilitating diagnostic radiology clinical guideline development. The process of using the decision aid exposes and makes transparent areas of weak evidence and barriers to consensus.

Long-term costs of introducing HPV-DNA post-treatment surveillance to national cervical cancer screening in Ireland.

INTRODUCTION: Co-testing (cytology plus human papillomavirus DNA testing) as part of cervical cancer surveillance in Ireland increases one-time testing costs. Of interest to policy makers was the long-term impact of these costs accompanied by decreases in intensity of recalls for women with no detected abnormalities. METHODS: A cost analysis of cytology-only and co-testing strategy was implemented using decision analytic modeling, aggregating testing utilization and costs for each of the two strategies over 12 years. RESULTS: Aggregated incremental costs of the co-testing strategy were positive for the first 3 years but became negative thereafter, generating a cost savings of roughly €20 million in favor of the cytology-only strategy over a 12-year period. Results were robust over a range of sensitivity analyses with respect to discount and attrition rates. DISCUSSION: This analysis provided valuable information to policy makers contributing to the introduction of co-testing for post-treatment surveillance (PTS) in Ireland.

Introducing Pathogen Reduction Technology in Poland: A Cost-Utility Analysis.

BACKGROUND: Mirasol® pathogen reduction technology (PRT) uses UV light and riboflavin to chemically inactivate pathogens and white blood cells in blood components. In the EU, Mirasol PRT is CE-marked for both plasma and platelet treatment. In Poland, the decision to introduce PRT treatment of the national supply of fresh frozen plasma has spurred interest in evaluating the cost-effectiveness of this strategy. METHODS: A decision-analytic model evaluated the incremental costs and benefits of introducing PRT to the existing blood safety protocols in Poland. RESULTS: Addition of PRT treatment of plasma to current screening in Poland is estimated to cost 2.595 million PLN per quality-adjusted life year (QALY) (610,000 EUR/QALY); treating both plasma and platelet components in addition to current safety interventions had a lower cost of 1.480 million PLN/QALY (348,000 EUR/QALY). CONCLUSIONS: The results suggest that in Poland the cost per QALY of PRT is high albeit lower than found in previous economic analyses of PRT and nucleic acid testing in North America. Treating both platelets and plasma components is more cost-effective than treating plasma alone. Wide confidence intervals indicate high uncertainty; to improve the precision of the health economic evaluation of PRT, additional hemovigilance data are needed.

Using indirect comparisons to compare interventions within a Cochrane review: a tool for comparative effectiveness research.

AIM: Assessing relative performance among competing interventions is an important part of comparative effectiveness research. Bayesian indirect comparisons add information to existing Cochrane reviews, such as which intervention is likely to perform best. However, heterogeneity variance priors may influence results and, potentially, clinical guidance. METHODS: We highlight the features of Bayesian indirect comparisons using a case study of a Cochrane review update in asthma care. The probability that one self-management educational intervention outperforms others is estimated. Simulation studies investigate the effect of heterogeneity variance prior distributions. RESULTS: Results suggest a 55% probability that individual education is best, followed by combination (39%) and group (6%). The intervention with few trials was sensitive to prior distributions. CONCLUSION: Bayesian indirect comparisons updates of Cochrane reviews are valuable comparative effectiveness research tools.

Applying quantitative benefit-risk analysis to aid regulatory decision making in diagnostic imaging: methods, challenges, and opportunities.

RATIONALE AND OBJECTIVES: Health agencies making regulatory marketing-authorization decisions use qualitative and quantitative approaches to assess expected benefits and expected risks associated with medical interventions. There is, however, no universal standard approach that regulatory agencies consistently use to conduct benefit-risk assessment (BRA) for pharmaceuticals or medical devices, including for imaging technologies. Economics, health services research, and health outcomes research use quantitative approaches to elicit preferences of stakeholders, identify priorities, and model health conditions and health intervention effects. MATERIALS AND METHODS: Challenges to BRA in medical devices are outlined, highlighting additional barriers in radiology. Three quantitative methods--multi-criteria decision analysis, health outcomes modeling and stated-choice survey--are assessed using criteria that are important in balancing benefits and risks of medical devices and imaging technologies. RESULTS: To be useful in regulatory BRA, quantitative methods need to: aggregate multiple benefits and risks, incorporate qualitative considerations, account for uncertainty, and make clear whose preferences/priorities are being used. Each quantitative method performs differently across these criteria and little is known about how BRA estimates and conclusions vary by approach. While no specific quantitative method is likely to be the strongest in all of the important areas, quantitative methods may have a place in BRA of medical devices and radiology. DISCUSSION: Quantitative BRA approaches have been more widely applied in medicines, with fewer BRAs in devices. Despite substantial differences in characteristics of pharmaceuticals and devices, BRA methods may be as applicable to medical devices and imaging technologies as they are to pharmaceuticals. Further research to guide the development and selection of quantitative BRA methods for medical devices and imaging technologies is needed.

Cancer Incidence and Mortality in a Cohort of US Blood Donors: A 20-Year Study.

Blood donors are considered one of the healthiest populations. This study describes the epidemiology of cancer in a cohort of blood donors up to 20 years after blood donation. Records from donors who participated in the Retroviral Epidemiology Donor Study (REDS, 1991-2002) at Blood Centers of the Pacific (BCP), San Francisco, were linked to the California Cancer Registry (CCR, 1991-2010). Standardized incidence ratios (SIR) were estimated using standard US 2000 population, and survival analysis used to compare all-cause mortality among donors and a random sample of nondonors with cancer from CCR. Of 55,158 eligible allogeneic blood donors followed-up for 863,902 person-years, 4,236 (7.7%) primary malignant cancers were diagnosed. SIR in donors was 1.59 (95% CI = 1.54,1.64). Donors had significantly lower mortality (adjusted HR = 0.70, 95% CI = 0.66-0.74) compared with nondonor cancer patients, except for respiratory system cancers (adjusted HR = 0.93, 95% CI = 0.82-1.05). Elevated cancer incidence among blood donors may reflect higher diagnosis rates due to health seeking behavior and cancer screening in donors. A "healthy donor effect" on mortality following cancer diagnosis was demonstrated. This population-based database and sample repository of blood donors with long-term monitoring of cancer incidence provides the opportunity for future analyses of genetic and other biomarkers of cancer.

Epidemiologic and laboratory findings from 3 years of testing United States blood donors for Trypanosoma cruzi.

BACKGROUND: At most blood centers in the United States routine testing of donations for Trypanosoma cruzi using an enzyme-linked immunosorbent assay (ELISA) is followed by supplemental testing by radioimmunoprecipitation assay (RIPA). The objective of this study was to report the results of routine testing and risk factor data from allogeneic blood donors. STUDY DESIGN AND METHODS: T. cruzi testing data from January 2007 through December 2009 were analyzed, and risk factor interviews and follow-up studies were conducted on seroreactive donors. Prevalences of confirmed infection and risk factors associated with infection were assessed using logistic and multivariable logistic regression. RESULTS: Of 2,940,491 allogeneic donations from 1,183,076 donors, 305 (0.01% per donation tested and 0.026% per blood donor) were repeat reactive (RR) and 89 of those were confirmed positive by RIPA, yielding an overall seroprevalence of 1 per 33,039 donations and 1 per 13,292 donors. Country of birth and US blood center location differences in the seroprevalence of T. cruzi were evident. The odds of confirmed infection were highest if the donor reported having been bitten by the reduviid (kissing) bug (odds ratio [OR], 76.1; 95% confidence interval [CI], 11.1-3173) followed by having lived in a rural area of Latin America (OR, 38.6; 95% CI, 15.1-102.5). In multivariable analyses, having spent 3 months or more in Mexico or Central and/or South America was associated with the highest odds of RIPA-confirmed infection (OR, 8.5; 95% CI, 2.7-26.5). Polymerase chain reaction (PCR) testing of ELISA RR donors exhibited low sensitivity (1/22 [4%] RIPA-confirmed donors was PCR positive). CONCLUSION: Risk factors for confirmed infection in US blood donors are consistent with the known epidemiology of Chagas disease. Blood donors or transfusions do not substantially contribute to the burden of T. cruzi infection in the United States.

Association between HLA class I and class II alleles and the outcome of West Nile virus infection: an exploratory study.

BACKGROUND: West Nile virus (WNV) infection is asymptomatic in most individuals, with a minority developing symptoms ranging from WNV fever to serious neuroinvasive disease. This study investigated the impact of host HLA on the outcome of WNV disease. METHODS: A cohort of 210 non-Hispanic mostly white WNV(+) subjects from Canada and the U.S. were typed for HLA-A, B, C, DP, DQ, and DR. The study subjects were divided into three WNV infection outcome groups: asymptomatic (AS), symptomatic (S), and neuroinvasive disease (ND). Allele frequency distribution was compared pair-wise between the AS, S, and ND groups using χ2 and Fisher's exact tests and P values were corrected for multiple comparisons (Pc). Allele frequencies were compared between the groups and the North American population (NA) used as a control group. Logistic regression analysis was used to evaluate the potential synergistic effect of age and HLA allele phenotype on disease outcome. RESULTS: The alleles HLA-A*68, C*08 and DQB*05 were more frequently associated with severe outcomes (ND vs. AS, P(A*68) = 0.013/Pc = 0.26, P(C*08) = 0.0075/Pc = 0.064, and P(DQB1*05) = 0.029/Pc = 0.68), However the apparent DQB1*05 association was driven by age. The alleles HLA-B*40 and C*03 were more frequently associated with asymptomatic outcome (AS vs. S, P(B*40) = 0.021/Pc = 0.58 and AS vs. ND P(C*03) = 0.039/Pc = 0.64) and their frequencies were lower within WNV(+) subjects with neuroinvasive disease than within the North American population (NA vs. S, P(B*40) = 0.029 and NA vs. ND, P(C*03) = 0.032). CONCLUSIONS: Host HLA may be associated with the outcome of WNV disease; HLA-A*68 and C*08 might function as "susceptible" alleles, whereas HLA-B*40 and C*03 might function as "protective" alleles.

The cost-effectiveness of pathogen reduction technology as assessed using a multiple risk reduction model.

BACKGROUND: Pathogen reduction technology (PRT) for labile blood components has the potential to reduce the risk of many adverse events associated with transfusion. Because of the potential broad-spectrum risk reduction capability of PRT, the health economics of PRT could be an important consideration in decision making for this technology. STUDY DESIGN AND METHODS: Decision analytic models comparing current blood safety screens and interventions to riboflavin-based whole blood PRT (currently in development) and separately to platelets (PLTs)-and-plasma PRT from the health care system perspective in Canada were used to assess the cost-utility of PRT in reducing the following adverse events: human immunodeficiency virus, hepatitis B virus, hepatitis C virus, human T-lymphotropic virus, syphilis, West Nile virus, bacteria, Chikungunya virus, cytomegalovirus, Trypanosoma cruzi, graft-versus-host disease, febrile nonhemolytic transfusion reactions, and transfusion-related immunomodulation. PRT was modeled as an addition to rather than a replacement for current interventions. The potential of PRT to reduce the risk of an unknown pathogen was not assessed. RESULTS: Whole blood PRT was estimated to have a cost-effectiveness of $1,276,000/quality-adjusted life-year (QALY; 95% confidence interval [CI] approximation, 600,000-3,313,000) compared to current screens and interventions. PLTs-and-plasma PRT was estimated to have a cost-effectiveness of $1,423,000/QALY (95% CI approximation, 834,000-2,818,000) on an all-transfusions basis. CONCLUSIONS: Because of the complexity of transfusion risks and practices, the cost-effectiveness of whole blood or PLTs-and-plasma PRT can be modeled provided that assumptions and simplifications are made. Uncertainty remains with respect to the risk reduction that can be achieved for some adverse events. Nevertheless, the results of this cost-effectiveness analysis can be used to inform policy decisions regarding PRT technology in the context of other initiatives designed to improve transfusion safety.

Cost-effectiveness of screening the US blood supply for Trypanosoma cruzi.

BACKGROUND: Trypanosoma cruzi, the etiologic agent of Chagas disease, is a potential threat to transfusion recipients in the United States. The cost-effectiveness of seven testing strategies was evaluated against no testing and hierarchically in incremental analysis. Donor-specific strategies included testing donors born in endemic countries, testing all donors a specific number of times, or testing all donors every time. Component-specific strategies are based on screening platelet-containing donations. STUDY DESIGN AND METHODS: A decision analytic model simulated the lifetime cost (US dollars) and health outcomes (quality-adjusted life-years [QALYs]) of two hypothetical cohorts of blood recipients, an all-ages and a younger subset, from a 2007 societal perspective. Model variable values were obtained from US screening data, Blood Systems Laboratory, the Health Care Utilization Project, and published literature. RESULTS: For the all-ages cohort, compared to no testing, the cost-effectiveness of testing all donors one time was $757,000 per QALY, all donors two times $970,000 per QALY, and universal testing $1.36 million per QALY. In the all-ages and the younger transfused populations, testing donors with geographical exposure was most cost-effective ($173,000 and $29,000/QALY, respectively). The most influential variables in the model were related to characteristics of the transfused population: survival and health state utilities. With respect to T. cruzi variables, results were most sensitive to seroprevalence and transmissibility. CONCLUSION: Selective T. cruzi screening generates nearly the same effectiveness as universal screening, but at a reduced cost. Outcomes and associated costs of Chagas disease take longer to materialize than the average life expectancy of transfusion recipients.