Transparency and Reproducibility of Observational Cohort Studies Using Large Healthcare Databases.

The scientific community and decision-makers are increasingly concerned about transparency and reproducibility of epidemiologic studies using longitudinal healthcare databases. We explored the extent to which published pharmacoepidemiologic studies using commercially available databases could be reproduced by other investigators. We identified a nonsystematic sample of 38 descriptive or comparative safety/effectiveness cohort studies. Seven studies were excluded from reproduction, five because of violation of fundamental design principles, and two because of grossly inadequate reporting. In the remaining studies, >1,000 patient characteristics and measures of association were reproduced with a high degree of accuracy (median differences between original and reproduction <2% and <0.1). An essential component of transparent and reproducible research with healthcare databases is more complete reporting of study implementation. Once reproducibility is achieved, the conversation can be elevated to assess whether suboptimal design choices led to avoidable bias and whether findings are replicable in other data sources.

Active safety monitoring of newly marketed medications in a distributed data network: application of a semi-automated monitoring system.

We developed a semi-automated active monitoring system that uses sequential matched-cohort analyses to assess drug safety across a distributed network of longitudinal electronic health-care data. In a retrospective analysis, we show that the system would have identified cerivastatin-induced rhabdomyolysis. In this study, we evaluated whether the system would generate alerts for three drug-outcome pairs: rosuvastatin and rhabdomyolysis (known null association), rosuvastatin and diabetes mellitus, and telithromycin and hepatotoxicity (two examples for which alerting would be questionable). Over >5 years of monitoring, rate differences (RDs) in comparisons of rosuvastatin with atorvastatin were -0.1 cases of rhabdomyolysis per 1,000 person-years (95% confidence interval (CI): -0.4, 0.1) and -2.2 diabetes cases per 1,000 person-years (95% CI: -6.0, 1.6). The RD for hepatotoxicity comparing telithromycin with azithromycin was 0.3 cases per 1,000 person-years (95% CI: -0.5, 1.0). In a setting in which false positivity is a major concern, the system did not generate alerts for the three drug-outcome pairs.

Assessing the comparative effectiveness of newly marketed medications: methodological challenges and implications for drug development.

Comparative-effectiveness research (CER) aims to produce actionable evidence regarding the effectiveness and safety of medical products and interventions as they are used outside of controlled research settings. Although CER evidence regarding medications is particularly needed shortly after market approval, key methodological challenges include (i) potential bias due to channeling of patients to the newly marketed medication because of various patient-, physician-, and system-related factors; (ii) rapid changes in the characteristics of the user population during the early phase of marketing; and (iii) lack of timely data and the often small number of users in the first few months of marketing. We propose a mix of approaches to generate comparative-effectiveness data in the early marketing period, including sequential cohort monitoring with secondary health-care data and propensity score (PS) balancing, as well as extended follow-up of phase III and phase IV trials, indirect comparisons of placebo-controlled trials, and modeling and simulation of virtual trials.

The effect of case management on the costs of health care for enrollees in Medicare Plus Choice plans: a randomized trial.

OBJECTIVE: To measure the effects of case management on an older population's costs of health care. DESIGN: A 1-year randomized controlled trial. SETTING: Multiple sites of care in San Francisco, California. PARTICIPANTS: Patients aged 65 or older of primary care physicians in a large provider organization bearing financial risk for their care (n = 6409). INTERVENTION: Screening for high risk and provision of social work-based case management. OUTCOME MEASURES: Volume and cost of hospital, physician, case management, and other health-related services. RESULTS: The experimental group used more case management services than the control group (0.09 vs. 0.02 months per person, P<.001). The experimental group's average total payments for health care were slightly lower ($3148 vs $3277, P = .40). CONCLUSIONS: This study provides no statistically significant evidence that social work-oriented case management reduces the use or the cost of health care for high-risk older people. Other potentially favorable effects of this type of case management need to be evaluated, as do the effects of other types of case management.