The Veterans Affairs Cardiac Risk Score: Recalibrating the Atherosclerotic Cardiovascular Disease Score for Applied Use.

BACKGROUND: Accurately estimating cardiovascular risk is fundamental to good decision-making in cardiovascular disease (CVD) prevention, but risk scores developed in one population often perform poorly in dissimilar populations. We sought to examine whether a large integrated health system can use their electronic health data to better predict individual patients' risk of developing CVD. METHODS: We created a cohort using all patients ages 45-80 who used Department of Veterans Affairs (VA) ambulatory care services in 2006 with no history of CVD, heart failure, or loop diuretics. Our outcome variable was new-onset CVD in 2007-2011. We then developed a series of recalibrated scores, including a fully refit "VA Risk Score-CVD (VARS-CVD)." We tested the different scores using standard measures of prediction quality. RESULTS: For the 1,512,092 patients in the study, the Atherosclerotic cardiovascular disease risk score had similar discrimination as the VARS-CVD (c-statistic of 0.66 in men and 0.73 in women), but the Atherosclerotic cardiovascular disease model had poor calibration, predicting 63% more events than observed. Calibration was excellent in the fully recalibrated VARS-CVD tool, but simpler techniques tested proved less reliable. CONCLUSIONS: We found that local electronic health record data can be used to estimate CVD better than an established risk score based on research populations. Recalibration improved estimates dramatically, and the type of recalibration was important. Such tools can also easily be integrated into health system's electronic health record and can be more readily updated.

Corrected ROC analysis for misclassified binary outcomes.

Creating accurate risk prediction models from Big Data resources such as Electronic Health Records (EHRs) is a critical step toward achieving precision medicine. A major challenge in developing these tools is accounting for imperfect aspects of EHR data, particularly the potential for misclassified outcomes. Misclassification, the swapping of case and control outcome labels, is well known to bias effect size estimates for regression prediction models. In this paper, we study the effect of misclassification on accuracy assessment for risk prediction models and find that it leads to bias in the area under the curve (AUC) metric from standard ROC analysis. The extent of the bias is determined by the false positive and false negative misclassification rates as well as disease prevalence. Notably, we show that simply correcting for misclassification while building the prediction model is not sufficient to remove the bias in AUC. We therefore introduce an intuitive misclassification-adjusted ROC procedure that accounts for uncertainty in observed outcomes and produces bias-corrected estimates of the true AUC. The method requires that misclassification rates are either known or can be estimated, quantities typically required for the modeling step. The computational simplicity of our method is a key advantage, making it ideal for efficiently comparing multiple prediction models on very large datasets. Finally, we apply the correction method to a hospitalization prediction model from a cohort of over 1 million patients from the Veterans Health Administrations EHR. Implementations of the ROC correction are provided for Stata and R. Published 2017. This article is a U.S. Government work and is in the public domain in the USA.

Effects of performance measure implementation on clinical manager and provider motivation.

BACKGROUND: Clinical performance measurement has been a key element of efforts to transform the Veterans Health Administration (VHA). However, there are a number of signs that current performance measurement systems used within and outside the VHA may be reaching the point of maximum benefit to care and in some settings, may be resulting in negative consequences to care, including overtreatment and diminished attention to patient needs and preferences. Our research group has been involved in a long-standing partnership with the office responsible for clinical performance measurement in the VHA to understand and develop potential strategies to mitigate the unintended consequences of measurement. OBJECTIVE: Our aim was to understand how the implementation of diabetes performance measures (PMs) influences management actions and day-to-day clinical practice. DESIGN: This is a mixed methods study design based on quantitative administrative data to select study facilities and quantitative data from semi-structured interviews. PARTICIPANTS: Sixty-two network-level and facility-level executives, managers, front-line providers and staff participated in the study. APPROACH: Qualitative content analyses were guided by a team-based consensus approach using verbatim interview transcripts. A published interpretive motivation theory framework is used to describe potential contributions of local implementation strategies to unintended consequences of PMs. KEY RESULTS: Implementation strategies used by management affect providers' response to PMs, which in turn potentially undermines provision of high-quality patient-centered care. These include: 1) feedback reports to providers that are dissociated from a realistic capability to address performance gaps; 2) evaluative criteria set by managers that are at odds with patient-centered care; and 3) pressure created by managers' narrow focus on gaps in PMs that is viewed as more punitive than motivating. CONCLUSIONS: Next steps include working with VHA leaders to develop and test implementation approaches to help ensure that the next generation of PMs motivate truly patient-centered care and are clinically meaningful.

Comparison of Veteran experiences of low-cost, home-based diet and exercise interventions.

Obesity is a significant health problem among Veterans who receive care from the Department of Veterans Affairs, as it is for so many other Americans. Veterans from Operation Enduring Freedom (OEF) and Operation Iraqi Freedom (OIF) experience a myriad of chronic conditions, which can make it difficult to maintain a physically active lifestyle. This pilot study tested the feasibility and user satisfaction with three low-cost, home-based diet and exercise programs, as well as point-of-decision prompts among these Veterans. The three programs target mechanisms that have been shown to improve healthy behavior change, including (1) online mediated social support, (2) objective monitoring of physical activity, and (3) structured high-intensity workouts. This was a randomized crossover trial; each participant used two of the three programs, and all used the point-of-decision prompts. Our qualitative results identified five overall themes related to social support, objective monitoring, structured activity, awareness and understanding, and the point-of-decision prompts. In general, participants were satisfied with and lost weight with each of the interventions. This study demonstrated that these low-cost interventions could be successful with the OIF/OEF Veteran population. A larger and longer study is planned to further investigate the effectiveness of these interventions.

Incorporating patient perspectives into the personal health record: implications for care and caring.

Electronic personal health records (ePHRs) can potentially maximize access and coordination of health information and improve patient/clinician collaboration, patient self-management, and health outcomes. Most ePHRs are designed by vendors, physicians, and other proprietary partners and have neglected the patient perspective. This study sought to incorporate patient feedback into an existing ePHR system. Patients participated in a semistructured interview after one to two weeks of using an ePHR. Interviews addressed strengths and weaknesses of the PHR. Two iterations of interviews, referred to as Wave 1 and Wave 2, occurred sequentially. An iterative process of theme identification was used, and three theme categories (User, System Acceptance, and Technology) were identified in the two waves. Seven technology themes with 40 specific questions were identified and were rank ordered by importance and feasibility, and 20 suggestions were subsequently implemented into the ePHR. Thus, incorporating patient feedback on specific utilities and functionality into an existing ePHR is possible.

Accelerated aging of intervertebral discs in a mouse model of progeria.

Intervertebral disc degeneration (IDD) is a common and debilitating disorder that results in reduced flexibility of the spine, pain, and reduced mobility. Risk factors for IDD include age, genetic predisposition, injury, and other environmental factors such as smoking. Loss of proteoglycans (PGs) contributes to IDD with advancing age. Currently there is a lack of a model for rapid investigation of disc aging and evaluation of therapeutic interventions. Here we examined progression of disc aging in a murine model of a human progeroid syndrome caused by deficiency of the DNA repair endonuclease, ERCC1-XPF (Ercc1(-/Δ) mice). The ERCC1-deficient mice showed loss of disc height and degenerative structural changes in their vertebral bodies similar to those reported for old rodents. Compared to their wild-type littermates, Ercc1(-/Δ) mice also exhibit other age-related IDD characteristics, including premature loss of disc PG, reduced matrix PG synthesis, and enhanced apoptosis and cell senescence. Finally, the onset of age-associated disc pathologies was further accelerated in Ercc1(-/Δ) mice following chronic treatment with the chemotherapeutic agent mechlorethamine. These results demonstrate that Ercc1(-/Δ) mice represent an accurate and rapid model of disc aging and provide novel evidence that DNA damage negatively impacts PG synthesis.