Echocardiography Core Laboratory Reproducibility of Cardiac Safety Assessments in Cardio-Oncology.

BACKGROUND: As the potential for cancer therapy-related cardiac dysfunction is increasingly recognized, there is a need for the standardization of echocardiographic measurements and cut points to guide treatment. The aim of this study was to determine the reproducibility of cardiac safety assessments across two academic echocardiography core laboratories (ECLs) at the University of Pennsylvania and the Duke Clinical Research Institute. METHODS: To harmonize the application of guideline-recommended measurement conventions, the ECLs conducted multiple training sessions to align measurement practices for traditional and emerging assessments of left ventricular (LV) function. Subsequently, 25 echocardiograms taken from patients with breast cancer treated with doxorubicin with or without trastuzumab were independently analyzed by each laboratory. Agreement was determined by the proportion (coverage probability [CP]) of all pairwise comparisons between readers that were within a prespecified minimum acceptable difference. Persistent differences in measurement techniques between laboratories triggered retraining and reassessment of reproducibility. RESULTS: There was robust reproducibility within each ECL but differences between ECLs on calculated LV ejection fraction and mitral inflow velocities (all CPs < 0.80); four-chamber global longitudinal strain bordered acceptable reproducibility (CP = 0.805). Calculated LV ejection fraction and four-chamber global longitudinal strain were sensitive to small but systematic interlaboratory differences in endocardial border definition that influenced measured LV volumes and the speckle-tracking region of interest, respectively. On repeat analyses, reproducibility for mitral velocities (CP = 0.940-0.990) was improved after incorporating multiple-beat measurements and homogeneous image selection. Reproducibility for four-chamber global longitudinal strain was unchanged after efforts to develop consensus between ECLs on endocardial border determinations were limited primarily by a lack of established reference standards. CONCLUSIONS: High-quality quantitative echocardiographic research is feasible but requires a commitment to reproducibility, adherence to guideline recommendations, and the time, care, and attention to detail to establish agreement on measurement conventions. These findings have important implications for research design and clinical care.

Assessment of Device-Related Thrombus and Associated Clinical Outcomes With the WATCHMAN Left Atrial Appendage Closure Device for Embolic Protection in Patients With Atrial Fibrillation (from the PROTECT-AF Trial).

Left atrial appendage closure with the WATCHMAN device is an alternative to anticoagulation for stroke prevention in selected patients with atrial fibrillation (AF). LA device-related thrombus (DRT) is poorly defined and understood. We aimed to (1) develop consensus echocardiographic diagnostic criteria for DRT; (2) estimate the incidence of DRT; and (3) determine clinical event rates in patients with DRT. In phase 1 (training), a training manual was developed and reviewed by 3 echocardiographers with left atrial appendage closure device experience. All available transesophageal (TEE) studies in the WATCHMAN left atrial appendage system for embolic protection in patients with atrial fibrillation (PROTECT-AF) trial patients with suspected DRT were reviewed in 2 subsequent phases. In phase 2 (primary blind read), each reviewer independently scored each study for DRT, and final echo criteria were developed. Unanimously scored studies were considered adjudicated, whereas all others were reevaluated by all reviewers in phase 3 (group adjudication read). DRT was suspected in 35 of 485 patients by the site investigator, the echocardiography core laboratory, or both; 93 of the individual TEE studies were available for review. In phase 2, 3 readers agreed on 67 (72%) of time points. Based on phases 1 and 2, 5 DRT criteria were developed. In phase 3, studies without agreement in phase 2 were adjudicated using these criteria. Overall, at least 1 TEE was DRT positive in 27 (5.7%) PROTECT-AF patients. Stroke, peripheral embolism, or cardiac/unexplained death occurred in subjects with DRT at a rate of 3.4 per 100 patient-years follow-up. In conclusion, DRT were identified on at least 1 TEE in 27 PROTECT-AF patients, indicating a DRT incidence of 5.7%. Primary efficacy events in patients with DRT occurred at a rate of 3.4 per 100 patient-years follow-up, intermediate in frequency between event rates previously reported for the overall device and warfarin arms in PROTECT-AF.

Choice of agreement indices for assessing and improving measurement reproducibility in a core laboratory setting.

Clinical core laboratories, such as Echocardiography core laboratories, are increasingly used in clinical studies with imaging outcomes as primary, secondary, or surrogate endpoints. While many factors contribute to the quality of measurements of imaging variables, an essential step in ensuring the value of imaging data includes formal assessment and control of reproducibility via intra-observer and inter-observer reliability. There are many different agreement/reliability indices in the literature. However, different indices may lead to different conclusions and it is not clear which index is the preferred choice as an overall indication of data quality and a tool for providing guidance on improving quality and reliability in a core lab setting. In this paper, we pre-specify the desirable characteristics of an agreement index for assessing and improving reproducibility in a core lab setting; we compare existing agreement indices in terms of these characteristics to choose a preferred index. We conclude that, among the existing indices reviewed, the coverage probability for assessing agreement is the preferred agreement index on the basis of computational simplicity, its ability for rapid identification of discordant measurements to provide guidance for review and retraining, and its consistent evaluation of data quality across multiple reviewers, populations, and continuous/categorical data.

ECG myocardial scar quantification predicts reverse left ventricular remodeling and survival after cardiac resynchronization therapy implantation: A retrospective pilot study.

INTRODUCTION: Electrocardiographic (ECG) LV scar quantification may improve prediction of CRT response. METHODS AND RESULTS: Data were abstracted in 76 patients who underwent a first CRT implantation at 2 US centers. Selvester QRS scar quantification was performed using the LBBB modified QRS scoring method. Seven clinical variables previously associated with reverse LV remodeling (RLVR) and QRS score were included in logistic regression analysis. Survival was compared across QRS score quartiles using Kaplan-Meier curves. RLVR occurred more frequently in patients with QRS score ≤ 5 (63%) than QRS score>5 (22%), (OR=5.83, 95% CI=2.11-16.07). After adjustment for clinical variables using logistic regression, QRS score>5 predicted RLVR (Chi-square=20.3, P=0.005, AUC=0.782). Patients in the lowest quartile of QRS score (<4) had improved survival compared to patients in the other QRS score quartiles (P=0.037). CONCLUSION: ECG quantified LV scar predicts RLVR and long-term survival in patients with LBBB undergoing CRT implantation.

Quality Improvement Implementation: Improving Reproducibility in the Echocardiography Laboratory.

BACKGROUND: Interpretative variability can adversely affect echocardiographic reliability, but there is no widely accepted method to minimize variability and improve reproducibility. METHODS: A continuous quality improvement process was devised that involves testing reproducibility by assessment of measurement differences followed by robust review, retraining, and retesting. Reproducibility was deemed acceptable if ≥80% of all interreader comparisons were within a prespecified acceptable difference. Readers not meeting this standard underwent retraining and retesting until acceptable reproducibility was achieved for the following parameters: left ventricular end-diastolic volume, biplane ejection fraction, mitral and aortic regurgitation, left ventricular outflow tract diameter, peak and mean aortic valve gradients, and aortic valve area. Eight hundred interreader comparisons for evaluation of reproducibility were generated from five readers interpreting 10 echocardiograms per testing cycle. The applicability and efficacy of this method were then evaluated by testing a second larger group of 10 readers and reevaluating reproducibility 1 year later. RESULTS: All readers demonstrated acceptable reproducibility for biplane ejection fraction, mitral regurgitation, and peak and mean aortic valve gradients. Acceptable reproducibility for left ventricular end-diastolic volume, aortic regurgitation, and aortic valve area was achieved by four of five readers. No readers attained acceptable reproducibility on initial evaluation of left ventricular outflow tract diameter. After review and retraining, all readers demonstrated acceptable reproducibility, which was maintained on subsequent testing 1 year later. A second larger group of 10 readers was also evaluated and yielded similar results. CONCLUSIONS: A continuous quality improvement process was devised that successfully reduced interpretative variability in echocardiography and improved reproducibility that was sustained over time.