Monitoring compliance with standards of care for chronic diseases using healthcare administrative databases in Italy: Strengths and limitations.

BACKGROUND: A recent comprehensive report on healthcare quality in Italy published by the Organization of Economic Co-operation and Development (OECD) recommended that regular monitoring of quality of primary care by means of compliance with standards of care for chronic diseases is performed. A previous ecological study demonstrated that compliance with standards of care could be reliably estimated on regional level using administrative databases. This study compares estimates based on administrative data with estimates based on GP records for the same persons, to understand whether ecological fallacy played a role in the results of the previous study. METHODS: We compared estimates of compliance with diagnostic and therapeutic standards of care for type 2 diabetes (T2DM), hypertension and ischaemic heart disease (IHD) from administrative data (IAD) with estimates from medical records (MR) for the same persons registered with 24 GP's in 2012. Data were linked at an individual level. RESULTS: 32,688 persons entered the study, 12,673 having at least one of the three diseases according to at least one data source. Patients not detected by IAD were many, for all three conditions: adding MR increased the number of cases of T2DM, hypertension, and IHD by +40%, +42%, and +104%, respectively. IAD had imperfect sensitivity in detecting population compliance with therapies (adding MR increased the estimate, from +11.5% for statins to +14.7% for antithrombotics), and, more substantially, with diagnostic recommendations (adding MR increased the estimate, from +23.7% in glycated hemoglobin tests, to +50.5% in electrocardiogram). Patients not detected by IAD were less compliant with respect to those that IAD correctly identified (from -4.8 percentage points in proportion of IHD patients compliant with a yearly glycated hemoglobin test, to -40.1 points in the proportion of T2DM patients compliant with the same recommendation). IAD overestimated indicators of compliance with therapeutic standards (significant differences ranged from 3.3. to 3.6 percentage points) and underestimated indicators of compliance with diagnostic standards (significant differences ranged from -2.3 to -14.1 percentage points). CONCLUSION: IAD overestimated the percentage of patients compliant with therapeutic standards by less than 6 percentage points, and underestimated the percentage of patients compliant with diagnostic standards by a maximum of 14 percentage points. Therefore, both discussions at local level between GP's and local health unit managers and discussions at central level between national and regional policy makers can be informed by indicators of compliance estimated by IAD, which, based on those results, have the ability of signalling critical or excellent clusters. However, this study found that estimates are partly flawed, because a high number of patients with chronic diseases are not detected by IAD, patients detected are not representative of the whole population of patients, and some categories of diagnostic tests are markedly underrecorded in IAD (up to 50% in the case of electrocardiograms). Those results call to caution when interpreting IAD estimates. Audits based on medical records, on the local level, and an interpretation taking into account information external to IAD, on the central level, are needed to assess a more comprehensive compliance with standards.

Automatic identification of type 2 diabetes, hypertension, ischaemic heart disease, heart failure and their levels of severity from Italian General Practitioners' electronic medical records: a validation study.

OBJECTIVES: The Italian project MATRICE aimed to assess how well cases of type 2 diabetes (T2DM), hypertension, ischaemic heart disease (IHD) and heart failure (HF) and their levels of severity can be automatically extracted from the Health Search/CSD Longitudinal Patient Database (HSD). From the medical records of the general practitioners (GP) who volunteered to participate, cases were extracted by algorithms based on diagnosis codes, keywords, drug prescriptions and results of diagnostic tests. A random sample of identified cases was validated by interviewing their GPs. SETTING: HSD is a database of primary care medical records. A panel of 12 GPs participated in this validation study. PARTICIPANTS: 300 patients were sampled for each disease, except for HF, where 243 patients were assessed. OUTCOME MEASURES: The positive predictive value (PPV) was assessed for the presence/absence of each condition against the GP's response to the questionnaire, and Cohen's κ was calculated for agreement on the severity level. RESULTS: The PPV was 100% (99% to 100%) for T2DM and hypertension, 98% (96% to 100%) for IHD and 55% (49% to 61%) for HF. Cohen's kappa for agreement on the severity level was 0.70 for T2DM and 0.69 for hypertension and IHD. CONCLUSIONS: This study shows that individuals with T2DM, hypertension or IHD can be validly identified in HSD by automated identification algorithms. Automatic queries for levels of severity of the same diseases compare well with the corresponding clinical definitions, but some misclassification occurs. For HF, further research is needed to refine the current algorithm.