Assessing the performance of indicators during their life cycle: the mixed QUID method.

BACKGROUND: Quality indicators (QI) are mandatory in French hospitals. After a decade of use, the Ministry of Health set up an expert workgroup to enhance informed decision-making regarding currently used national QI, i.e. to propose a decision of withdrawing, revising or continuing their use. We report the development of an integrated method for a comprehensive appraisal of quality/safety indicators (QI) during their life cycle, for three purposes, quality improvement, public disclosure and regulation purposes. The method was tested on 10 national QI on use for up to 10 years to identify operational issues. METHODS: A modified Delphi technique to select relevant criteria and a development of a mixed evaluation method by the workgroup. A 'real-life' test on 10 national QI. RESULTS: Twelve criteria were selected for the appraisal of QI used for regulation goals, 11 were selected for hospital improvement and seven for public disclosure. The perceived feasibility and relevance were studied including hospital workers, patients and health authorities professionals; the scientific soundness of the indicator development phase was reviewed by analyzing reference documents; the metrological performance (limited to the discriminatory power and dynamics of change during the life cycle dimensions) was analyzed on the national datasets.Applied to the 10 QI, the workgroup proposed to withdraw four of them and to modify or suspend the six others. CONCLUSIONS: The value of the method was supported by the clear-cut conclusions and endorsement of the proposed decisions by the health authorities.

Organizational and managerial factors associated with clinical practice guideline adherence: a simulation-based study in 36 French hospital wards.

OBJECTIVES: To identify managerial and organizational characteristics of multi-specialty medicine wards and individual characteristics of health professionals that are most strongly associated with clinical practice guidelines (CPG) adherence. DESIGN: Cross-sectional stratified cluster sample design. SETTING: Data were gathered from 36 randomly selected multi-specialty medicine wards. PARTICIPANTS: The study population included all health professionals involved in patient care working in the participating wards. MAIN OUTCOME MEASURES: The degree of CPG adherence was measured using clinical vignettes on three topics: pain management, managing heart failure and managing diabetes. Responses from each professional to each clinical case were quantified using a 10-point scale. Managerial and organizational characteristics of medical department and individual characteristics of health professionals were obtained using three questionnaires. RESULTS: The study sample consisted of 859 professionals (362 orderlies, 361 nurses and 136 physicians). Factors independently and positively associated with CPG adherence were (i) individual factors: low age of professionals, expertise in diabetology and activity in cardiology; (ii) organizational and managerial factors: good understanding between physicians and other personnel; and (iii) structural factors: computer-based test results and prescriptions, presence of medical specialists, inter-department mobility of orderlies, medium-length stay (between 7 and 10 days) and large bed capacity. CONCLUSIONS: Good CPG adherence in general medicine needs institutional dynamism, availability of clinical competence and team culture based on cooperation.

How to limit the burden of data collection for Quality Indicators based on medical records? The COMPAQH experience.

BACKGROUND: Our objective was to limit the burden of data collection for Quality Indicators (QIs) based on medical records. METHODS: The study was supervised by the COMPAQH project. Four QIs based on medical records were tested: medical record conformity; traceability of pain assessment; screening for nutritional disorders; time elapsed before sending copy of discharge letter to the general practitioner. Data were collected by 6 Clinical Research Assistants (CRAs) in a panel of 36 volunteer hospitals and analyzed by COMPAQH. To limit the burden of data collection, we used the same sample of medical records for all 4 QIs, limited sample size to 80 medical records, and built a composite score of only 10 items to assess medical record completeness. We assessed QI feasibility by completing a grid of 19 potential problems and evaluating time spent. We assessed reliability (kappa coefficient) as well as internal consistency (Cronbach alpha coefficient) in an inter-observer study, and discriminatory power by analysing QI variability among hospitals. RESULTS: Overall, 23 115 data items were collected for the 4 QIs and analyzed. The average time spent on data collection was 8.5 days per hospital. The most common feasibility problem was misunderstanding of the item by hospital staff. QI reliability was good (kappa: 0.59-0.97 according to QI). The hospitals differed widely in their ability to meet the quality criteria (mean value: 19-85%). CONCLUSION: These 4 QIs based on medical records can be used to compare the quality of record keeping among hospitals while limiting the burden of data collection, and can therefore be used for benchmarking purposes. The French National Health Directorate has included them in the new 2009 version of the accreditation procedure for healthcare organizations.

[Quality and outcome assessment in gastrointestinal endoscopy: study of 202 colonoscopies]. / Evaluation de la qualité des soins en endoscopie digestive: étude de 202 coloscopies totales consécutives.

AIM: The aim of this study was to evaluate the quality of the medical care associated with the colonoscopies performed in our gastrointestinal endoscopy unit. This 6-month prospective, comparative study used process and result indicators. Its long-term objective was to improve the quality of care and to select a few pertinent quality indicators for continual monitoring. METHODS: An audit was conducted of all total colonoscopies performed at our general hospital from November 2005 through May 2006. It assessed the principal published process and result indicators for the medical aspects of the procedure as well as patient satisfaction (indication, procedure itself, results, complications, satisfaction). These indicators were also compared between endoscopists. RESULTS: During the 6-month study period, 202 total colonoscopies were performed. Waiting time for the procedure, appropriateness of the indication, proportion of colonoscopies completed, proportion of neoplastic lesions found, adequacy of bowel preparation, frequency of complications, and patient satisfaction were similar to and even above the standard values. However, patient information and the completeness of the legal documents were insufficient. Although the mean values were satisfactory, detailed individual analysis showed statistically significant differences between endoscopists. For example, the proportion of withdrawal times exceeding 6 minutes was 16.3%, 25.0% and 86.4% according to endoscopist. The mean number of polyps each removed per procedure was respectively 1, 2, and 3 (p<0.05). An improvement plan was implemented in view of these results, and appropriate simple indicators were selected for prospective monitoring. CONCLUSION: Despite the overall results, which essentially complied with standard guidelines, our prospective and comparative audit of colonoscopies pointed out significant variations between individual endoscopists and helped us to define improvement actions and indicators.

Cost-minimization analysis of a wide-area teleradiology network in a French region.

OBJECTIVE: The objective of our study was to perform a cost-minimization analysis of a wide-area teleradiology network. DESIGN: A prospective analysis of all transmissions over 1 year (data transmitted at the time of the remote consultation, and health outcomes of patients from medical record). INTERVENTION: The inter-hospital teleradiology network of the Aquitaine area (RIHRA) is a telemedicine system enabling the management of remote emergencies and elective radiology consultations. MAIN OUTCOME MEASURE: A cost-minimization study enabled a comparison of care procedures following the use of the network with those which would have been implemented without the network. The outcome measures of effectiveness were the transfers, hospitalizations, and consultations avoided or added. Fixed and variable costs were estimated. RESULTS: Among the 664 transmissions included in the study, 562 (85%) were performed in emergency and 102 (15%) for elective (non-emergency) cases. In emergency, 48% of transfers were avoided. For elective teleconsultations, a transfer was avoided for 37% of the patients and hospitalization for 12%. An extra consultation occurred after remote consultation for 2% of the patients. Annual saving can be estimated at 102,779 EUR for the Aquitaine area. CONCLUSIONS: This study underlines the efficiency of an inter-hospital teleradiology network. A qualitative evaluation of the impact of the use of the system should be carried out to improve technical and organizational operations.

Evaluation of the benefits for the patient arising from an inter-hospital teleradiology network in a French administrative area.

We conducted a prospective study of the teleradiology network which connects 15 hospitals in the Aquitaine area. All transmissions sent over a one-year period were examined (data transmitted at the time of the remote consultation and health outcomes of patients from their medical records). For emergency cases, the main outcome measure of effectiveness was the proportion of avoided transfers. For non-emergency cases, the main outcome measure of effectiveness was the proportion of transfers, hospitalizations and consultations avoided. There were 737 transmissions, of which 664 (90%) met the inclusion criteria. Of these, 562 (85%) were for emergency care and 102 (15%) for non-emergency care. In emergency care, the pathologies most often associated with a remote consultation were cerebral pathologies (88%) and traumatic spinal pathologies (8%); the proportion of avoided transfers was 48%. In non-emergency care, the specialties most often concerned with remote consultations were neurology/neurosurgery (36%), cardiology and pulmonary diseases (17%) and gastroenterology (14%). Transfer was avoided for 37% of the patients and hospitalization for 12%. An additional consultation occurred after remote consultation for 2% of the patients. The results confirm the effectiveness of an inter-hospital teleradiology network.

Results of the first 100 accreditation procedures in France.

PURPOSE: To identify the areas of needed improvement that were most frequently identified in the first 100 accredited hospitals by the French Accreditation College (FAC) according to the standards manual. To compare the outcomes of accreditation procedures according to the status and size of the accredited hospitals. DATA SOURCES: We analyzed the first 100 summaries of accreditation reports available on the website of the Agence Nationale d'Accréditation et d'Evaluation en Santé (ANAES). DATA EXTRACTION: Data were collected on hospitals, accreditation processes, and outcomes (decisions of the FAC). For each decision, we assessed the relationship with accreditation manual criteria, and analyzed their distribution by chapter. RESULTS: of data synthesis. Among the 100 accredited hospitals (40 public, 43 private, 17 mixed), nine were accredited without recommendations for improvement, 47 with recommendations, 40 with reservations, and four with major reservations. All of them received requests for improvement. No significant difference was found concerning the FAC decisions according to status and size of hospitals, although there was a trend that the larger the hospital, the more numerous and more serious the decisions of the FAC. The main topics addressed by decisions were those given high priority by the FAC (information given to patients and its traceability on patient records, and signing of prescriptions for medication). CONCLUSION: Despite wide heterogeneity in the summaries on accreditation and in FAC decision-making, this study provides an initial insight into common quality defects and ANAES priorities for hospitals in France.

A multicenter cluster randomized controlled trial of strategies to improve thyroid function testing.

OBJECTIVES: This project aimed to compare the independent and combined effectiveness of two implementation interventions of guidelines for ordering thyroid function tests: a Memorandum Pocket Card (MPC) and a Test Request Form (TRF). RESEARCH DESIGN: Intervention groups were wards. The study used an experimental 2*2 factorial design with matching hospitals according to size and activity and wards according to preintervention appropriateness for test ordering. Four ward groups were established: the dual intervention group, the order form group, the pocket card group and the control group. Physicians in all groups received guidelines and were invited to a local information meeting. MEASURES: The main outcome measure of effectiveness was the Guideline Conformity Rate (GCR). RESULTS: Six hospitals participated in the study (two middle-sized hospitals, two small-sized hospitals and two psychiatric hospitals). A total of 1412 orders for thyroid function tests were collected. GCR was 78% in the dual intervention group, 83% in the order form group, 73% in the pocket card group and 62% in the control group. The interaction between TRF and MPC was not significant (beta = -0.70; P = 0.21). Compared with simple information, TRF was effective in increasing GCR (OR, 2.65; 95% CI, 1.52-4.62), unlike MPC (OR, 1.28; CI, 0.75-2.19). CONCLUSIONS: Using a robust design, our study shows the greater effectiveness of TRF than MPC and their association in implementing thyroid function test guidelines.