Testing the reliability and validity of a set of clinical indicators for evaluation of the quality of urosurgical care in Hospitals

ABSTRACT

The objectives of the present study were to develop a set of clinical indicators for evaluation of quality of urosurgical care in hospitals and to test the reliability and validity of the developed set of indicators. The study was conducted at the two Health Insurance Organisation hospitals in Alexandria. A preliminary set of indicators was selected based on literature review, and consultation with quality assurance physician in the study hospitals. Delphi technique was used to validate this set. Reliability of indicators was assessed in terms of agreement among the investigator, QA physician of study hospital [Ql] and QA physician of the other study hospital [Q2] as well as accuracy of identification of indicator occurrences by different reviewers. The validity of different indicators was assessed in terms of the proportion of identified occurrences by the study reviewers that merits further investigation as determined through independent review of each case of indicator occurrence by the head of urology unit from which the patient was discharged, Ql, and researchers. In case of disagreement among reviewers, the record was submitted to a urosurgery consultant, to obtain his judgement. The results of the present study revealed that response rate in round one Delphi ranged from 61%-71% and 87.5%-100% in round two. Post-operative mortality ranked first as the most useful indicator in both Delphi rounds. The mean of all indicator thresholds was lower in round two than in round one. The overall percentage of unacceptable occurrences for different indicators was 69.6% at one hospital and 74.5% at the other. At both hospitals the observed post-operative mortality and ICU admission rates were below the mean expert threshold, whereas all other indicator rates markedly crossed the mean expert threshold. The overall agreement was in the range 80%-97%, the specific agreement was in the range 71%-95% and Kappa statistic was in the range 0.73-0.96. The accuracy of indicators identification was calculated using sensitivity, and specificity. At both hospitals specificity and sensitivity were highest for the investigator followed by a lower value for both Ql and Q2