Quality assessment of economic evaluations published in PharmacoEconomics. The first four years (1992 to 1995).

Our objective was to assess the quality of reporting of original economic research articles in PharmacoEconomics from inception to the end of 1995, in order to identify areas of strength and weakness, and analyse trends over time. Each regular issue of the journal was examined for original economic evaluations. Accepted articles were categorised by study type and by year of publication. A previously developed 13-item quality-scoring checklist was applied. The maximum possible score that an article could be assigned was 4.0. Quality scores were analysed over time and by study type. 54 articles were identified for analysis. Mean overall score (OS) ranged from a minimum of 1.80 to a maximum of 3.75, with a mean OS of 3.01 [standard deviation (SD) = 0.47]. The item with the highest mean score was the 'definition of study aim' (mean OS = 3.46, SD = 0.69). The item with the lowest score was 'ethical problems discussed and identified' (mean OS = 1.44, SD = 0.92). Only 4 items on the checklist had mean scores lower than 3.0. No significant time trend was apparent for OS (R2 = 0.002). Cost-benefit (mean OS = 3.25, SD = 0.85, n = 5), cost-effectiveness (mean OS = 3.11, SD = 0.97, n = 27), and cost-utility (mean OS = 3.29, SD = 0.93, n = 6) analyses had mean scores significantly higher than cost-analysis/cost-of-illness studies (mean OS = 2.51, SD = 1.14, n = 8). The mean OS for cost-minimisation studies was 2.74 (SD = 0.49, n = 8). Despite some weaknesses in particular aspects of economic evaluations published in PharmacoEconomics, we conclude that the journal has offered publications with acceptable overall quality and adequate methodology.

Effect of neonatal circumcision on pain response during subsequent routine vaccination.

BACKGROUND: Preliminary studies suggested that pain experienced by infants in the neonatal period may have long-lasting effects on future infant behaviour. The objectives of this study were to find out whether neonatal circumcision altered pain response at 4-month or 6-month vaccination compared with the response in uncircumcised infants, and whether pretreatment of circumcision pain with lidocaine-prilocaine cream (Emla) affects the subsequent vaccination response. METHODS: We used a prospective cohort design to study 87 infants. The infants formed three groups--uncircumcised infants, and infants who had been randomly assigned Emla or placebo in a previous clinical trial to assess the efficacy of Emla cream as pretreatment for pain in neonatal circumcision. Infants were videotaped during vaccination done at the primary care physician's clinic. Videotapes were scored without knowledge of circumcision or treatment status by a research assistant who had been trained to measure infant facial action, cry duration, and visual analogue scale pain scores. FINDINGS: Birth characteristics and infant characteristics at the time of vaccination, including age and temperament scores, did not differ significantly among groups. Multivariate ANOVA revealed a significant group effect (p < 0.001) in difference (vaccination minus baseline) values for percentage facial action, percentage cry time, and visual analogue scale pain scores. Univariate ANOVAs were significant for all outcome measures (p < 0.05): infants circumcised with placebo had higher difference scores than uncircumcised infants for percentage facial action (136.9 vs 77.5%), percentage cry duration (53.8 vs 24.7%), and visual analogue scale pain scores (5.1 vs 3.1 cm). There was a significant linear trend on all outcome measures, showing increasing pain scores from uncircumcised infants, to those circumcised with Emla, to those circumcised with placebo. INTERPRETATION: Circumcised infants showed a stronger pain response to subsequent routine vaccination than uncircumcised infants. Among the circumcised group, preoperative treatment with Emla attenuated the pain response to vaccination. We recommend treatment to prevent neonatal circumcision pain.

Sensitivity analysis in health economic and pharmacoeconomic studies. An appraisal of the literature.

The objective of this study was to analyse the extent of reporting of sensitivity analyses in the health economics, medical and pharmacy literature between journal types and over time. 90 articles were chosen from each of the bodies of literature on health economics, medicine and pharmacy. MEDLINE, EMBASE and International Pharmaceutical Abstracts were searched for English-language economic studies published between 1989 and 1993. The studies chosen for inclusion had to be original articles published in one of the selected journals between January 1989 and December 1993, involving a comparison between drugs, treatments or services, and evaluating both costs and outcomes. 123 articles initially met these criteria; however, 16 were inappropriate, 17 were randomised out, leaving 90 studies (73%) that were used (30 from each literature group). Data were extracted independently by 5 raters using a validated checklist. Inter-rater reliability was assessed by calculating kappa. 53 of the 90 articles (59%) conducted sensitivity analyses. 39 (74%) stated explicitly that a sensitivity analysis was being performed; this was noted in the Methods section of 35 papers (67%). 80% of health economics journals, 70% of medical journals and 20% of pharmacy journals conducted sensitivity analyses. Despite the fact that all published pharmacoeconomic guidelines suggest the use of sensitivity analysis, only 59% of studies between 1989 and 1993 did so. Improvement is required, especially in the pharmacy literature. No time trends in the conduct of sensitivity analyses were detected. However, the sample may not have been sufficient to detect such trends. Pharmacoeconomic guidelines should provide more details on preferred methods of sensitivity analysis and on desired parameters.

Meta-analysis of parenteral clindamycin dosing regimens.

OBJECTIVE: We used meta-analysis to compare clinical cure and success rates for parenteral clindamycin 600 mg q8h or 900 mg q8h therapy to treat adult intraabdominal or female pelvic infections. DATA SOURCES: We located English-language articles describing clindamycin use in humans using MEDLINE, International Pharmaceutical Abstracts, and Embase and from personal and drug information center files, plus all article references. STUDY SELECTION: Eligible studies used parenteral clindamycin 600 mg q8h or 900 mg q8h to treat intraabdominal or pelvic infection in at least 1 arm of a study and provided a definition of clinical outcome. Accepted were comparative trials in adults who were not critically ill or expected to die. DATA SYNTHESIS: The DerSimonian and Laird method was used to calculate weighted overall success rates for cure and success (cure plus improved) rates along with 95% confidence intervals for each dosage in intraabdominal and pelvic infections. Regimens were compared with respect to both cure and success rates using the Mann-Whitney U test. MAIN RESULTS: Twenty-three articles were eligible for inclusion. Abdominal cure rates were 75.6% and 90.5% for clindamycin 600 mg q8h and 900 mg q8h, respectively (p = 0.03): success rates were 89.8% and 92.5%, respectively (p = 0.29). Pelvic cure rates were 82.8% and 89.4%, respectively (p = 0.51): success rates were 87.2% and 89.9%, respectively (p = 0.51). CONCLUSIONS: In pelvic infections, a dosage of clindamycin 600 mg q8h appears to be clinically acceptable for all patients. Although clinical outcomes for intraabdominal infections are generally similar for both regimens, the significantly higher cure rate with a dosage of clindamycin 900 mg q8h suggests that dosage recommendations should be patient specific.

Quality of nonstructured and structured abstracts of original research articles in the British Medical Journal, the Canadian Medical Association Journal and the Journal of the American Medical Association.

OBJECTIVE: To assess and compare the quality of nonstructured and structured abstracts of original research articles in three medical journals. DESIGN: Blind, criterion-based observational study. SAMPLE: Random sample of 300 abstracts (25 abstracts per journal each year) of articles published in the British Medical Journal (BMJ), the Canadian Medical Association Journal and the Journal of the American Medical Association (JAMA) in 1988 and 1989 (nonstructured abstracts) and in 1991 and 1992 (structured abstracts). MAIN OUTCOME MEASURES: The quality of abstracts was measured against 33 objective criteria, which were divided into eight categories (purpose, research design, setting, subjects, intervention, measurement of variables, results and conclusions). The quality score was determined by dividing the number of criteria present by the number applicable; the score varied from 0 to 1. RESULTS: The overall mean quality scores for nonstructured and structured abstracts were 0.57 and 0.74 respectively (p < 0.001). The frequency in meeting the specific criteria was generally higher for the structured abstracts than for the nonstructured ones. The mean quality score was higher for nonstructured abstracts in JAMA than for those in BMJ (0.60 v. 0.54, p < 0.05). The scores for structured abstracts did not differ significantly between the three journals. CONCLUSIONS: The findings support recommendations that promote the use of structured abstracts. Further studies should be performed to assess the effect of time on the quality of abstracts and the extent to which abstracts reflect the content of the articles.

A pilot study of process and outcome assessment in antibiotic therapy.

A quality assurance survey of cefazolin therapy was conducted by pharmacists using process-related and outcome-related assessments. The purpose of this survey was to study the possibility of having pharmacists review and categorize the appropriateness and success of antibiotic therapy. During a three week period, 168 orders for cefazolin were identified and 67 prophylactic and medical therapies were selected and submitted for possible pharmacist review. Thirty-seven therapies were reviewed by staff pharmacists who scored each therapy for the acceptability of risk of adverse drug effect, the cost-effectiveness, and the overall appropriateness. An evaluation form was used, but explicit utilization criteria were not provided. The average scores (+/- SD) on a 10 centimeter visual analog scale were 9.1 (+/- 0.71), 8.7 (+/- 1.21), and 8.8. +/- 0.79) respectively. Twenty-six (70%) of these therapies were monitored to resolution, and 24 (65%) were successful in achieving the therapeutic goal. No adverse effects were noted. The average estimated times to complete the initial review and follow-up review were 10.1 (+/- 5.60) and 3.5 (+/- 2.29) minutes respectively, less than the 19.5 minutes estimated using the Canadian Hospital Pharmacy Workload Measurement System. This survey demonstrated that pharmacists can provide both process-related and outcome-related QA data.

Quality of abstracts of original research articles in CMAJ in 1989.

PURPOSE: To evaluate the quality of abstracts of original research articles. DESIGN: Blind, criterion-based survey. SAMPLE: Systematic sample of 33 abstracts of original research articles published in CMAJ in 1989. MEASUREMENT: The quality of abstracts was measured against a checklist of evaluation criteria, which were divided into eight categories. A score for each abstract was obtained by dividing the number of criteria present by the number applicable. The overall mean score was also determined. RESULTS: The overall mean score of abstract quality was 0.63 (standard deviation 0.13) out of 1. Of the abstracts reporting study design 56% did not include specific technical descriptors. About 52% did not explicitly describe the study variables. In describing subject selection 79% failed to use specific technical terms. Of the abstracts reporting results 66% did not provide appropriate supporting data. Of those that gave conclusions 86% did not address study limitations and 93% made no recommendations for future study. CONCLUSION: Most of the abstracts provided some information pertaining to each evaluation criterion but did not provide detail sufficient to enhance the reader's understanding of the article. On the basis of the study sample the abstracts need improvement in description of research design, reporting of subject selection and results, and statements of limitations and recommendations. The small sample from one journal and the absence of comparison between the contents of the abstracts and the contents of the articles were limitations. Future studies should address these issues and compare the quality of traditional and structured abstracts.

Workload measurement in long term care pharmacy.

This paper describes the implementation of the Canadian Hospital Pharmacy Workload Measurement System (CHPWMS) by the pharmacy department of a 475-bed long term care facility. The CHPWMS description of a computerized individual patient prescription service provided the basis for the implementation. Following an initial appraisal of similarities between chronic care pharmacy services and the services outlined by CHPWMS, a program was developed and evaluated with the assistance of a pharmacist consultant. During a four-month trial all measurable work was recorded in a process that separated reportable (CHPWMS-defined) activities from other work activities. Eighty percent of the department's workload was accountable as reportable units using standard times. Nonreportable workload was documented using actual times, and adapted standard times. Dispensing accounted for the largest proportion of workload (79%). Manufacturing was the most frequently performed nonreportable activity. Pharmacist workload supported the desired level of 50% nondispensing activities. The productivity index of available worked hours was 70%. The consistent pattern of the recording suggested staff commitment. The program has been used to justify and monitor staffing levels.

Direction through accreditation--CCHA (Canadian Council on Hospital Accreditation) recommendations to Ontario hospital pharmacies.

A questionnaire was circulated in December 1986 to 157 Ontario hospital pharmacy directors that asked about the recent recommendations made by CCHA surveyors. Sixty-four directors responded and assigned their received recommendations to four categories of pharmacy service. Similar recommendations were tabulated by category of service and by number of hospital beds (less than 200 and greater than 200 beds). The results show that drug distribution was the most emphasized area of service cited for improvement. Frequent recommendations concerned patient medication profiles, night supply services, hospital formulary, Pharmacy and Therapeutics Committee policy review, adverse drug reaction programs, quality assurance, and regular medication room checks. Responses indicated that 27 of the 62 eligible hospital pharmacies (44%) that received recommendations have acquired administrative support to implement changes, and 33 of 62 (53%) have had specific action as a result of an accreditation survey.