A randomized controlled trial of a computer-based physician workstation in an outpatient setting: implementation barriers to outcome evaluation.

OBJECTIVE: A research prototype Physician Workstation (PWS) incorporating a graphical user interface and a drug ordering module was compared with the existing hospital information system in an academic Veterans Administration General Medical Clinic. Physicians in the intervention group received recommendations for drug substitutions to reduce costs and were alerted to potential drug interactions. The objective was to evaluate the effect of the PWS on user satisfaction, on health-related outcomes, and on costs. DESIGN: A one-year, two-period, randomized controlled trial with 37 subjects. MEASUREMENTS: Differences in the reliance on noncomputer sources of information, in user satisfaction, in the cost of prescribed medications, and in the rate of clinically relevant drug interactions were assessed. RESULTS: The study subjects logged onto the workstation an average of 6.53 times per provider and used it to generate 2.8% of prescriptions during the intervention period. On a five-point scale (5 = very satisfied, 1 = very dissatisfied), user satisfaction declined in the PWS group (3.44 to 2.98 p = 0.008), and increased in the control group (3.23 to 3.72, p < 0.0001). CONCLUSION: The intervention physicians did not use the PWS frequently enough to influence information-seeking behavior, health outcomes, or cost. The study design did not determine whether the poor usage resulted from satisfaction with the control system, problems using the PWS intervention, or the functions provided by the PWS intervention. Evaluative studies should include provisions to improve the chance of successful implementation as well as to yield maximum information if a negative study occurs.

A randomized evaluation of a computer-based physician's workstation: design considerations and baseline results.

We are performing a randomized, controlled trial of a Physician's Workstation (PWS), an ambulatory care information system, developed for use in the General Medical Clinic (GMC) of the Palo Alto VA. Goals for the project include selecting appropriate outcome variables and developing a statistically powerful experimental design with a limited number of subjects. As PWS provides real-time drug-ordering advice, we retrospectively examined drug costs and drug-drug interactions in order to select outcome variables sensitive to our short-term intervention as well as to estimate the statistical efficiency of alternative design possibilities. Drug cost data revealed the mean daily cost per physician per patient was 99.3 cents +/- 13.4 cents, with a range from 0.77 cent to 1.37 cents. The rate of major interactions per prescription for each physician was 2.9% +/- 1%, with a range from 1.5% to 4.8%. Based on these baseline analyses, we selected a two-period parallel design for the evaluation, which maximized statistical power while minimizing sources of bias.