Standardized problem list generation, utilizing the Mayo canonical vocabulary embedded within the Unified Medical Language System.

UNLABELLED: VOCABULARY: The Mayo problem list vocabulary is a clinically derived lexicon created from the entries made to the Mayo Clinic's Master Sheet Index and the problem list entries made to the Impression/ Report/Plan section of the Clinical Notes System over the last three years. The vocabulary was reduced by eliminating repetition including lexical variants, spelling errors, and qualifiers (Administrative or Operational terms). Qualifiers are re-coordinated with other terms, at run-time, which greatly increased the number of input strings which our system is capable of recognizing. IMPLEMENTATION: The Problem Manager is implemented using standard windows tools in a Windows NT environment. The interface is designed using Object Pascal. HTTP calls are passed over the World Wide Web to a UNIX based vocabulary server. The server returns a document, which is read into Object Pascal structures, parsed, filtered and displayed. STUDY: This paper reports the results of a recent Usability Trial focused on assessing the viability of this mechanism for standardized problem entry. Eight clinicians engaged in eleven scenarios and responded as to their satisfaction with the systems performance. These responses were observed, videotaped and tabulated. Clinicians in this study were able to find acceptable diagnoses in 91.1% of the scenarios. The response time was acceptable in 92.5% of the scenarios. The presentation of related terms was stated to be useful in at least one scenario by seven of the eight participants. All clinicians wanted to make use of shortcuts which would minimize the amount of typing necessary to encode the concept they were searching for (e.g. Abbreviations, Word Completion). CONCLUSIONS: Clinicians are willing to choose a canonical term from a suggested list (as opposed to their own wording). Clinicians want an "intelligent" system, which would suggest terms within a category (e.g. Types of "Migraine"). They are able to make functional use of our system, in its current state of development. Finally, all clinicians appreciate the value of encoding their problems in a standardized vocabulary, toward improved research, education and practice.

A randomized controlled trial of a computer-based physician workstation in an outpatient setting: implementation barriers to outcome evaluation.

OBJECTIVE: A research prototype Physician Workstation (PWS) incorporating a graphical user interface and a drug ordering module was compared with the existing hospital information system in an academic Veterans Administration General Medical Clinic. Physicians in the intervention group received recommendations for drug substitutions to reduce costs and were alerted to potential drug interactions. The objective was to evaluate the effect of the PWS on user satisfaction, on health-related outcomes, and on costs. DESIGN: A one-year, two-period, randomized controlled trial with 37 subjects. MEASUREMENTS: Differences in the reliance on noncomputer sources of information, in user satisfaction, in the cost of prescribed medications, and in the rate of clinically relevant drug interactions were assessed. RESULTS: The study subjects logged onto the workstation an average of 6.53 times per provider and used it to generate 2.8% of prescriptions during the intervention period. On a five-point scale (5 = very satisfied, 1 = very dissatisfied), user satisfaction declined in the PWS group (3.44 to 2.98 p = 0.008), and increased in the control group (3.23 to 3.72, p < 0.0001). CONCLUSION: The intervention physicians did not use the PWS frequently enough to influence information-seeking behavior, health outcomes, or cost. The study design did not determine whether the poor usage resulted from satisfaction with the control system, problems using the PWS intervention, or the functions provided by the PWS intervention. Evaluative studies should include provisions to improve the chance of successful implementation as well as to yield maximum information if a negative study occurs.

A randomized evaluation of a computer-based physician's workstation: design considerations and baseline results.

We are performing a randomized, controlled trial of a Physician's Workstation (PWS), an ambulatory care information system, developed for use in the General Medical Clinic (GMC) of the Palo Alto VA. Goals for the project include selecting appropriate outcome variables and developing a statistically powerful experimental design with a limited number of subjects. As PWS provides real-time drug-ordering advice, we retrospectively examined drug costs and drug-drug interactions in order to select outcome variables sensitive to our short-term intervention as well as to estimate the statistical efficiency of alternative design possibilities. Drug cost data revealed the mean daily cost per physician per patient was 99.3 cents +/- 13.4 cents, with a range from 0.77 cent to 1.37 cents. The rate of major interactions per prescription for each physician was 2.9% +/- 1%, with a range from 1.5% to 4.8%. Based on these baseline analyses, we selected a two-period parallel design for the evaluation, which maximized statistical power while minimizing sources of bias.

Semantic integration of information in a physician's workstation.

Patient care is an information-intensive activity, yet physicians have few tools to effectively access and manage patient data. We studied physicians' information needs in an outpatient clinic, and developed a prototype physician's workstation (PWS) to address those needs. The PWS provides integrated access to patient information and uses embedded domain knowledge to enhance the presentation of clinical information to the physician. All the applications in the PWS share a common patient context, defined by the state of the internal patient model--semantic integration. Relevant data are presented together and higher-order alerts are generated by combining notable events with relevant data from the patient context. Semantic integration allows us to present and to operate on all patient data in a given patient's context, significantly enhancing the effectiveness with which information is presented to the physician.

Implementing a Physician's Workstation using client/server technology and the distributed computing environment.

PWS is a physician's workstation research prototype developed to explore the use of information management tools by physicians in the context of patient care. The original prototype was implemented in a client/server architecture using a broadcast message server. As we expanded the scope of the prototyping activities, we identified the limitations of the broadcast message server in the areas of scalability, security, and interoperability. To address these issues, we reimplemented PWS using the Open Software Foundation's Distributed Computing Environment (DCE). We describe the rationale for using DCE, the migration process, and the benefits achieved. Future work and recommendations are discussed.

An evaluation of explanations of probabilistic inference.

Providing explanations of the conclusions of decision-support systems can be viewed as presenting inference results in a manner that enhances the user's insight into how these results were obtained. The ability to explain inferences has been demonstrated to be an important factor in making medical decision-support systems acceptable for clinical use. Although many researchers in artificial intelligence have explored the automatic generation of explanations for decision-support systems based on symbolic reasoning, research in automated explanation of probabilistic results has been limited. We present the results of an evaluation study of INSITE, a program that explains the reasoning of decision-support systems based on Bayesian belief networks. In the domain of anesthesia, we compared subjects who had access to a belief network with explanations of the inference results to control subjects who used the same belief network without explanations. We show that, compared to control subjects, the explanation subjects demonstrated greater diagnostic accuracy, were more confident about their conclusions, were more critical of the belief network, and found the presentation of the inference results more clear.

Automated identification of relevant patient information in a physician's workstation.

The introduction of computer-based patient records offers an opportunity to improve the ability of physicians to browse the medical record and to monitor patient events. We describe a methodology to identify relevant information in the patient record. This methodology combines a patient-specific physiological model with functions to determine relevance of patient information. The model consists of a qualitative representation of physiological parameters and influences, custom-tailored to a particular patient's problems, medications, and test results. We describe two applications of this model in the context of an integrated physician's workstation: automatic linking of relevant patient information for configuration of user-interface displays, and monitoring of patient events to prevent oversight of noteworthy information.

An evaluation of explanations of probabilistic inference.

Providing explanations of the conclusions of decision-support systems can be viewed as presenting inference results in a manner that enhances the user's insight into how these results were obtained. The ability to explain inferences has been demonstrated to be an important factor in making medical decision-support systems acceptable for clinical use. Although many researchers in artificial intelligence have explored the automatic generation of explanations for decision-support systems based on symbolic reasoning, research in automated explanation of probabilistic results has been limited. We present the results of an an evaluation study of INSITE, a program that explains the reasoning of decision-support systems based on Bayesian belief networks. In the domain of anesthesia, we compared subjects who had access to a belief network with explanations of the inference results, to control subjects who used the same belief network without explanations. We show that, compared to control subjects, the explanation subjects demonstrated greater diagnostic accuracy, were more confident about their conclusions, were more critical of the belief network, and found the presentation of the inference results more clear.