Medical student database development: a model for record management in a multi-departmental setting.

Student records flow through medical school offices at a rapid rate. Much of this data is often tracked on paper, spread across multiple departments. The Medical Student Informatics Group at the University of Utah School of Medicine identified offices and organizations documenting student information. We assessed departmental needs, identified records, and researched database software available within the private sector and academic community. Although a host of database applications exist, few publications discuss database models for storage and retrieval of student records. We developed and deployed an Internet based application to meet current requirements, and allow for future expandability. During a test period, users were polled regarding utility, security, stability, ease of use, data accuracy, and potential project expansion. Feedback demonstrated widespread approval, and considerable interest in additional feature development. This experience suggests that many medical schools would benefit from centralized database management of student records.

Use of Iliad to improve diagnostic performance of nurse practitioner students.

Nurse practitioners (NPs) have dual goals as primary care providers, combining the traditional goals of nursing with extended goals as diagnosticians. Diagnostic reasoning, therefore, is a critical component of NP education. Iliad, a computerized diagnostic reasoning expert system, has been used effectively to teach diagnostic skills to medical students. A pilot study was undertaken to determine the effects of Iliad training on NP students' diagnostic skill performance and to identify technical and instructional issues of implementation. The study found that the use of Iliad improved NP students' diagnostic reasoning, and that the training effects were modified by prior nursing experience. Successful use of Iliad required planning, faculty commitment, and technical support.

Department of Veterans Affairs, University of Utah consortium participation in the NLM/AHCPR Large Scale Vocabulary Test.

The Large Scale Vocabulary Test (LSVT) was designed to evaluate how well the Metathesaurus plus planned additions to Meta covered the documentation needs of clinicians. Our consortium collected 10,538 clinical narratives from patient problem lists recorded at 65 Veterans Hospitals, internal medicine ambulatory care practices, diagnostic history and physical examination data elements from Iliad, and nursing shift notes and emergency transport patient records. The results showed 94% of submitted terms resulted in acceptable matches. 49% of submitted terms were judged to be synonymous with the match terms, 35% were judged to be more specific (usually due to modifiers), 2%, were less specific, and 6% had an associative relationship. In 8% of cases either no match was found by the LSVT interface or all proposed matches were rejected by the raters. The LSVT content was quite suitable for coding our narratives. Necessary improvements for an electronic record would include the ability to compose modifiers together with root concepts.

Inter-rater reliability and review of the VA unresolved narratives.

To better understand how VA clinicians use medical vocabulary in every day practice, we set out to characterize terms generated in the Problem List module of the VA's DHCP system that were not mapped to terms in the controlled-vocabulary lexicon of DHCP. When entered terms fail to match those in the lexicon, a note is sent to a central repository. When our study started, the volume in that repository had reached 16,783 terms. We wished to characterize the potential reasons why these terms failed to match terms in the lexicon. After examining two small samples of randomly selected terms, we used group consensus to develop a set of rating criteria and a rating form. To be sure that the results of multiple reviewers could be confidently compared, we analyzed the inter-rater agreement of our rating process. Two rates used this form to rate the same 400 terms. We found that modifiers and numeric data were common and consistent reasons for failure to match, while others such as use of synonyms and absence of the concept from the lexicon were common but less consistently selected.

A computer-generated reminder system improves physician compliance with diabetes preventive care guidelines.

Computerized reminder systems have been shown to be effective in improving physician compliance with preventive services guidelines. Very little has been published about the use of computerized reminders for preventive care in diabetes. We implemented a computer-generated reminder system for diabetes care guidelines in a randomized controlled study in the outpatient clinics of 35 internal medicine residents at the University of Utah and Salt Lake Veterans Affairs Hospitals. After a six month study period, compliance with the recommended care significantly improved in both the intervention group that received patient-specific reminders about the guidelines (38.0% at baseline, 54.9% at follow-up) and the control group that received a nonspecific report (34.6% at baseline, 51.0% at follow-up). There was no significant difference between the two groups. Both clinic sites showed similar improvement over baseline levels of compliance. Residents who completed encounter forms used by the system showed a significantly greater improvement in compliance than those who did not complete encounter forms (19.7% vs. 7.6%, p = 0.006). The improvements in guideline compliance were seen in all areas of diabetes preventive care studied, and significant improvements were seen with recommended items from the medical history, physical exam, laboratory testing, referrals, and patient education. The use of encounter forms by the providers significantly improved documented compliance with the guidelines in almost all categories of preventive care. These results suggest that computerized reminder systems improve compliance with recommended care more by facilitating the documentation of clinical findings and the ordering of recommended procedures than by providing the clinician with patient-specific information about guideline compliance status. Further study is needed to understand the implications of these findings to the development of future computerized reminder systems for chronic diseases such as diabetes.

Assessing the behavioral impact of a diagnostic decision support system.

This paper describes a prototype for research to evaluate the impact of diagnostic decision support systems on the behavior of physicians. Several indices that can be used to quantify the magnitude of impact are proposed. A large medical diagnostic knowledge base in internal medicine (the Iliad knowledge base) was used in this evaluation. The impact on behavior when different inference models are run against this knowledge base is evaluated for two different case domains and physician's specialties.

Development and implementation of a computer-generated reminder system for diabetes preventive care.

Diabetes mellitus is a chronic condition with several late complications that can be delayed or avoided through proper preventive health care. Although practice guidelines have been established to improve the preventive care in diabetics, dissemination of these guidelines among physicians and educational programs have been only moderately successful in changing physicians' practice patterns. Previous efforts, however, did not utilize computer-generated reminders. We developed a system of computer-generated reminders for diabetic preventive care. We completed an implementation of the system in the outpatient clinics of internal medicine residents at our institution. This paper describes the development and implementation of this system. Our results showed that the system flagged an average of 13 items that deviated from diabetes guideline compliance, out of a possible 21 items per patient. The residents completed encounter forms used by the system for 37% of patients seen during a six month period. Physician users exhibited positive attitudes toward the use of guidelines which they judged improved quality at no additional cost of care. However, the complexity and length of the guideline encounter forms and the additional time demands proved to be significant obstacles to current routine use. Our experience will help to improve the system so that it is more usable and acceptable to physicians, especially in the future as health care increasingly makes use of electronic medical record systems.

Evaluating consensus among physicians in medical knowledge base construction.

This study evaluates inter-author variability in knowledge base construction. Seven board-certified internists independently profiled "acute perinephric abscess", using as reference material a set of 109 peer-reviewed articles. Each participant created a list of findings associated with the disease, estimated the predictive value and sensitivity of each finding, and assessed the pertinence of each article for making each judgment. Agreement in finding selection was significantly different from chance: seven, six, and five participants selected the same finding 78.6, 9.8, and 1.6 times more often than predicted by chance. Findings with the highest sensitivity were most likely to be included by all participants. The selection of supporting evidence from the medical literature was significantly related to each physician's agreement with the majority. The study shows that, with appropriate guidance, physicians can reproducibly extract information from the medical literature, and thus established a foundation for multi-author knowledge base construction.

Iliad's role in the generalization of learning across a medical domain.

Medical informatics could facilitate more effective analysis and use of clinical knowledge by means of expert systems. To be most effective, such systems should be constructed in a manner which is consistent with physicians' cognitive processes. Our past five years' work with a system called Iliad indicates that it provides effective medical training and education. The current research extends our previous work by using a wider array of training and test cases. We also evaluated whether training on specific cases could generalize to improved testing performance on related cases, which featured similar complaints and pathophysiologic mechanisms, but different final diagnoses. In their junior internal medicine clerkship, students (n = 100) completed 1300 Iliad training cases covering 48 diagnoses. The findings indicated improved problem solving on the specifically trained cases as well as the generalization cases. We discuss a possible training model for expert systems such as Iliad.

Comparison of different information content models by using two strategies: development of the best information algorithm for Iliad.

Iliad is a diagnostic expert system for internal medicine. Iliad's "best information" mode is used to determine the most cost-effective findings to pursue next at any stage of a work-up. The "best information" algorithm combines an information content calculation together with a cost factor. The calculations then provide a rank-ordering of the alternative patient findings according to cost-effectiveness. The authors evaluated five information content models under two different strategies. The first, the single-frame strategy, considers findings only within the context of each individual disease frame. The second, the across-frame strategy, considers the information that a single finding could provide across several diseases. The study found that (1) a version of Shannon's information model performed the best under both strategies---this finding confirms the result of a previous independent study, (2) the across-frame strategy was preferred over the single-frame strategy.

Iliad training enhances medical students' diagnostic skills.

Iliad is a computerized, expert system for internal medical diagnosis. The system is designed to teach diagnostic skills by means of simulated patient case presentations. We report the results of a controlled trial in which junior students were randomly assigned to received Iliad training on one of two different simulated case mixes. Each group was subsequently tested in both their "trained" and "untrained" case domain. The testing consisted of computerized, simulated patient cases for which no training feedback was provided. Outcome variables were designed to measure the students' performance on these test cases. The results indicate that students made fewer diagnostic errors and more conclusively confirmed their diagnostic hypotheses when they were tested in their trained domain. We conclude that expert systems such as Iliad can effectively teach diagnostic skills by supplementing trainees' actual case experience with computerized simulations.

Exploring a new best information algorithm for Iliad.

Iliad is a diagnostic expert system for internal medicine. One important feature that Iliad offers is the ability to analyze a particular patient case and to determine the most cost-effective method for pursuing the work-up. Iliad's current "best information" algorithm has not been previously validated and compared to other potential algorithms. Therefore, this paper presents a comparison of four new algorithms to the current algorithm. The basis for this comparison was eighteen "vignette" cases derived from real patient cases from the University of Utah Medical Center. The results indicated that the current algorithm can be significantly improved. More promising algorithms are suggested for future investigation.

Iliad training effects: a cognitive model and empirical findings.

Iliad is a diagnostic expert system consisting of an "inference engine" (collection of rules and procedures for making decisions) and a "knowledge base" (collection of medical facts). Iliad's internal medicine knowledge base recognizes 5000 medical findings and covers 1150 diagnostic conditions in 10 subspecialty fields. We used Iliad's simulator mode to train diagnostic skills in junior-year medical students. The results corroborate previous findings documenting Iliad's teaching efficacy. Recent developments in cognitive psychology provide a framework for explaining Iliad's training effects.

Pulmonary telangiectasia without hypoxemia.

We describe an elderly patient with an unusual presentation of hereditary hemorrhagic telangiectasia (Rendu-Osler-Weber disease) involving the lung. He had recurrent "pneumonia" caused by massive hemorrhage from endobronchial telangiectases. When stable, he was normoxic, had no evidence of right-to-left shunting, and had mild pulmonary arterial hypertension. His pulmonary telangiectases may be isolated to the bronchial circulation. We report hemodynamic data and show the first photographs of endobronchial telangiectases.

Dissociation between diaphragmatic and rib cage muscle fatigue.

To assess rib cage muscle fatigue and its relationship to diaphragmatic fatigue, we recorded the electromyogram (EMG) of the parasternal intercostals (PS), sternocleidomastoid (SM), and platysma with fine wire electrodes and the EMG of the diaphragm (DI) with an esophageal electrode. Six normal subjects were studied during inspiratory resistive breathing. Two different breathing patterns were imposed: mainly diaphragmatic or mainly rib cage breathing. The development of fatigue was assessed by analysis of the high-to-low (H/L) ratio of the EMG. To determine the appropriate frequency bands for the PS and SM, we established their EMG power spectrum by Fourier analysis. The mean and SD for the centroid frequency was 312 +/- 16 Hz for PS and 244 +/- 48 Hz for SM. When breathing with the diaphragmatic patterns, all subjects showed a fall in H/L of the DI and none had a fall in H/L of the PS or SM. During rib cage emphasis, four out of five subjects showed a fall in H/L of the PS and five out of six showed a fall in H/L of the SM. Four subjects showed no fall in H/L of the DI; the other two subjects were unable to inhibit diaphragm activity to a substantial degree and did show a fall in H/L of the DI. Activity of the platysma was minimal or absent during diaphragmatic emphasis but was usually strong during rib cage breathing. We conclude that fatigue of either the diaphragm or the parasternal and sternocleidomastoid can occur independently according to the recruitment pattern of inspiratory muscles.(ABSTRACT TRUNCATED AT 250 WORDS)