Diagnostic certainty and hospital resource use.

OBJECTIVE: To determine whether the house staff's subjective probability estimates of their initial admitting diagnoses are independent predictors of in-hospital resource consumption. DESIGN: Descriptive correlational study. SETTING: Academic medical center inpatient setting. PATIENTS: Patients admitted to general medicine wards. MEASUREMENTS: A visual analog scale with hash marks at 0, 25, 50, 75 and 100 was used to obtain a subjective probability estimate that the house staff's initial admitting diagnosis was a correct diagnosis. This provided the measure of diagnostic "certainty" at the time of admission. Patient demographic data, prior hospital stays, distance from hospital, MedisGroups scores, outlier status and vital status at discharge were obtained from administrative systems. Length of stay, total charges, cost estimates (total departmental), and number of consultations were obtained from the hospital-based transaction master database. RESULTS: House staff evaluated 1,778 admissions, 77.2% were assigned a diagnostic certainty rating of 75% or higher. In univariate analysis, the certainty rating did not vary with the MedisGroups score, outlier status or vital status at discharge. It varied with prior stays and measures of resource use. In multivariate analysis, the certainty rating was a significant factor accounting for variation in each of the measures of resource utilization with the exception of adjusted pharmacy charges. CONCLUSIONS: Even in a teaching hospital only a small proportion of patients had an "uncertain" diagnosis (22.8%). Nonetheless, the certainty variables were significantly related to measures of resource consumption including length of stay, total costs and number of consults obtained.

Information needs of health care professionals in an AIDS outpatient clinic as determined by chart review.

OBJECTIVE: To examine the information needs of health care professionals in HIV-related clinical encounters, and to determine the suitability of existing information sources to address those needs. SETTING: HIV outpatient clinic. PARTICIPANTS: Seven health care professionals with diverse training and patient care involvement. METHODS: Based on patient charts describing 120 patient encounters, participants generated 266 clinical questions. Printed and on-line information sources were used to answer questions in two phases: using commonly available sources and using all available medical library sources. MEASUREMENTS: The questions were divided into 16 categories by subject. The number of questions answered, their categories, the information source(s) providing answers, and the time required to answer questions were recorded for each phase. RESULTS: Each participant generated an average of 3.8 clinical questions per chart. Five categories accounted for almost 75% of all questions; the treatment protocols/regimens category was most frequent (24%). A total of 245 questions (92%) were answered, requiring an average of 15 minutes per question. Most (87%) of the questions were answered via electronic sources, even though paper sources were consulted first. CONCLUSIONS: The participating professionals showed considerable information needs. A combination of on-line and paper sources was necessary to provide the answers. The study suggests that present-day information sources are not entirely satisfactory for answering clinical questions generated by examining charts of HIV-infected patients.

Evaluating consensus among physicians in medical knowledge base construction.

This study evaluates inter-author variability in knowledge base construction. Seven board-certified internists independently profiled "acute perinephric abscess", using as reference material a set of 109 peer-reviewed articles. Each participant created a list of findings associated with the disease, estimated the predictive value and sensitivity of each finding, and assessed the pertinence of each article for making each judgment. Agreement in finding selection was significantly different from chance: seven, six, and five participants selected the same finding 78.6, 9.8, and 1.6 times more often than predicted by chance. Findings with the highest sensitivity were most likely to be included by all participants. The selection of supporting evidence from the medical literature was significantly related to each physician's agreement with the majority. The study shows that, with appropriate guidance, physicians can reproducibly extract information from the medical literature, and thus established a foundation for multi-author knowledge base construction.

Comparing contents of a knowledge base to traditional information sources.

Physicians rely on the medical literature as a major source of medical knowledge and data. The medical literature, however, is continually evolving and represents different sources at different levels of coverage and detail. The recent development of computerized medical knowledge bases has added a new form of information that can potentially be used to address the practicing physician's information needs. To understand how the information from various sources differs, we compared the description of a disease found in the QMR knowledge base to those found in two general internal medicine textbooks and two specialized nephrology textbooks. The study shows both differences in coverage and differences in the level of detail. Textbooks contain information about pathophysiology and therapy that is not present in the diagnostic knowledge base. The knowledge base contains a more detailed description of the associated findings, more quantitative information, and a greater number of references to peer-reviewed medical articles. The study demonstrates that computerized knowledge bases, if properly constructed, may be able to provide clinicians with a useful new source of medical knowledge that is complementary to existing sources.

Physicians' use of computer software in answering clinical questions.

Descriptive data about the use of medical information software were gathered from physicians who were early users of these resources. Eight clinically active internists and medical subspecialists were lent a microcomputer loaded with six commercially available medical information software products. Participants used the software for two weeks to answer questions arising in their practice and completed written questionnaires. They recorded a total of 50 questions (between 3 and 11 per participant per two-week study period). Using the workstation, participants answered 20 questions (40% of the total), partially answered 16 questions (32%), and did not obtain useful information for 14 questions (28%). Participants found answers outside the workstation to 8 of the 14 questions (57%) not answered by using the software. The most common question topic was drug information (16 questions, or 32% of the total). The most common problems encountered using the workstation were retrieval of incomplete information (20 questions, or 40% of the total) and difficulty navigating the software (16 questions, or 32%). Other problems included difficulty translating clinical problems into questions, inappropriate resource selection, inadequate training for using the software, and excessive time required to access information. The study highlights several opportunities for medical librarians and others involved in clinical information management to facilitate the use of computer software for solving clinical problems.

A method for assessing the impact of a computer-based decision support system on health care outcomes.

The wide variation in utilization of diagnostic resources has not been decreased by the proliferation of new diagnostic technologies. We wish to test the hypothesis that the introduction of a medical decision support system into clinical practice could potentially lead to more efficient use of diagnostic information, and therefore lead to a reduction in overall laboratory use and cost of care. We have devised and are currently implementing a randomized controlled trial of a computer based decision support system, the University of Pittsburgh version of Quick Medical Reference (QMR). The main purpose of the study is to determine the effect of the QMR program on specific outcome measures: length of hospital stay, number and types of diagnostic tests ordered, and overall charges. An important part of this evaluation is relating the initial level of diagnostic uncertainty expressed by the admitting housestaff team to utilization of diagnostic resources. The purpose of this paper is to describe the methodology for carrying out this controlled trial, and to describe our initial experiences with its implementation.

Heuristic determination of quantitative data for knowledge acquisition in medicine.

Knowledge acquisition for medical knowledge bases can be aided by programs that suggest possible values for portions of the data. The paper presents an experiment which was used in designing a heuristic to help the process of knowledge acquisition. The heuristic helps to determine numerical data from stylized literature excerpts in the context of knowledge acquisition for the QMR medical knowledge base. Quantitative suggestions from the heuristics are shown to agree substantially with the data incorporated in the final version of the knowledge base. The experiment shows the potential of knowledge base specific heuristics in simplifying the task of knowledge base creation.

Physicians' information needs: analysis of questions posed during clinical teaching.

OBJECTIVE: To describe information requests expressed during clinical teaching. SETTING: Residents' work rounds, attending rounds, morning report, and interns' clinic in a university-based general medicine service. SUBJECTS: Attending physicians, medical house staff, and medical students in a general medicine training program. METHODS: An anthropologist observed communication among study subjects and recorded in field notes expressions of a need for information. We developed a coding scheme for describing information requests and applied the coding scheme to the data recorded. Based on assigned codes, we created a subset of strictly clinical requests. MEASUREMENTS: Five hundred nineteen information requests recorded during 17 hours of observed clinical activity were selected for detailed analysis. These requests related to the care of approximately 90 patients by 24 physicians and medical students. Sixty-five requests were excluded because they were not strictly clinical, leaving a subset of 454 clinical questions for analysis. MAIN RESULTS: On average, five clinical questions were raised for each patient discussed. Three hundred thirty-seven requests (74%) concerned patient care. Of these 337 questions, 175 (52%) requested a fact that could have been found in a medical record. Seventy-seven (23%) of these questions, motivated by the needs of patient care, were potentially answerable by a library, a textbook, a journal, or MEDLINE. Eighty-eight (26%) of the questions asked for patient care required synthesis of patient information and medical knowledge. CONCLUSIONS: Clinicians in the study settings requested information frequently. Many of these information needs required the synthesis of patient information and medical knowledge and thus were potentially difficult to satisfy. A typology is proposed that characterizes information needs as consciously recognized, unrecognized, and currently satisfied.

A prospective analysis of inter-rater agreement between a physician and a physician's assistant in selecting QMR vocabulary terms.

We prospectively identified a cohort of 19 hospital inpatients to serve as a test set to determine the agreement between a physician and a physician's assistant in selecting and entering Quick Medical Reference (QMR) vocabulary terms for use in QMR case analysis mode. For positive findings, there was good agreement between data entered by the physician's assistant and data entered by the physician. Overall matches for positive findings were 0.66 concordance for the initial state, and 0.67 for the final state. Concordance for negative findings was substantially lower. The QMR generated differential diagnoses lists of the physician and physician's assistant were judged as similar in 14 (74%) of the 19 cases when QMR critique and assert functions were used.

Effect of a computer-assisted general medicine diagnostic consultation service on housestaff diagnostic strategy.

Quick Medical Reference (QMR) is a microcomputer-based decision support system designed to provide diagnostic assistance in the field of internal medicine. In addition to providing plausible diagnostic hypotheses based upon patient specific findings, the program highlights history, physical and laboratory items which are potentially useful in discriminating among the diagnoses under consideration. We have evaluated the impact of a computer-assisted diagnostic consultation service on the diagnostic and management strategy of a housestaff in a university internal medicine training program. Differential diagnoses were obtained before and after the use of the program, and a questionnaire was used to assess the educational value of the service and the effect of the service on the diagnosis and planned management. Over an eight week period, 31 cases were identified which met inclusion criteria. The QMR consultation added a diagnosis to the original list in 14 out of 31 cases. The consultation reordered the diagnosis in an additional 7 cases, and in 8 cases a diagnosis was ruled out by the use of the program. After the use of the program the housestaff reported they would obtain an additional lab test in 10 cases, change the order of planned tests in two cases and eliminate a lab test in one case. The use of the program was rated as helpful educationally in 81% of the cases, and helpful with respect to management in also 81% of the cases.

A computer-assisted medical diagnostic consultation service. Implementation and prospective evaluation of a prototype.

STUDY OBJECTIVE: To evaluate the accuracy of a computer-aided consultation service using academic general internists and the Quick Medical Reference (QMR) diagnostic program: and to study the impact of the consultation on the diagnostic behavior of physicians caring for patients. DESIGN: Prospective study of the diagnostic accuracy of computer-aided consultation in 31 cases, as well as a prospective study of ward team diagnoses and opinions before and after consultation. SETTING: General medicine wards of two tertiary care centers. PARTICIPANTS: Thirty-one patients identified as posing a diagnostic challenge and meeting eligibility criteria, as well as the housestaff caring for these patients. MEASUREMENTS AND MAIN RESULTS: After 6 months follow-up, diagnoses were established in 20 of 31 cases. The diagnostic sensitivity of the computer-assisted diagnoses, 85% (95% CI, 56% to 97%), was similar to that of the consult service physicians, 80% (95% CI, 55% to 94%), but better than that of the ward teams, 60% (95% CI, 33% to 81%; P = 0.03 using the binomial test). The consultation influenced the postconsultation differential diagnoses of the ward teams in 26 of the 31 cases (95% CI, 92% to 95%). House officers rated the consultation service as being educationally helpful in 25 of the 31 cases (95% CI, 62% to 94%). CONCLUSIONS: Computer-aided diagnostic consultation, when provided by physicians familiar with the limitations of the system and capable of overriding inappropriate suggestions, was both accurate and educationally helpful in most cases. The system provided reasonable diagnostic suggestions not previously considered by the ward teams and these suggestions were valued sufficiently to cause alteration of the original differential diagnoses.