Electronic health records: is the evidence base any use?

Information-technology policies in many countries are full of aspirational statements and not generally based on the available evidence. This paper aims to contribute to a proper discussion of the evidence on electronic health records (EHR). The paper describes the methods we used in a review of the impact of EHR on clinical work and administration, summarizes our findings, and discusses their relevance in the context of policy makers' information requirements. We argue that the literature has, to date, largely failed to deliver usable findings.

Systematic reviews of clinical decision tools for acute abdominal pain.

OBJECTIVES: To review for acute abdominal pain (AAP), the diagnostic accuracies of combining decision tools (DTs) and doctors aided by DTs compared with those of unaided doctors. Also to evaluate the impact of providing doctors with an AAP DT on patient outcomes, clinical decisions and actions, what factors are likely to determine the usage rates and usability of a DT and the associated costs and likely cost-effectiveness of these DTs in routine use in the UK. DESIGN: Electronic databases were searched up to 1 July 2003. REVIEW METHODS: Data from each eligible study were extracted. Potential sources of heterogeneity were extracted for both questions. For the accuracy review, meta-analysis was conducted. Among studies comparing diagnostic accuracies of DTs with unaided doctors, error rate ratios provided estimates of the differences between the false-negative and false-positive rates of the DT and unaided doctors' performance. Pooled error rate ratios and 95% confidence intervals (CIs) for false-negative rates and false-positive rates were computed. Metaregression was used to explore heterogeneity. RESULTS: Thirty-two studies from 27 articles, all based in secondary care, were eligible for the review of DT accuracies, while two were eligible for the review of the accuracy of hospital doctors aided by DTs. Sensitivities and specificities for DTs ranged from 53 to 99% and from 30 to 99%, respectively. Those for unaided doctors ranged from 64 to 93% and from 39 to 91%, respectively. Thirteen studies reported false-positive and false-negative rates for both DTs and unaided doctors, enabling a direct comparison of their performance. In random effects meta-analyses, DTs had significantly lower false-positive rates (error rate ratio 0.62, 95% CI 0.46 to 0.83) than unaided doctors. DTs may have higher false-negative rates than unaided doctors (error rate ratio 1.34, 95% CI 0.93 to 1.93). Significant heterogeneity was present. Two studies compared the diagnostic accuracies of doctors aided by DTs to unaided doctors. In a multiarm cluster randomised controlled trial (n = 5193), the diagnostic accuracy of doctors not given access to DTs was not significantly worse (sensitivity 28.4% and specificity 96.0%) than that of three groups of aided doctors (sensitivities of 42.4-47.9%, and specificities of 95.5-96.5%, respectively). In an uncontrolled before-and-after study (n = 1484), the sensitivities and specificities of aided and unaided doctors were 95.5% and 91.5% (p = 0.24) and 78.1% and 86.4% (p < 0.001), respectively. The metaregression of DTs showed that prospective test-set validation at the site of the tool's development was associated with considerably higher diagnostic accuracy than prospective test-set validation at an independent centre [relative diagnostic odds ratio (RDOR) 8.2; 95% CI 3.1 to 14.7]. It also showed that the earlier in the year the study was performed the higher the performance (RDOR 0.88, 0.83 to 0.92), that when developers evaluated their own DT there was better performance than when independent evaluators carried out the study (RDOR = 3.0, 1.3 to 6.8), and that there was no evidence of association between other quality indicators and DT accuracy. The one eligible study of the impact study review, a four-arm cluster randomised trial (n = 5193), showed that hospital admission rates of patients by doctors not allocated to a DT (42.8%) were significantly higher than those by doctors allocated to three combinations of decision support (34.2-38.5%) (p < 0.001). There was no evidence of a difference between perforation rates (p = 0.19) and negative laparotomy rates in the four trial arms (p = 0.46). Usage rates of DTs by doctors in accident and emergency departments ranged from 10 to 77% in the six studies that reported them. Possible determinants of usability include the reasoning method used, the number of items used and the output format. A deterministic cost-effectiveness comparison demonstrated that a paper checklist is likely to be 100-900 times more cost-effective than a computer-based DT, under stated assumptions. CONCLUSIONS: With their significantly greater specificity and lower false-positive rates than doctors, DTs are potentially useful in confirming a diagnosis of acute appendicitis, but not in ruling it out. The clinical use of well-designed, condition-specific paper or computer-based structured checklists is promising as a way to improve impact on patient outcomes, subject to further research.

European and Latin-American countries associated in a networked database of outstanding guidelines in unusual clinical cases (ELCANO).

The aim of ELCANO, an EU-funded project, is to build a virtual multilingual multimedia library of unusual clinical cases related to gastroenterology. Based on a standardisation of the format to report and the representation of clinical information on WWW, the multilingual multimedia case has been developed and till now 350 cases have been included. User satisfaction and user acceptance of ELCANO are under evaluation.

An information technology course in the medical curriculum.

This paper concerns the information technology course now running in year 1 of the undergraduate curriculum of the medical school of Leeds University. The background and objectives of the course are described, and the course content is outlined in terms of knowledge and practical skills. By the end of the course, 95% of students could successfully accomplish 95% of the skill tasks taught. As regards attitudes to IT, independent studies comparing two randomly selected groups suggested students who had taken the course (a) used IT more frequently, and (b) viewed IT more favourably than their counterparts.

The British Society of Gastroenterology early gastric cancer/dysplasia survey: an interim report.

This presentation describes interim findings in a series of 319 patients referred from 41 hospitals on the basis of histopathological findings of 'early gastric cancer', 'dysplasia', or 'worrying mucosal appearances'. Data were recorded using a predefined proforma, and histopathological material circulated amongst a 'panel' of three further pathologists. After this process, 132 patients were classified as having early gastric cancer and 63 as dysplasia. There was good agreement between pathologists as to whether the cases had cancer or dysplasia - but 39 cases said by referring pathologists to have early gastric cancer were classified by the panel as having more extensive disease. Most early gastric cancer cases were diagnosed only after histopathological examination. Cancer or 'possible cancer' was only mentioned after 36% of the radiological investigations and 40.5% of the endoscopies. Computer aided analysis of the patients' symptoms placed 91.3% of the early gastric cancer cases into a 'high risk' group - but was unable to distinguish between early gastric cancer and dysplasia. The five year survival rate of the cases agreed to be early gastric cancer by the panel was well over 90%, but the four year survival rate of cases registered as 'early gastric cancer' but said by the panel to have more advanced disease was under 75%. These findings may account for some of the differences between series, and emphasise the need for precise, widely agreed criteria for the diagnosis of early gastric cancer and gastric dysplasia.

OMGE international upper gastrointestinal bleeding survey, 1978-1986.

This presentation describes a further survey conducted under the auspices of the OMGE Research Committee: an international survey of over 4000 patients presenting to 205 clinicians in 44 centres in 21 countries. This multinational survey has been the first in this area of medicine to use specifically designed proformata and pre-agreed definitions of terminology, alongside automated data analysis. A total of 4506 patients' data were forwarded from 44 centres, the vast majority (4431; 98.3%) containing acceptable data fro subsequent analysis. The commonest cause of bleeding was peptic ulceration (36.7% of the total group), followed by oesophageal varices (13.3%) and gastric erosions (6.7%). Over 600 cases exhibited dual pathology. As regards outcome, in 1137 patients bleeding failed to settle (25.5%), whilst 409 patients died (9.2% of the total group, and 36.0% of those whose bleeding failed to settle). Detailed computer-aided studies show it is currently impossible to determine the source of bleeding without recourse to high technology such as radiologic or endoscopic intervention. If identification of the source of the bleeding is desirable clinically, then these high-technology investigations must be employed. As regard prognosis, a computer program has enabled categorisation of patients into risk categories as regards further or continued bleeding or death. This program has been tested on 2623 patients with considerable accuracy. In the computer's 'high-risk' category, 60% of patients re-bled, and 32% died. Conversely in the 'very low' risk category, only 4% re-bled, and no patient died. These findings have enabled the construction of a simple prognostic system for use in remote areas.(ABSTRACT TRUNCATED AT 250 WORDS)

Do the stigmata of recent haemorrhage have additional prognostic value in patients with bleeding duodenal ulcer?

To assess the additional prognostic value of endoscopic stigmata of recent haemorrhage (SRH) in addition to clinical data, a pilot study was conducted on 207 duodenal ulcer patients from the OMGE survey. The incidence of SRH was compared in 145 patients who settled and 62 who re-bled. Only active bleeding emerged as significantly commoner in the latter group. Two computer-assisted predictions of further bleeding (before and after addition of the stigmata to a clinical database) were compared with the final outcome. Little improvement, in terms of prognosis for further bleeding, was obtained when SRH data were added. Similar findings were observed in a smaller series from Marburg, FRG.

Use of a multinational survey to provide clinical guidelines for upper gastrointestinal bleeding in developing countries.

The OMGE multinational survey of patients with upper gastrointestinal bleeding demonstrated that it was possible to predict patient outcome, using a computer and a database of information from many centres. It remained to be seen whether this database could be used in specific remote areas of the world. To answer this question, two areas have been studied, Sikkim and China. In Sikkim, when the computer-aided prognostic system was used, 69% of patients put into a high-risk group for rebleeding actually did so; and 54% died. By contrast, only 2% of patients placed into a low-risk group for rebleeding did so. As there is little computer technology in Sikkim, a simplified scoring system was developed which gave the same predictive accuracy as the computer system. In China there was a slightly lower accuracy with both systems. Hence a new database and scoring system were created, using only information from Chinese patients. This database improved the results. From the studies it is suggested that these types of systems can be of value to patients in remote areas by targeting patients at high risk rebleeding or dying, so that the scarce resources available can be best used.

The OMGE multinational inflammatory bowel disease survey 1976-1986. A further report on 3175 cases.

This presentation describes the progress during 1982-1986 of the OMGE Multinational Survey of patients with inflammatory bowel disease. After a brief description of the study design and protocol, the status of the survey in 1986 is presented. In all, 40 centres contributed 3175 cases at that time, data collection being meticulous via previously designed proformata. Diagnostic criteria are next discussed. Little change between 1976 and 1986 is noted, with wide congruence of diagnostic thought, now codified into a simple (and recommended) OMGE diagnostic scoring system. Patients seen prior to 1978 were reviewed in 1986. Where attempted, a follow-up of over 86% was achieved, usually more than 4 years after the original presentation involving no less than 5215 'patient-years' of observed follow-up. Following these overall considerations, details of four subprojects are annexed, each of which was presented as a 'free paper' at the 8th World Congress and concerning, respectively, the changing natural history of IBD, risks of perforation and toxic megacolon in the 1980s, IBD in elderly patients, and features associated with recurrence in Crohn's disease.

The natural history of inflammatory bowel disease: has there been a change in the last 20 years?

This presentation reviews the course and outcome of disease in 2657 cases of inflammatory bowel disease (IBD) registered into the OMGE multinational survey and considers whether there are differences between the natural history of IBD observed in this series and that observed in earlier large-scale series. It is concluded that several such differences exist. The current mortality from ulcerative colitis (4% in 10 years) is similar to that from Crohn's disease and quite different from that recorded in earlier series (over 20% in 10 years). This may be because the disease itself has changed (the proportion of patient-years with severe attack has fallen from 14.6% in the 1960s to under 10% in the present series). It may also be because of increasing use of maintenance sulphasalazine (since the attack rate per year is significantly lower than patients on maintenance therapy). Finally, cancer is now equally common in Crohn's disease and ulcerative colitis patients; whilst perforation is more common in Crohn's disease (possibly because ulcerative colitis patients now tend to come to surgery earlier).

Perforation of the intestine in inflammatory bowel disease. An OMGE survey.

This presentation assesses the incidence of perforation of the intestine in patients with inflammatory bowel disease by reviewing the incidence of this complication in a total of 3175 patients from the OMGE inflammatory bowel disease multi-national survey. Amongst 1928 patients with ulcerative colitis, perforation was noted in only 5 (0.3%), which is much lower than in previous series (from 1% to 2%). By contrast, 19 of 1247 patients with Crohn's disease had perforated (1.5%). The survey thus suggests that the incidence of perforation in ulcerative colitis has fallen in the last 2 decades, probably as a result of the widespread implementation of early surgery. The risk of perforation of Crohn's disease remains quite high.

Inflammatory bowel disease in the elderly patient.

Studies of inflammatory bowel disease (IBD) undertaken in the 1960s reported a highly unfavourable course and prognosis in patients over the age of 60 years. However, recent surveys have suggested that the pattern of IBD in the elderly patient is similar to the overall pattern of disease in other age groups. We have, therefore, reviewed data relating to 2657 patients from the OMGE series with particular reference to the pattern of disease in 244 patients (9.1% of the total group) aged over 60 years at the time of registration into the survey. The course and outcome of disease in these 244 patients were similar to other age groups in terms of response to therapy, major complications, and rate of recourse to surgery. However, the death rate in the elderly (2.4%) was higher than that in younger patients (0.8%). We concur with recent studies that the course of IBD is now more favourable in elderly patients than hitherto suspected. The cause is probably multifactorial, involving both a change in therapy and a change in the natural history of the disease.

Factors affecting recurrence after surgery for Crohn's disease.

The prospect of recurrence following surgery for Crohn's disease (CD) poses an important and difficult problem in routine clinical practice. Out of the 1247 CD patients registered in the OMGE multinational inflammatory bowel disease survey, a high proportion came to surgery. A detailed study has been undertaken of 154 patients (77 who did recur matched with 77 who did not recur) who had undergone surgery for CD. An assessment was made which established a 'risk factor' (RF) for each item of patient information. Findings indicated the risk of recurrence to be higher in patients under 20 years both at onset of disease (RF, 2.2:1) and at time of their operation (RF, 2.7:1) and in patients with distal colonic Crohn's disease (RF, 1.8:1). Histopathologic findings, number of previous operations, and symptomatic status prior to operation did not appear to affect the subsequent risk of recurrence. The margin of clearance at operation appeared to affect further recurrence. Amongst patients with 10cm or more margin of clearance (i.e. histopathologically normal bowel) only 21% recurred during follow-up, versus 50% in those with a smaller margin of clearance.

Prognostic factors in upper G.I. bleeding.

This presentation draws upon the experience of the O.M.G.E. Multi-national Upper G.I. Bleeding Survey, using data collected during 1980-1982 by 185 clinicians from 44 centres in 21 countries to discuss two questions. First, can prognostic factors be identified in patients presenting to hospital with upper G.I. bleeding, and if so what are they? Second, is it possible - by combining the two technologies of endoscopy and computers - to provide an individual patient with a short-term prognostic prediction sufficiently accurate to affect patient management. Amongst 4,010 patients, a number of clinical factors were found to affect short-term prognosis. These included patient age, previous history of heart or liver disease, confusion and dehydration on admission, jaundice, and ascites. Identification of the bleeding source via endoscopy was shown to aid short-term prognosis - especially in the period of the 2nd to 10th days post-admission. Use of computer analysis enabled "high risk" patients to be defined (of whom 63.8% suffered further bleeding and 30.0% died), and also a comparable "low risk" group (of whom only 4% suffered further bleeding and none died). Finally, "time-dependence" studies have been used to identify a group of patients who (by the 2nd day post-admission) have a residual risk of further bleeding sufficiently low (well under 1%) to suggest that considerable resources can be saved by the judicious use of endoscopy and computer science.