Key performance indicators to benchmark hospital information systems - a delphi study.

OBJECTIVES: To identify the key performance indicators for hospital information systems (HIS) that can be used for HIS benchmarking. METHODS: A Delphi survey with one qualitative and two quantitative rounds. Forty-four HIS experts from health care IT practice and academia participated in all three rounds. RESULTS: Seventy-seven performance indicators were identified and organized into eight categories: technical quality, software quality, architecture and interface quality, IT vendor quality, IT support and IT department quality, workflow support quality, IT outcome quality, and IT costs. The highest ranked indicators are related to clinical workflow support and user satisfaction. Isolated technical indicators or cost indicators were not seen as useful. The experts favored an interdisciplinary group of all the stakeholders, led by hospital management, to conduct the HIS benchmarking. They proposed benchmarking activities both in regular (annual) intervals as well as at defined events (for example after IT introduction). Most of the experts stated that in their institutions no HIS benchmarking activities are being performed at the moment. CONCLUSION: In the context of IT governance, IT benchmarking is gaining importance in the healthcare area. The found indicators reflect the view of health care IT professionals and researchers. Research is needed to further validate and operationalize key performance indicators, to provide an IT benchmarking framework, and to provide open repositories for a comparison of the HIS benchmarks of different hospitals.

Medical informatics education needs information system practicums in health care settings--experiences and lessons learned from 32 practicums at four universities in two countries.

OBJECTIVES: To report about the themes and about experiences with practicums in the management of information systems in health care settings (health information management) for medical informatics students. METHODS: We first summarize the topics of the health information management practicums/projects that the authors organized between 1990 and 2003 for the medical informatics programs at Heidelberg/Heilbronn, Germany, UMIT, Austria, as well as for the informatics program at the University of Leipzig, Germany. Experiences and lessons learned, obtained from the faculty that organized the practicums in the past 14 years, are reported. RESULTS: Thirty (of 32) health information management practicums focused on the analysis of health information systems. These took place inside university medical centers. Although the practicums were time-intensive and required intensively tutoring students with regard to health information management and project management, feedback from the students and graduates was mainly positive. DISCUSSION: It is clearly recommended that students specializing in medical informatics need to be confronted with real-world problems of health information systems during their studies.