Use of assessment to reinforce patient safety as a habit.

The US spends far more than any other nation on health care. Physicians undergo lengthy and comprehensive training that is carefully scrutinized, and are held to high standards in national examinations. At best the care delivered matches or exceeds that in any other country. And yet, often simple preventable medical errors occur at alarming and unacceptable rates. The public, corporate consumers of health care, large payors and malpractice insurance carriers are all becoming impatient with the pace of improvement. The medical profession recognizes that dealing with this problem is an urgent priority and is grappling to find the best approaches. This paper focuses on the constructive use of assessment to embed a pervasive and proactive culture of patient safety into practice, starting with the trainee and extending out into the practice years. This strategy is based on the adage that "assessment drives curriculum" and proposes a series of new assessment tools to be added to all phases of the training-practice continuum.

An investigation of the difficulty of computer-based case simulations.

This study investigated the characteristics of computer-based case simulations (CCS) that may be associated with case difficulty. Difficulty was defined as the average rating by physicians of examinee performance on a nine-point scale or the passing rate on the cases. Two data sets were used, one from an administration of 18 cases, the other from an administration of 22 cases with 13 cases used on both occasions. Stepwise regression procedures were used separately for case properties and for analytic scoring of key variables to identify the best sets of predictors of case difficulty. Because of the small number of cases, regression results were evaluated for consistency across both data and both difficulty measures. For key variables, the best set of predictors included the number of different serious errors of commission, risk items, and benefit items. In general, cases were more difficult for higher values of these variables. For case variables, the only consistent variable was the length of the paragraph that provided patient history, with longer paragraphs associated with more difficult cases. Other variables were less consistent, but were often related to the structure of the simulation or the severity of the patient condition. Although the findings for case variables were limited, the analyses were very helpful in illuminating the interconnections among the variables within cases.

Continued research on computer-based testing.

The National Board of Medical Examiners has developed computer-based examination formats for use in evaluating physicians in training. This paper describes continued research on these formats including attitudes about computers and effects of factors not related to the trait being measured; differences between paper-administered and computer-administered multiple-choice questions; and the characteristics of simulation formats. The implications for computer-based testing and further research are discussed.