Are students ready for meaningful use?

BACKGROUND: The meaningful use (MU) of electronic medical records (EMRs) is being implemented in three stages. Key objectives of stage one include electronic analysis of data entered into structured fields, using decision-support tools (e.g., checking drug-drug interactions [DDI]) and electronic information exchange. OBJECTIVE: The authors assessed the performance of medical students on 10 stage-one MU tasks and measured the correlation between students' MU performance and subsequent end-of-clerkship professionalism assessments and their grades on an end-of-year objective structured clinical examination. PARTICIPANTS: Two-hundred and twenty-two third-year medical students on the internal medicine (IM) clerkship. DESIGN/MAIN MEASURES: From July 2010 to February 2012, all students viewed 15 online tutorials covering MU competencies. The authors measured student MU documentation and performance in the chart of a virtual patient using a fully functional training EMR. Specific MU measurements included, adding: a new problem, a new medication, an advanced directive, smoking status, the results of screening tests; and performing a DDI (in which a major interaction was probable), and communicating a plan for this interaction. KEY RESULTS: A total of 130 MU errors were identified. Sixty-eight (30.6%) students had at least one error, and 30 (13.5%) had more than one (range 2-6). Of the 130 errors, 90 (69.2%) were errors in structured data entry. Errors occurred in medication dosing and instructions (18%), DDI identification (12%), documenting smoking status (15%), and colonoscopy results (23%). Students with MU errors demonstrated poorer performance on end-of-clerkship professionalism assessments (r =-0.112, p=0.048) and lower observed structured clinical examination (OSCE) history-taking skills (r =-0.165, p=0.008) and communication scores (r= - 0.173, p=0.006). CONCLUSIONS: MU errors among medical students are common and correlate with subsequent poor performance in multiple educational domains. These results indicate that without assessment and feedback, a substantial minority of students may not be ready to progress to more advanced MU tasks.

Mobile technology for the facilitation of direct observation and assessment of student performance.

BACKGROUND: We developed, implemented, and assessed a web-based clinical evaluation application (i.e., CEX app) for Internet-enabled mobile devices, including mobile phones. The app displays problem-specific checklists that correspond to training problems created by the Clerkship Directors in Internal Medicine. PURPOSE: We hypothesized that use of the CEX app for directly observing students' clinical skills would be feasible and acceptable, and would demonstrate adequate reliability and validity. METHODS: Between July 2010 and February 2012, 266 third-year medical students completed 5 to 10 formative CEXs during their internal medicine clerkship. The observers (attendings and residents), who performed the CEX, used the app to guide and document their observations, record their time observing and giving feedback to the students, and their overall satisfaction with the CEX app. Interrater reliability and validity were assessed with 17 observers who viewed 6 videotaped student-patient encounters, and by measuring the correlation between student CEX scores and their scores on subsequent standardized-patient Objective Structured Clinical Examination (OSCE) exams. RESULTS: A total of 2,523 CEXs were completed by 411 observers. The average number of evaluations per student was 9.8 (± 1.8 SD), and the average number of CEXs completed per observer was 6 (± 11.8 SD). Observers spent less than 10 min on 45.3% of the CEXs and 68.6% of the feedback sessions. An overwhelming majority of observers (90.6%) reported satisfaction with the CEX. Interrater reliability was measured at 0.69 among the observers viewing the videotapes, and their ratings discriminated between competent and noncompetent performances. Student CEX grades, however, did not correlate with their end of 3rd-year OSCE scores. CONCLUSIONS: The use of this CEX app is feasible and it captures students' clinical performance data with a high rate of user satisfaction. Our embedded checklists had adequate interrater reliability and concurrent validity. The grades measured on this app, however, were not predictive of subsequent student performance.

Documenting clinical performance problems among medical students: feedback for learner remediation and curriculum enhancement.

INTRODUCTION: We operationalized the taxonomy developed by Hauer and colleagues describing common clinical performance problems. Faculty raters pilot tested the resulting worksheet by observing recordings of problematic simulated clinical encounters involving third-year medical students. This approach provided a framework for structured feedback to guide learner improvement and curricular enhancement. METHODS: Eighty-two problematic clinical encounters from M3 students who failed their clinical competency examination were independently rated by paired clinical faculty members to identify common problems related to the medical interview, physical examination, and professionalism. RESULTS: Eleven out of 26 target performance problems were present in 25% or more encounters. Overall, 37% had unsatisfactory medical interviews, with 'inadequate history to rule out other diagnoses' most prevalent (60%). Seventy percent failed because of physical examination deficiencies, with missing elements (69%) and inadequate data gathering (69%) most common. One-third of the students did not introduce themselves to their patients. Among students failing based on standardized patient (SP) ratings, 93% also failed to demonstrate competency based on the faculty ratings. CONCLUSIONS: Our review form allowed clinical faculty to validate pass/fail decisions based on standardized patient ratings. Detailed information about performance problems contributes to learner feedback and curricular enhancement to guide remediation planning and faculty development.

The safety of preoperative vardenafil in patients undergoing coronary artery bypass graft surgery.

Phosphodiesterase 5 inhibitors are cardioprotective against myocardial reperfusion ischemic injury in animal models but are contraindicated in patients with coronary disease who take nitrates because of a risk for hypotension. We investigated the safety of vardenafil in patients undergoing coronary artery bypass grafting (CABG) surgery. A single dose of vardenafil was given to 10 patients before CABG surgery. The postoperative course of these 10 patients was compared with the postoperative course of 47 patients who did not receive vardenafil before CABG surgery. There were no perioperative deaths and no episodes of hypotension in the group receiving vardenafil. The clinical and operative characteristics of the 2 study groups were similar. There were no significant differences in postoperative serum troponin levels (9.1 ± 8.3 vs 12.5 ± 9.3 ng/mL; P = 0.29, respectively), duration of postoperative intubation (21.4 ± 10.1 vs 27.4 ± 15.2 hours; P = 0.14, respectively), or length of hospital stay (11.1 ± 13.2 vs 10.0 ± 4.7 days; P = 0.8, respectively) between the group receiving vardenafil and the control group. This pilot study of 10 patients suggests that vardenafil use is safe in patients before CABG surgery. A larger study is needed to explore the myocardial protective effect of the drug.

The implementation of a mobile problem-specific electronic CEX for assessing directly observed student-patient encounters.

BACKGROUND: Facilitating direct observation of medical students' clinical competencies is a pressing need. METHODS: We developed an electronic problem-specific Clinical Evaluation Exercise (eCEX) based on a national curriculum. We assessed its feasibility in monitoring and recording students' competencies and the impact of a grading incentive on the frequency of direct observations in an internal medicine clerkship. Students (n = 56) at three clinical sites used the eCEX and comparison students (n = 56) at three other clinical sites did not. Students in the eCEX group were required to arrange 10 evaluations with faculty preceptors. Students in the second group were required to document a single, faculty observed 'Full History and Physical' encounter with a patient. Students and preceptors were surveyed at the end of each rotation. RESULTS: eCEX increased students' and evaluators' understanding of direct-observation objectives and had a positive impact on the evaluators' ability to provide feedback and assessments. The grading incentive increased the number of times a student reported direct observation by a resident preceptor. CONCLUSIONS: eCEX appears to be an effective means of enhancing student evaluation.

Using patient encounter logs for mandated clinical encounters in an internal medicine clerkship.

BACKGROUND: Patient encounter logs help assess a student's educational experience. The use of a grading incentive linked to the mandatory documentation of prespecified clinical encounters has been insufficiently studied. PURPOSES: Given this, our objectives were to determine (a) if mandating student exposure to patients with 18 key training problems leads to the successful documentation of these encounters, (b) the degree of difficulty students and clerkship directors experienced in meeting these mandates, (c) the accuracy of mandated log entries, and (d) how often the log entries were questioned or rejected by preceptors. METHODS: Ninety-two 3rd-year internal medicine students and 6 internal medicine clerkship directors at 9 geographically dispersed hospitals and 30 ambulatory sites participated in the study. Over a single academic year, we directly measured the completion rate of mandated logs, the degree of their accuracy as judged against faculty-generated logs of two required videotaped standardized patient encounters, and the percentage of logs that were not validated. We surveyed students and directors on the perceived degree of difficulty in meeting mandated requirements. RESULTS: Ninety-eight percent of students met our mandated requirements and 93.8% of students found it "easy" or "very easy" to meet this requirement. The amount of estimated time spent by clerkship directors helping students meet mandated requirements for the entire year was 4.5 hr. The accuracy of submitted logs was 77%; however, almost all inaccurate log entries were "validated" by preceptors. CONCLUSIONS: Mandating encounters is effective in assuring that students document encounters with patients who present with specific clinical problems. The accuracy of our students' mandated logs is similar to previously published data. However, even inaccurate logs were rarely questioned or rejected by preceptors.