Mobile technology for the facilitation of direct observation and assessment of student performance.

BACKGROUND: We developed, implemented, and assessed a web-based clinical evaluation application (i.e., CEX app) for Internet-enabled mobile devices, including mobile phones. The app displays problem-specific checklists that correspond to training problems created by the Clerkship Directors in Internal Medicine. PURPOSE: We hypothesized that use of the CEX app for directly observing students' clinical skills would be feasible and acceptable, and would demonstrate adequate reliability and validity. METHODS: Between July 2010 and February 2012, 266 third-year medical students completed 5 to 10 formative CEXs during their internal medicine clerkship. The observers (attendings and residents), who performed the CEX, used the app to guide and document their observations, record their time observing and giving feedback to the students, and their overall satisfaction with the CEX app. Interrater reliability and validity were assessed with 17 observers who viewed 6 videotaped student-patient encounters, and by measuring the correlation between student CEX scores and their scores on subsequent standardized-patient Objective Structured Clinical Examination (OSCE) exams. RESULTS: A total of 2,523 CEXs were completed by 411 observers. The average number of evaluations per student was 9.8 (± 1.8 SD), and the average number of CEXs completed per observer was 6 (± 11.8 SD). Observers spent less than 10 min on 45.3% of the CEXs and 68.6% of the feedback sessions. An overwhelming majority of observers (90.6%) reported satisfaction with the CEX. Interrater reliability was measured at 0.69 among the observers viewing the videotapes, and their ratings discriminated between competent and noncompetent performances. Student CEX grades, however, did not correlate with their end of 3rd-year OSCE scores. CONCLUSIONS: The use of this CEX app is feasible and it captures students' clinical performance data with a high rate of user satisfaction. Our embedded checklists had adequate interrater reliability and concurrent validity. The grades measured on this app, however, were not predictive of subsequent student performance.

An online method for diagnosis of difficult TB cases for developing countries.

Optimal use of limited human, technical and financial resources is a major concern for tuberculosis (TB) control in developing nations. Further impediments include a lack of trained physicians, and logistical difficulties in arranging face-to-face (f-2-f) TB Diagnostic Committee (TBDC) consultations. Use of e-Health for virtual TBDCs (Internet and "iPath"), to address such issues is being studied in the Philippines and Pakistan. In Pakistan, radiological diagnosis of 88 sputum smear negative but suspected TB patients has been compared with the 'gold standards' (TB culture, and 2-month clinical follow up). Of 88 diagnostic decisions made by primary physicians at the spoke site and electronic TBDC (e-TBDC) at hub site, there was agreement in 71 cases and disagreement on 17 cases. The turn-around time (TAT; patient registration at spoke site for f-2-f diagnosis to receiving the electronic diagnosis), averaged 34.6 hours; ranging 9 minutes to 289.2 hours. Average TAT at the rural site (59.15 hours) was more than the urban site (15.9 hours). Comparison of e-TBDC and f-2-f diagnosis with the gold standards showed only slight differences. Using culture as the gold standard, e-TBDC decisions showed greater accuracy (sensitivity - 32.4%) as compared to f-2-f (27.6%); using 2-month clinical follow-up as the gold standard, f-2-f diagnosis showed slightly better improvement in patient symptoms and weight as compared to e-TBDC. In Philippines "iPath" was trialed and demonstrated that e-TBDCs have potential. Such groups could review cases, diagnose, and write comments remotely, reducing the diagnosis and treatment delay compared to usual care.