Tools to Assess Behavioral and Social Science Competencies in Medical Education: A Systematic Review.

PURPOSE: Behavioral and social science (BSS) competencies are needed to provide quality health care, but psychometrically validated measures to assess these competencies are difficult to find. Moreover, they have not been mapped to existing frameworks, like those from the Liaison Committee on Medical Education (LCME) and Accreditation Council for Graduate Medical Education (ACGME). This systematic review aimed to identify and evaluate the quality of assessment tools used to measure BSS competencies. METHOD: The authors searched the literature published between January 2002 and March 2014 for articles reporting psychometric or other validity/reliability testing, using OVID, CINAHL, PubMed, ERIC, Research and Development Resource Base, SOCIOFILE, and PsycINFO. They reviewed 5,104 potentially relevant titles and abstracts. To guide their review, they mapped BSS competencies to existing LCME and ACGME frameworks. The final included articles fell into three categories: instrument development, which were of the highest quality; educational research, which were of the second highest quality; and curriculum evaluation, which were of lower quality. RESULTS: Of the 114 included articles, 33 (29%) yielded strong evidence supporting tools to assess communication skills, cultural competence, empathy/compassion, behavioral health counseling, professionalism, and teamwork. Sixty-two (54%) articles yielded moderate evidence and 19 (17%) weak evidence. Articles mapped to all LCME standards and ACGME core competencies; the most common was communication skills. CONCLUSIONS: These findings serve as a valuable resource for medical educators and researchers. More rigorous measurement validation and testing and more robust study designs are needed to understand how educational strategies contribute to BSS competency development.

Unobtrusive measurement of daily computer use to detect mild cognitive impairment.

BACKGROUND: Mild disturbances of higher order activities of daily living are present in people diagnosed with mild cognitive impairment (MCI). These deficits may be difficult to detect among those still living independently. Unobtrusive continuous assessment of a complex activity such as home computer use may detect mild functional changes and identify MCI. We sought to determine whether long-term changes in remotely monitored computer use differ in persons with MCI in comparison with cognitively intact volunteers. METHODS: Participants enrolled in a longitudinal cohort study of unobtrusive in-home technologies to detect cognitive and motor decline in independently living seniors were assessed for computer use (number of days with use, mean daily use, and coefficient of variation of use) measured by remotely monitoring computer session start and end times. RESULTS: More than 230,000 computer sessions from 113 computer users (mean age, 85 years; 38 with MCI) were acquired during a mean of 36 months. In mixed-effects models, there was no difference in computer use at baseline between MCI and intact participants controlling for age, sex, education, race, and computer experience. However, over time, between MCI and intact participants, there was a significant decrease in number of days with use (P = .01), mean daily use (∼1% greater decrease/month; P = .009), and an increase in day-to-day use variability (P = .002). CONCLUSIONS: Computer use change can be monitored unobtrusively and indicates individuals with MCI. With 79% of those 55 to 64 years old now online, this may be an ecologically valid and efficient approach to track subtle, clinically meaningful change with aging.

Models of cognitive performance based on home monitoring data.

Modeling cognitive performance using home monitoring data is a new approach to managing neurologic conditions and for monitoring the effects of cognitive exercise interventions. The data consists of activity monitoring from motion sensors and specific cognitive metrics embedded within our adaptive computer games. The frequency and continuity of data collection allows us to analyze within subject trends of cognitive performance and to assess day to day variability. This approach provides a framework for clinicians and care managers to have the potential of detecting patients' cognitive problems early and to have timely feedback on treatment interventions.

The exploration & forensic analysis of computer usage data in the elderly.

Unobtrusive in-home computer monitoring could one day be used to deliver cost-effective diagnostic information about the cognitive abilities of the elderly. This could allow for early detection of cognitive impairment and would additionally be coupled with the cost advantages that are associated with a semi-automated system. Before using the computer usage data to draw conclusions about the participants, we first needed to investigate the nature of the data that was collected. This paper represents a forensics style analysis of the computer usage data that is being collected as part of a larger study of cognitive decline, and focuses on the isolation and removal of non user-generated activities that were recorded by our computer monitoring software (CMS).