Systems-based practice assessed with a performance-based examination simulated and scored by standardized participants in the health care system: feasibility and psychometric properties.

BACKGROUND: Systems-based practice is one of the six general competencies proposed by the Accreditation Council for Graduate Medical Education in their Outcome Project. However, little has been published on its assessment--possibly because the systems-based practice competency has been viewed as difficult to define and measure. PURPOSE: The purpose of this study was to determine whether a full performance-based examination of systems-based practice cases simulated and scored by standardized participants in the health care system could feasibly be constructed and implemented that would provide reliable and valid measurements. METHODS: In the 1st year of the project (2008), four systems-based practice cases were developed and pilot tested with 13 residents. Videotapes of residents were studied to develop an instrument for subsequent assessment of performance by standardized participants. In the 2nd year (2009), the examination was expanded to a full 12 cases, which were completed by 11 second-year residents, and psychometric analyses were performed on the scores. RESULTS: The generalizability coefficient for the full 12-case examination based on scoring by standardized participants was .71, which is nearly equal to that based on scoring by faculty physician observers, which was .78. The correlation between total scores obtained with standardized participants and physician observers was .78. CONCLUSIONS: A performance-based examination can provide a feasible and reliable assessment of systems-based practice. However, attempts to evaluate convergent validity and discriminant validity-by correlating systems-based practice performance assessments with mean global ratings of residents on the 6 competencies by faculty throughout training-were unsuccessful, due to a lack of independence between the rated dimensions.

Aging Couple Across the Curriculum.

Aging Couple Across the Curriculum is a unique program designed around a couple who "age" a decade with each year of medical school. In these half-day sessions, students encounter the aging couple through a standardized patient experience. Interactive breakout sessions conducted by multidisciplinary professionals enhance student learning and appreciation of the contributions of the team of professionals. A panel of elder specialists provides personal insight into how they have overcome and/or adapted to various health-related problems. Evaluation measures have indicated that students are benefiting from the program and that it is affecting their attitudes in a positive way toward caring for older adults.

Development of the objective structured system-interaction examination.

STUDY OBJECTIVE: The purpose of this study was to develop an objective method of evaluating resident competency in systems-based practice. STUDY DESIGN: Faculty developed a 12-station examination, the Objective Structured System-Interaction Examination (OSSIE), patterned after the Objective Structured Clinical Examinations (OSCEs), to evaluate residents' ability to effectively work within the complex medical system of care. Scenarios consisted of multiple situations, such as patient hand-offs, consultations, complicated discharges, and family meetings, in which residents interacted with simulated professionals, simulated patients, and simulated family members to demonstrate the systems-based skills. Twelve second-year residents participated in the OSSIE. FINDINGS: Along with the standardized professionals, a faculty member provided the resident with immediate feedback and completed an evaluation form designed specifically to assess systems-based practice. Residents, faculty, and staff evaluated the OSSIE and felt it provided a rich learning experience and was a beneficial means of formative assessment. The residents' third-year learning experiences were adapted to meet their needs, and suggestions were offered for curriculum revision. DISCUSSION: The OSSIE is unique in that it uses standardized professionals, involves scenarios in a variety of settings, and incorporates current technology, including an electronic health record and a state-of-the-art simulation laboratory, into the examination. Challenges to implementation include faculty time, scheduling of residents, and availability of resources. CONCLUSION: By using the OSSIE, faculty are able to assess, provide constructive feedback, and tailor training opportunities to improve resident competence in systems-based practice. Reliability and validity of an instrument developed for use with the OSSIE are currently being determined.

The physician's role in assessing and counseling aging drivers: a training session for undergraduate medical students.

The critical need for physicians to become entrenched in the issues of older drivers and public safety is the focus of a training initiative developed as a component of an innovative geriatrics curriculum, Aging (Couple) Across the Curriculum. As the number of aging drivers in the United States rises, physicians can play an important role in helping older patients continue safe driving practices and in counseling those who need to cease driving. This article describes an inventive course designed to sensitize medical students to the many complex issues entangled in driving and aging and to prepare them to competently and compassionately assess and counsel older drivers. The act of driving connotes more than mobility. It reinforces one's independence, including the ability to go when and where one chooses. There must be a careful balance between an older person's privilege to drive and the public's right to safety. Through this training program, students can experience the complexity of this significant issue. This model has the potential to be utilized in other medical schools and could be adapted for use in interdisciplinary education.

A simulated couple aging across the four years of medical school.

BACKGROUND: The medical students of today will deliver health care to an ever-increasing aging population. Regardless of specialties, physicians must view older patients holistically with attention to medical, psychosocial, and family issues and functional status. DESCRIPTION: This program was designed around a couple who ages from 60 to 90 over the 4 years of medical school. The woman represents the healthy aging process, whereas the man represents an elder experiencing multiple health problems. Through standardized patient experiences, small-group encounters, and paper-based learning modules, students address typical age-related issues that provide experiences representative of aging in today's society. EVALUATION: Sessions were rated good to excellent. Comments were positive, especially regarding interactions with the aging couple. CONCLUSIONS: Aging Couple Across the Curriculum allowed the interweaving of multiple issues that are characteristic of many geriatric cases. Students were able to see the complexity of issues and the impact on other family members.