Developing clinical competencies to assess learning needs and outcomes: the experience of the CS2day initiative.

An outcomes-based education (OBE) approach was desired for the CS2day initiative, and the size and scope of the initiative compelled a consistent and cohesive framework in order to apply such an approach. A series of competency statements were developed to provide that framework. The competency statements were based on current clinical guidelines, and further refined through stakeholder interviews and expert feedback. These competency statements were utilized throughout the CS2day initiative as the foundation for needs assessments, activity planning and development, and outcomes measurement. The CS2day partners found these competencies useful in developing over 150 educational activities in the initiative. The competencies became the firm linkages between the needs and outcomes measures used for all CS2day activities, and enabled a summative evaluation to be developed for the entire initiative. This article describes the rationale for developing the competencies, the methods used to deploy them, and the tools that incorporated these competencies.

Evaluation of a collaborative program on smoking cessation: translating outcomes framework into practice.

INTRODUCTION: Although evaluating at multiple outcome levels has been proposed for continuing education activities and programs, it is a complex undertaking and is not done routinely, especially in collaborative, multicomponent programs. This article reports on strategies used and results obtained in an evaluation project that examined multiple outcomes of a US-based collaborative, multicomponent smoking cessation educational program for clinicians. METHODS: Evaluation was organized conceptually around the 6 levels of an outcomes-based evaluation model and was conducted using registration data, postactivity evaluations, clinical vignettes with questions assessing knowledge and competence in participants and in a comparison group, a commitment to change approach, data from patient charts to assess clinician compliance on 8 performance measures, and tobacco cessation rates. Additional methods included a success case method study of 9 practices participating in performance improvement (PI) activities and assessment of partner collaboration using a written survey and interviews. RESULTS: The program reached more than 43,000 clinicians who participated in a variety of activities. Participants indicated a high level of satisfaction with the program's educational activities and demonstrated higher scores than a comparison group on 6 of 7 competencies. The majority of participants who responded to commitment to change questions reported intended and implemented practice changes consistent with desired outcomes. Performance outcomes of 3 PI activities varied, with greater improvements observed in 1 activity (9.0% to 36.2% improvement across 8 measures). Lower performance outcomes, but a smoking quit rate of 46.8%, was observed in 2 other PI activities. DISCUSSION: The program had an overall positive impact on the measured variables for clinicians and their patients. Use of 1 outcomes assessment framework acceptable to all members of the collaborative, common measures and evaluation techniques, and centralized data repositories contributed to the success of the program evaluation reported here and is recommended to others who are considering a collaborative program evaluation.

Sharing collaborative designs of tobacco cessation performance improvement CME projects.

INTRODUCTION: Performance Improvement Continuing Medical Education (PI CME) provides an important opportunity for CME providers to combine educational and quality health care improvement methodologies. Very few CME providers take on the challenges of planning this type of intervention because it is still a new practice and there are limited examples from which to model. This article offers case examples of educational design, results, and lessons learned from 4 tobacco cessation PI CME activities. PURPOSE: To share with the CME community different cases of PI CME educational design and results so that CME providers may have examples to draw from and develop more PI CME activities. METHODS: This is a case report. RESULTS: Four of 9 partners of the Cease Smoking Today (CS2day) initiative developed the 4 tobacco cessation PI CME activities. Each project was designed and implemented using a common planning framework and clinical performance measures but with varying operational and educational design components depending on the strengths and resources of the leading partner. Three projects that are completed show improvements in aggregated practitioner performance and smoking quit rates. One project is currently under way. DISCUSSION: These cases highlight the value of collaboration, identify influences of operational and educational designs on variation in compliance with performance measures, and lead to a discussion of similarities in barriers, successes, and lessons for future practice.