Enhancing provider engagement in practice improvement: a conceptual framework.

Engaging individual members of clinical teams in practice improvement initiatives is a challenge. In this commentary, we first summarize evidence supporting enhanced practitioner engagement through the creation of a work environment that builds on mutually respectful relationships and valued interdependencies. We then propose a phased, collaborative process that employs practice talk, a term that describes naturally occurring, collegial conversations among members of clinical teams. Planned interactions among team members, facilitated by individuals trained in dialogic techniques, enable health care providers and support staff to share their experiences and expertise, agree on what improvements they would like to make, and test the success of these changes. Participants would be encouraged to express their own suggestions for better practice and disclose strategies that are already working. Dissent would be regarded as an opportunity rather than a barrier. Iterative, sense-making conversations would generate a shared vision, enabling team members to engage in the entire process. Given that practice improvement ultimately depends on frontline providers, we encourage the exploration of innovative engagement strategies that will enable entire clinical teams to develop the collaborative learning skills needed to accomplish their goals.

Motivating learning and assessing outcomes in continuing medical education using a personal learning plan.

INTRODUCTION: Although there is increasing focus on provider behavior change as an outcome of continuing medical education (CME), it has long been known that an increase in knowledge alone is rarely sufficient to induce such change. The Personal Learning Plan (PLP), designed to motivate and assess CME learning, was partly derived from SMART goals (specific, measurable, attainable, realistic, and timely), a concept well supported as a strategy to promote behavior change. The goal of this study was to explore the relationship between SMART goals developed after attending a CME conference and subsequent provider behavior change, using the PLP as a tool. METHODS: PLPs were used as the outcome measure for Dartmouth-Hitchcock Continuing Medical Education conferences conducted during the fall of 2010. Three months later, participants were asked how close they were to completing their goals. All participants' goals were analyzed according to SMART criteria. RESULTS: Of the 841 participants attending conferences in fall 2010, 347 completed a PLP. An independent t-test found that among the 125 participants who completed the follow-up survey, those who indicated that they had completed their goal or were "very close" or "extremely close" to completing their goal wrote SMARTer goals than those who reported being "not at all close" to "moderately close" to completing their goal (t = 2.48, df = 123, p = 0.015). DISCUSSION: Our results corroborate previous research that has found "use of specific strategies to implement research-based recommendations seems to be necessary to ensure that practices change." Future directions include both a study of use of a PLP compared to a simple intent to change document and work on helping participants to write SMARTer goals.

The effect of computers for weight loss: a systematic review and meta-analysis of randomized trials.

BACKGROUND: The use of computers to deliver education and support strategies has been shown to be effective in a variety of conditions. We conducted a systematic review and meta-analysis to evaluate the impact of computer-based technology on interventions for reducing weight. METHODS: We searched MEDLINE, CENTRAL, CINAHL, PsycINFO, Google Scholar and ClinicalTrials.gov (all updated through June 2010) for randomized controlled trials evaluating the effect of computer-based technology on education or support interventions aimed at reducing weight in overweight or obese adults. We calculated weighted mean differences (WMD) and 95% confidence intervals (CI) using random effects models. RESULTS: Eleven trials with 13 comparisons met inclusion criteria. Based on six comparisons, subjects who received a computer-based intervention as an addition to the standard intervention given to both groups lost significantly more weight (WMD -1.48 kg, 95% CI -2.52, -0.43). Conversely, based on six comparisons, subjects for whom computer-based technology was substituted to deliver an identical or highly comparable intervention to that of the control group lost significantly less weight (WMD 1.47 kg, 95% CI 0.13, 2.81). Significantly different weight loss seen in "addition" comparisons with less than six months of follow-up (WMD -1.95 kg, 95% CI -3.50, -0.40, two comparisons) was not seen in comparisons with longer follow-up (-1.08 kg, 95% CI -2.50, 0.34, four comparisons). Analyses based on quality and publication date did not substantially differ. CONCLUSIONS: While the addition of computer-based technology to weight loss interventions led to statistically greater weight loss, the magnitude (<1.5 kg) was small and unsustained.

Shared decision making: skill acquisition for year III medical students.

BACKGROUND: A foundation of care within a Patient-centered Medical Home (PCMH) is respect for patients' values and preferences. Shared decision making (SDM) involves a set of principles and approaches to working with patients that integrates medical information and data with the preferences, values, and support systems of individual patients facing medical decisions. The value of SDM is increasingly evidenced by the incorporation of principles of SDM into the definitions of patient-centered care and PCMHs, accountable care organizations, and the language of the Health Reform Act of 2010. METHODS: We developed and integrated a curriculum on SDM in the third-year Family Medicine Clerkship at Dartmouth Medical School. The curriculum consisted of a mix of experiential, classroom, and online experiences designed to provide students with opportunities to learn content, practice skills, and share observations from their preceptorships. RESULTS: Student feedback was an important component of evaluating the SDM curriculum. Themes identified from students' reflections on their own behavior in a Simulated Patient Encounter included an increase in confidence and competence in their ability to use SDM, while noting the disconnect that may exist between what is taught in the clerkship and what they experience in their preceptorships. CONCLUSIONS: As this curriculum has developed, we have acquired a deep appreciation of the benefits and challenges of attempting to teach sophisticated communication and decision-making precepts to medical students who are working to master fundamentals of clinical work and who may or may not see such precepts reinforced in practice.

Enhancing practice improvement by facilitating practitioner interactivity: new roles for providers of continuing medical education.

Research into networking and interactivity among practitioners is providing new information that has the potential to enhance the effectiveness of practice improvement initiatives. This commentary reviews the evidence that practitioner interactivity can facilitate emergent learning and behavior change that lead to practice improvements. Insights from learning theories provide a framework for understanding emergent learning as the product of interactions between individuals in trusted relationships, such as occurs in communities of practice. This framework helps explain why some groups respond more favorably to improvement initiatives than others. Failure to take advantage of practitioner interactivity may explain in part the disappointingly low mean rates of practice improvement reported in studies of the effectiveness of practice improvement projects. Examples of improvement models in primary care settings that explicitly use relationship building and facilitation techniques to enhance practitioner interactivity are provided. Ingredients of a curriculum to teach relationship building in communities of practice and facilitation skills to enhance learning in small group education sessions are explored. Sufficient evidence exists to support the roles of relationships and interactivity in practice improvement initiatives such that we recommend the development of training programs to teach these skills to CME providers.

Mixed methods evaluation of an international internet-based continuing medical education course for pediatric HIV providers in Pune, India.

CONTEXT: Studies of HIV care in Pune, a high-HIV-prevalence city in India, have shown that a significant proportion of practitioners were not adhering to national guidelines due to inadequate awareness and understanding. OBJECTIVES: This study examined the effectiveness of a pilot Internet-based continuing medical education course in increasing knowledge of pediatric HIV diagnosis and treatment among providers in Pune. The study also explored perceived factors limiting the effectiveness of the pilot course. METHODS: The mixed methods evaluation design consisted of quantitative pre- and post-course knowledge assessments, and qualitative focus groups and in-depth interviews conducted on site with healthcare providers with experience treating HIV to explore the barriers to optimal course utilization, the applicability of the course content, and the systemic barriers to the implementation of physician knowledge. FINDINGS: There were significant increases (p<0.05) in mean knowledge scores on the global knowledge assessment and for two of five individual course modules. Perceived barriers to optimal course utilization were identified as being related to Internet access. The course content was reported to be generally useful, although certain guidelines and information were described as not congruent with local resource availability. Participants reported that the major barriers to implementing their knowledge were stigma that prevented patients from seeking care and financial resource limitations affecting physician practice. DISCUSSION AND CONCLUSIONS: This course resulted in a modest increase in pediatric HIV knowledge among Pune healthcare providers. Identification of perceived factors limiting the effectiveness of the course provides guidance for improving future Internet-based courses.

Shift work, light at night, and the risk of breast cancer.

Studies of the effect of shift work have identified several negative health outcomes, most notably breast cancer. Disruption of circadian rhythm by exposure to light at night has been identified as the mechanism likely responsible for this outcome. This article recommends that health care institutions work with occupational health nurses to develop and implement hazard communication and policies concerning shift work, exposure to light at night, and increased risk for negative health outcomes, particularly breast cancer.

Self-directed community health assessment projects in a required family medicine clerkship: an effective way to teach community-oriented primary care.

BACKGROUND AND OBJECTIVES: Community-oriented primary care (COPC) is a key teaching objective of many medical school family medicine clerkships. Though many programs are in place, little is published evaluating the effectiveness of curricula. METHODS: Within the family medicine clerkship at Dartmouth Medical School, students complete community health assessments. To assess the degree to which the student projects were meeting the goals and expectations of the clerkship assignment and COPC, project papers for 1 year were reviewed and coded using content analysis. RESULTS: Virtually all students fulfilled the stated goals of the project. The majority of students also demonstrated new skills, such as use of a database or creation of an improvement in the community. Students frequently covered populations such as homeless, children, or Native Americans, and selected topics not covered elsewhere in the medical school curriculum, such as oral health, effect of the environment, and educating providers. CONCLUSIONS: Students developed a variety of self-selected community health projects and public health interventions. Completing these projects in a core family medicine clerkship encouraged students to expand their views of health beyond the clinic and into the community.

Using mixed methods research in medical education: basic guidelines for researchers.

CONTEXT: Mixed methods research involves the collection, analysis and integration of both qualitative and quantitative data in a single study. The benefits of a mixed methods approach are particularly evident when studying new questions or complex initiatives and interactions, which is often the case in medical education research. Basic guidelines for when to use mixed methods research and how to design a mixed methods study in medical education research are not readily available. METHODS: The purpose of this paper is to remedy that situation by providing an overview of mixed methods research, research design models relevant for medical education research, examples of each research design model in medical education research, and basic guidelines for medical education researchers interested in mixed methods research. CONCLUSIONS: Mixed methods may prove superior in increasing the integrity and applicability of findings when studying new or complex initiatives and interactions in medical education research. They deserve an increased presence and recognition in medical education research.

Improving information management in primary care: the proof is in the pudding.

Generalists in both the USA and UK have been at the forefront of improving information management skills, defined here as the abilities required to locate and utilise synthesised information for patient care that is accessible, current, relevant and valid. Over the past decade, a variety of interventions designed to improve knowledge and skills relative to information management has been implemented. The goals of training are for learners to demonstrate long-term retention of knowledge and skills gained and to be able to transfer this learning from the context of training into different situations and contexts, such as those encountered in the workplace. Thus, to conclude that learning has taken place, it is essential to study performance after learners have acquired knowledge and skills to see how well those have been retained and generalised. The current study builds on previous work conducted by the authors that described and evaluated an intervention designed to improve information management knowledge, skills and use of Web-based resources by participants from generalist primary care practices. This cross-over study found that both groups of participants--those who received training initially and those who received training later--showed the same improvements when assessed 15 months and three months, respectively, after training. Given the definition of learning as 'relatively permanent', we wondered if these improvements would last. Participants in the original three phases of the study completed questionnaires during each phase; for the current study they were asked to complete a fourth questionnaire administered 27 and 15 months, respectively, after their original training. All variables showed non-significant differences between participants' scores at the end of the original study, where learning was assessed as having occurred, and the current administration of the questionnaire. Demonstrated long-term retention of knowledge and skills and generalisation to the workplace show that the goals of training have been met.

Do lessons learned in a training intervention on web-based health care resources diffuse to nonexposed members in the primary care setting? A comparative study.

OBJECTIVE: The Internet offers a significant information resource for health professionals. A strategy to improve the use of these resources is for health care providers and staff to receive specific training. The aim of this study was to determine whether those who attended an Internet health care resource training intervention transferred knowledge and skills to others in the practice. METHODS: Twenty-four primary care practices participated in the study in which 64 providers and staff attended a training intervention and 288 did not. A preintervention questionnaire that assessed knowledge, skill, and Internet usage was compared with a postintervention questionnaire. The main effect of interest in the linear model was the group by time interaction term, to determine whether knowledge and skill improved for both groups. RESULTS: There were 41 attendees and 222 nonattendees that completed both pre- and postintervention questionnaires. There were 9 variables that showed a possible diffusion pattern, in which both attendees and nonattendees improved between pre- and postintervention. Overall, the training intervention seemed to have impacted knowledge and skills of the respondents and also reported improvements in the clinical area of patient education, but frequency of use for most Web resources for medical decision making did not improve. CONCLUSION: An improvement strategy that depends on a training intervention for a few members in a practice may not necessarily transfer relative to all aspects of patient care.

A training intervention to improve information management in primary care.

BACKGROUND AND OBJECTIVES: Training programs designed to improve information management have been implemented but not adequately tested. Three critical components for information management were tested in a randomized control study: (1) knowledge of valid, synthesized summary information, (2) skills to use Web-based resources that provide access to these summaries, and (3) use of Web-based resources in clinical practice. METHODS: Twenty-four primary care practices were provided with computers and high-speed Internet access and then matched, with half randomly assigned to receive training and half to receive training at a later date. Training was designed to address knowledge, skills, and use of Web-based information. Outcomes were assessed by comparing baseline and follow-up questionnaires that focused on five conceptual domains related to Web-based resource use for patient care decisions and patient education. RESULTS: Compared to the delayed training group, the initial training group increased their knowledge and skill of Web-based resources and use for patient care decisions. Some measures of communication with patients about using Web-based resources and of incorporating use of Web-based resources into daily practice increased from baseline to follow-up for all participants. CONCLUSIONS: Our findings suggest that training and providing computers and Internet connections have measurable effects on information management behaviors.

Effects of a small-scale, very short-term service-learning experience on college students.

This study examined the effects of a small-scale, very short-term (8-10h) service-learning experience on college undergraduates. Repeated measures analysis of variance indicated that students participating in this experience reported maintenance of their sense of social responsibility, an increased sense of the meaningfulness of college, and an increased likelihood of choosing a service-related occupation, when compared to non-participating matched counterparts. These findings provide support for the notion that minimally resource-intensive service-learning programs can provide students with some of the benefits of service-learning that have been identified in longer, more intensive experiences.

The Dartmouth sleep knowledge and attitude survey: development and validation.

OBJECTIVES: To construct and validate an instrument for the purpose of assessing effectiveness of curriculum development and educational interventions in sleep medicine. BACKGROUND: Medical school and graduate medical curricula have historically contained quite limited instruction in sleep physiology and sleep medicine. Recent initiatives, particularly the Sleep Academic Award program, have attempted to address this issue through support of programs designed to develop educational resources and implement curriculum change. Effective measures for assessment of educational outcome are an essential component of these interventions. METHODS: A panel of sleep experts, other medical professionals, and education consultants developed a knowledge and attitude survey, designed to address a broad range of topics in sleep physiology and medicine. The instrument was modified in response to pilot testing, and a final 24-item survey has been administered to two cohorts of college undergraduates before and immediately following a ten-week course of instruction and to a single cohort of first-year medical students. A group of sleep medicine experts served as the comparison population. Results were analyzed for item difficulty, item discrimination, and instructional sensitivity. RESULTS: Analyses of item difficulty, item discrimination, and instructional sensitivity were in line with acceptable norms for criterion-referenced tests. There was evidence of discriminative validity, as shown by scores on the measure and acknowledged expertise. CONCLUSION: This instrument demonstrates both logical and empirical evidence of validity and may serve as a useful tool in assessing outcome of educational interventions in sleep medicine.

Developing resources to teach and assess the core competencies: a collaborative approach.

Graduate medical education programs face new challenges as they seek to comply with the mandate from the Accreditation Council on Graduate Medical Education to demonstrate that they are teaching and assessing residents on the six core competencies. The authors describe a project designed as a collaborative venture between the American Academy of Allergy, Asthma, and Immunology (AAAAI) and the Center for Educational Outcomes at Dartmouth College (CEdO) to provide residency programs in allergy/immunology with resources for teaching and assessing the core competencies. The goal was to create a set of learning and assessment resources that maximized the content knowledge expertise provided by the AAAAI and the learning expertise provided by CEdO. A highly interactive, iterative process was used to create a set of Web-based modules. Bilateral communication, buy-in, and active involvement in the process were seen as crucial to the development of resources and their successful implementation. Approximately 18 months after the modules were made available to training program directors, 80% of the directors surveyed were aware of and had accessed the modules. The joint creation process used in this project, designed to be generally applicable across specialties, reveals how the burden of meeting new requirements can be decreased when experts in content knowledge and experts in learning collaborate.

Fostering professionalism in medical education: a call for improved assessment and meaningful incentives.

Increasing attention has been focused on developing professionalism in medical school graduates. Unfortunately, the culture of academic medical centers and the behaviors that faculty model are often incongruent with our image of professionalism. The need for improved role modeling, better assessment of student behavior, and focused faculty development is reviewed. We propose that the incentive structure be adjusted to reward professional behavior in both students and faculty. The third-year medicine clerkship provides an ideal opportunity for clinician-educators to play a leading role in evaluating, rewarding, and ultimately fostering professionalism in medical school graduates.

A Longitudinal Study of Determinants of Career Satisfaction in Medical Students.

CONTEXT: There is evidence of significant career dissatisfaction among practicing physicians and those considering medicine as a profession. Most research on career satisfaction has examined practicing physicians. This study was undertaken to look at determinants of satisfaction in those at the earliest stage of their medical careers - medical students. METHODS: As part of a larger study, students comprising one class at the University of Washington School of Medicine were surveyed three times over the course of their medical education. For the present study we examined measures specifically related to determinants of career satisfaction. FINDINGS: Over time, students' sense of the importance of most measured determinants of satisfaction showed significant change, the majority of which were in the direction of decreased importance. However, most of the change was relative. That is, factors that students considered to be most important at the start of medical school continued to be most important throughout the educational experience and those factors students considered to be least important at Year 1 continued to be least important at Years 2 and 4. DISCUSSION: These findings have implications for medical education, a time when students are forming expectations that will impact their career satisfaction. In addition to information on career satisfaction, students should understand the professional values of medicine, their own values and expectations, current practice patterns, economics, and the role of advocacy.

Factors associated with palliative care knowledge among internal medicine house staff.

PURPOSE: To assess knowledge and associated factors in palliative care. METHODS: Self-administered survey of 88 internal medicine house officers in 1996. RESULTS: Twenty-one interns and 36 residents completed the survey for a response rate of 65%. Most house officers reported 1-5 hours of prior formal training in palliative care, 1-5 hours in pain management, and 6-20 hours in ethics. The mean knowledge score was 75% correct (SD = 8); pain management scores were lowest (70%). Overall, interns had a significantly lower mean score than residents (70% vs. 77%; p = 0.001). In multivariate analysis, only the year of residency was significantly associated with knowledge score; prior formal training in palliative care, pain management, or ethics was not. One third of house officers rated themselves as "not at all" or "only slightly" at ease in caring for a dying patient. These self-ratings were not associated with prior training or knowledge, but were higher in residents compared to interns. CONCLUSIONS: Palliative care knowledge and ease with dying patients were higher in later years of residency but were not associated with prior formal palliative care training. These data highlight the continued need to evaluate and improve training in palliative care and pain management.