Evaluating the effectiveness of 2 educational interventions in family practice.

BACKGROUND: Structured feedback of information can produce change in physician behaviour. The objective of this study was to assess the effectiveness of 2 educational interventions for improving the quality of care provided by family physicians in Ontario: the Practice Assessment Report (PAR) and the Continuing Medical Education Plan (CMEP) with a follow-up visit by a mentor. METHODS: The study was a randomized controlled trial. Physicians in the control group received only the PAR, whereas those in the experimental group received the PAR, CMEP and mentor interventions. The participants were 56 family physicians and general practitioners (27 in the PAR group and 29 in the CMEP group) in southern Ontario who agreed to participate in the interventions and provide data. A total of 2395 patients randomly sampled from the practices returned questionnaires and consented to have their medical records abstracted. The outcome measures were global scores in 4 areas--quality of care, charting, prevention and overall use of medications--and patient ratings of satisfaction with care and preventive practices. The measures were applied at the beginning (phase 1) and end (phase 2) of the study. RESULTS: The mean global scores at the end of the study for the PAR group were 70.1% for quality of care, 84.7% for prevention, 77.7% for charting and 82.2% for overall use of medications. The corresponding scores for the CMEP group were 68.3%, 82.1%, 76.4% and 83.2%. In the patient satisfaction component, the personal care scores at phase 2 were 93.6% for the PAR group and 94.6% for the CMEP group. Examples of the scores for prevention for the PAR group were 98.3% for children's current immunization, 96.6% for blood pressure measured within the previous 5 years, 79.4% for referral of women of the appropriate age for mammography within the previous 2 years, and 58.4% for discussion about alcohol use. The corresponding scores for the CMEP group were 95.8%, 97.6%, 77.6% and 64.6%. The changes in mean scores between phase 1 and phase 2 ranged from -1.9 to 2.3 points. There were no significant differences between the 2 groups in phase 1 or phase 2 scores or in change in scores. A total of 64.3% of the physicians rated the PAR as useful, 26.5% found the CMEP to be useful, and 41.0% considered the mentor strategy to be a useful form of continuing medical education. Although changes in practice related to the PAR, CMEP or mentor were reported by some physicians, they were not related to chart audit or patient scores. INTERPRETATION: Educational interventions based on quality-of-care assessments and directed to global improvements in quality of care did not result in improvements in the outcome measures. Educational interventions may have to be targeted to specific areas of the practice, with physicians being monitored and receiving ongoing feedback on their performance.

Recruiting family physicians as participants in research.

Obtaining the voluntary participation of family physicians in quality of care research is a major problem in family practice research. An innovative approach was therefore required to recruit 120 randomly selected family physicians in southern Ontario in a quality of care study by the College of Family Physicians of Canada. A network of physician recruiters oriented to the study was organized for each district. This recruitment method resulted in an 84.5% participation rate. The relationship of these physician recruiters to the candidate and the method of approach were important factors in the enrolment process: the highest participation rate (95%) was obtained when the recruiters were friends of the candidate and when a personal meeting was arranged (91%). Recruiters were given an information package to help them in the recruitment process and rated the most useful items as follows: a policy statement about confidentiality, a description of the study and reprints of a published feasibility study. These results illustrate that cooperation in research in family physicians' offices can become a reality.

Quality of care in family practice: does residency training make a difference?

As the proportion of physicians who enter residency training in family practice steadily increases, so does the need to evaluate the impact of their training and postgraduate education on the quality of care in their practices. We audited the practices of 120 randomly selected family physicians in Ontario, who were separated into four groups: nonmembers of the College of Family Physicians of Canada (CFPC), members of the CFPC with no certification in family medicine, certificated members without residency training in family medicine and certificated members with residency training in family medicine. The practices were assessed according to predetermined criteria for charting, procedures in periodic health examination, quality of medical care and use of indicator drugs. Generally the scores were significantly higher for CFPC members with residency training in family medicine than for those in the other groups, nonmembers having the lowest scores. Patient questionnaires indicated no difference in satisfaction with specific aspects of care between the four groups. Self-selection into residency training and CFPC membership may account for some of the results; nevertheless, the findings support the contention that residency training in family medicine should be mandatory for family physicians.

Study of relation of continuing medical education to quality of family physicians' care.

A random sample of 120 physicians in Ontario was studied to assess quality of care in primary care and test an hypothesis that quality of care was related to continuing medical education (CME) activities. The quality-of-care scores were obtained by an in-office audit of a random selection of charts. The scores were global scores for charting, prevention, the use of 13 classes of drugs, and care of a two-year period for 182 different diagnoses. There were no relationships between global quality-of-care scores based on these randomly chosen charts and either the type or quantity of the physicians' CME activities. These activities were reading journals, attending rounds, attending scientific conferences, having informal consultations, using audio and video cassettes, and engaging in self-assessment. The implications of these findings are significant for future research in CME and for planners of present CME programs.

Assessing the quality of care in family physicians' practices.

This feasibility study by the Practice Assessment Committee of the College of Family Physicians of Canada was conducted to define and produce instruments that could be used to assess quality of care rendered in family physicians' offices. The favorable response to these evaluations and the acceptance of the results indicates that this method can be useful to family physicians. The instruments identify family physicians' strengths and deficiencies so that with appropriate changes in the quality and efficiency of care, they are able to achieve higher levels of professional satisfaction. These methods may ultimately be used to establish acceptable standards for care given by family physicians in their offices.