Development and testing of an assessment instrument for the formative peer review of significant event analyses.

AIM: To establish the content validity and specific aspects of reliability for an assessment instrument designed to provide formative feedback to general practitioners (GPs) on the quality of their written analysis of a significant event. METHODS: Content validity was quantified by application of a content validity index. Reliability testing involved a nested design, with 5 cells, each containing 4 assessors, rating 20 unique significant event analysis (SEA) reports (10 each from experienced GPs and GPs in training) using the assessment instrument. The variance attributable to each identified variable in the study was established by analysis of variance. Generalisability theory was then used to investigate the instrument's ability to discriminate among SEA reports. RESULTS: Content validity was demonstrated with at least 8 of 10 experts endorsing all 10 items of the assessment instrument. The overall G coefficient for the instrument was moderate to good (G>0.70), indicating that the instrument can provide consistent information on the standard achieved by the SEA report. There was moderate inter-rater reliability (G>0.60) when four raters were used to judge the quality of the SEA. CONCLUSIONS: This study provides the first steps towards validating an instrument that can provide educational feedback to GPs on their analysis of significant events. The key area identified to improve instrument reliability is variation among peer assessors in their assessment of SEA reports. Further validity and reliability testing should be carried out to provide GPs, their appraisers and contractual bodies with a validated feedback instrument on this aspect of the general practice quality agenda.

Use of the paired-comparison technique to determine the most valued qualities of the McMaster Medical Programme Admissions Process.

The Undergraduate Medical Programme at McMaster University selects students using a comprehensive set of tools. Attempts to modify the selection process over many years have been impeded by an inability to reconcile very strongly held views among stakeholders as to the importance of the selection tools and, indeed, the very purposes of the admission process. The objective of this study was to identify key 'qualities' of the selection process and to measure their relative importance to admissions process assessors. Through a qualitative review of internal research documents, Medical Programme Admissions Committee meeting minutes, memos and accreditation surveys eight qualities of the admissions process were identified: validity, fairness, accessibility, comprehensiveness, affordability, legal defensibility, contribution to class diversity and the role of the process as a public statement of the Programme's values. Faculty, students and community admissions assessors were surveyed, by mail, using a paired-comparisons technique. The overall response rate was 58%. By a wide margin, all three groups of admissions assessors valued validity and fairness most highly. The least valued qualities were affordability and the role of the process as a statement of our values. Possible applications of this approach to the admissions process deliberations are discussed.

Attitudes toward tuberculosis of final year medical students from Canada, India, and Uganda.

BACKGROUND: Tuberculosis, although both preventable and treatable, continues to be the world's leading cause of death from an infectious agent. PURPOSE: To extend the results of our previous study of knowledge and practices of final year medical students regarding tuberculosis at three medical schools from endemic and nonendemic areas and to describe their attitudes. METHODS: Eight statements assessing attitudes, as part of a larger survey, were administered to final year medical students at McMaster University in Canada, Christian Medical College in India, and Makerere University in Uganda. RESULTS: One hundred sixty surveys were returned with 155 completed attitude responses. The response rate was 68.4% (65 of 95) for McMaster University, 39.7% (23 of 58) for the Christian Medical College, and 78.3% (72 of 92) for Makerere University. Analysis showed that six of eight attitude items were slightly statistically different among the schools with minimal effect of curriculum time and patient exposure. CONCLUSIONS: Despite quite varied exposure to tuberculosis, students from endemic and nonendemic areas responded similarly on statements addressing attitudes toward tuberculosis.

Randomised, blinded, placebo controlled trial of doxycycline for chronic seronegative arthritis.

OBJECTIVE: To determine whether long term doxycycline improves symptoms in patients with chronic seronegative or reactive arthritis. METHODS: A randomised, triple blind, controlled clinical trial of three months' treatment with doxycycline or placebo of patients with chronic reactive or seronegative arthritis was conducted. The primary study end points were three month pain and functional status measured by a self administered Arthritis Impact Measurement Scales version 2 (AIMS2) quality of life questionnaire. Secondary end points were pain and functional status at 6-12 months, three month rheumatologist assessed joint count, pain, and arthritis activity, and treatment efficacy in those with previous exposure to chlamydia. RESULTS: Of 60 patients randomly allocated to receive doxycycline or placebo, results from 37 were evaluable at three months. Groups were well balanced for major prognostic variables. Doxycycline had no detectable effect at three months on pain change scores (mean difference 1.5, 95% CI -1.2 to 4.2, p=0.25) or composite functional change scores (mean difference 0.8, 95% CI -5.6 to 7.1, p=0.81). Furthermore, there were no differences in secondary study end points, and no apparent treatment effect in patients with previous chlamydia infection. CONCLUSION: Three months' treatment with doxycycline did not improve pain or functional status in patients with chronic reactive or seronegative arthritis.

Weekly intravenous methotrexate with folinic acid for nonmetastatic gestational trophoblastic neoplasia.

OBJECTIVE: The objective of this study was to determine the complete response rate to weekly intravenous methotrexate at 100 mg/m(2) with folinic acid for patients with nonmetastatic gestational trophoblastic neoplasia. METHODS: From 1988 to 1999, 22 women with nonmetastatic gestational trophoblastic neoplasia were treated with weekly intravenous methotrexate with folinic acid at the Hamilton Regional Cancer Centre. Complete response was defined as the attainment of a serum beta-hCG level <5 IU/L for 3 consecutive weeks. Toxicity was graded according to the National Cancer Institute of Canada-Clinical Trials Group criteria for chemotherapy toxicity. RESULTS: There were 10 women who achieved complete response with weekly intravenous methotrexate alone (45.5%). Of the 12 who did not achieve complete response with methotrexate, 10 received actinomycin D and 2 received EMA as second-line chemotherapy. Patients successfully treated with methotrexate required a median of 6.5 cycles (including 2 cycles for consolidation) to achieve complete response. The only significant prognostic factor for failure with methotrexate was pretreatment beta-hCG (P = 0.01). CONCLUSIONS: Only a select group of patients with low pretreatment beta-hCG titers would be expected to achieve complete response with this regimen. Large randomized studies are required to determine the optimal treatment for nonmetastatic gestational trophoblastic neoplasia.

Knowledge and practices regarding tuberculosis: a survey of final-year medical students from Canada, India and Uganda.

CONTEXT: Tuberculosis is one of the most common infectious diseases worldwide and is responsible for the largest number of deaths from a single infectious cause. OBJECTIVE: The objective of this study was to compare the knowledge of and practices regarding tuberculosis in final-year medical students at schools from endemic and non-endemic areas. SUBJECTS: Final-year medical students at McMaster University in Canada, the Christian Medical College in India, and Makerere University in Uganda. METHODS: A questionnaire consisting of 20 multiple-choice questions assessing knowledge, practices, and exposure. A total knowledge score (maximum=13) and a total practice score (maximum=5) were created for each study site. RESULTS: 160 questionnaires were returned; the response rate was 68.4% (65/95) for McMaster University, 39.7% (23/58) for the Christian Medical College and 78.3% (72/92) for Makerere University. Students from Makerere University had the highest knowledge scores but differences were non-significant after adjustment for patient exposure and curriculum time (F(2,153)= 1.80, P=0.16). Differences in practice scores, however, remained significant after adjusting for curriculum time and patient exposure (F(2,153)=5.14, P=0.006). Knowledge score (F(1,156)=5.05, P=0.02), patient exposure (F(1,153)=9.11, P=0.003), and curriculum time and patient exposure (F(2,153)=5.14, P=0.006) were statistically significant positive predictors of the total practice score. CONCLUSIONS: This study demonstrated significant differences in undergraduate exposure to tuberculosis, total knowledge, and practice competency at three medical schools in Canada, India, and Uganda. In general, the knowledge base and practice competency of all three graduating classes was adequate.

Predicting performance on the Royal College of Physicians and Surgeons of Canada internal medicine written examination.

BACKGROUND: Although the written component of the Royal College of Physicians and Surgeons of Canada (RCPSC)internal medicine examination is important for obtaining licensure and certification as a specialist, no methods exist to predict a candidate's performance on the examination. METHOD: We obtained data from 5 Canadian universities from 1988 to 1998 in order to compare raw scores from the American Internal Medicine In-Training Examination (AIMI-TE) with raw scores and outcomes (pass or fail) of the written component of the RCPSC internal medicine examination. RESULTS: Mean scores on the AIMI-TE correlated well with scores on the RCPSC internal medicine written examination for all postgraduate years (r = 0.62, r = 0.55 and r = 0.65 for postgraduate years 1, 2 and 3 respectively). Scores above the 50th percentile on the AIMI-TE w/ere predictive of a low failure rate (< 1.5%) on the RCPSC internal medicine written examination, whereas scores at or below the 10th percentile were associated with a high failure rate (about 24%). INTERPRETATION: Candidates who are eligible to take the written component of the RCPSC certification examination in internal medicine can use the AIMI-TE to predict their performance on the Canadian examination. The AIMI-TE is a useful test for residents in all levels of training, because the examination scores have a strong relation to expected performance on the Canadian examination for each year of postgraduate training.

IOIBD questionnaire on the clinical use of azathioprine, 6-mercaptopurine, cyclosporin A and methotrexate in the treatment of inflammatory bowel diseases.

OBJECTIVE: To obtain information on the clinical experience with azathioprine (AZA), 6-mercaptopurine (6-MP), cyclosporin A (CyA) and methotrexate (MTX) in the treatment of patients with inflammatory bowel disease (IBD) by gastroenterologists and internists in different countries. DESIGN: A questionnaire designed by the International Organization for the Study of Inflammatory Bowel Disease (IOIBD) was mailed to 300 gastroenterologists, living in North America (n = 76) and Europe (n = 224) (12 countries), to obtain information on clinical experience. PARTICIPANTS: More than half of the respondents (168/298; 56.4%) worked in university hospitals and 58/298 (19.5%) in general (non-university) hospitals. Two-thirds (65%) had more than 10 years' experience in gastroenterology. RESULTS: The respondents had personal experience with AZA (88.4%), 6-MP (33.3%), CyA (48.7%) and MTX (36.3%). AZA was prescribed more frequently in Europe (92.6%) than in North America (74.2%) (P = 0.0002), 6-MP less frequently by the European than the North American respondents (23.8 and 53.3% respectively, P = 0.0001). Two-thirds (69.7%) usually prescribed AZA together with steroids to Crohn's disease patients; 62.4% of the respondents prescribed AZA for periods longer than 24 months. For ulcerative colitis, 77.9% had experience with AZA (Europe > North America, P = 0.0001). AZA had been prescribed by 69 respondents to pregnant patients, without apparent toxicity. Acute pancreatitis had been observed after AZA by 56.7% respondents; 25 malignancies were mentioned (six lymphoma, three leukaemia, three colon cancer, four renal carcinoma, nine others). CyA had been prescribed in acute ulcerative colitis by 140/291 respondents (North America 45.1%, Europe 49.1 %); of all respondents 63.9% treated < 5 patients with CyA, 36.1% 6-20 cases. CyA results were considered good in 29.5%, acceptable but with recurrences in 58.6%, and poor in 14.3%. MTX was prescribed in North America by 47.8% of the respondents, and by 33.9% in Europe (not significant). Several significant differences were observed between the prescription behaviour of respondents working at university hospitals and non-university hospitals, in particular in relation to participation in clinical trials. CONCLUSIONS: Considerable experience exists in the use of immunosuppressive therapy in IBD; however, differential prescription behaviour exists in the choice of immunosuppressives between North America and Europe. These IOIBD study results may contribute to a better insight in the daily use of immunosuppressive agents in IBD by gastroenterologists and other specialists.

An 18-month follow-up on the effectiveness of a sexuality workshop: some methodological pitfalls.

Students from four health sciences programs (Medicine, Nursing, Occupational Therapy and Physiotherapy) participated in a two-day Sexuality Workshop and completed pre- and post-workshop questionnaires dealing with sexual attitudes, comfort, and knowledge. Eighteen months later, these students were asked to complete the same questionnaires. Not only had they sustained the gains made during the workshop, but they continued to show improvement in the three areas during this interval. However, of greater importance than the positive finding is the need to understand the range of methodological problems encountered in carrying out this type of follow-up educational inquiry and in seeking measures for improving the validity and generalizability of such research.