Healthcare systems and the sciences of health professional education.

Health professions education is that part of the education system which applies educational philosophy, theory, principles and practice in a complex relationship with busy clinical services, where education is not the primary role. While the goals are clear-to produce the health workforce that society needs to improve health outcomes-both education and healthcare systems continue to evolve concurrently amidst changes in knowledge, skills, population demographics and social contracts. In observing a significant anniversary of this journal, which sits at the junction of education and healthcare systems, it is appropriate to reflect on how the relationship is evolving. Health professions educators must listen to the voices of regulators, employers, students and patients when adapting to new service delivery models that emerge in response to pressures for change. The recent COVID-19 pandemic is one example of disruptive change, but other factors, such as population pressures and climate change, can also drive innovations that result in lasting change. Emerging technology may act as either a servant of change or a disruptor. There is a pressing need for interdisciplinary research that develops a theory and evidence base to strengthen sustainability of change.

Twelve tips for establishing a new medical school.

Many new medical programs have been established during the last 20 years, and this trend seems set to continue as the health care needs of the world's populations become more complex and demand increases for more physicians to provide the necessary health care. In this paper, we address how best to establish a new medical school, based on our experiences in new ventures in several countries. Success requires a combination of boldness of vision, support from many stakeholder groups, adequate financial and human resources, educational expertise, confidence, patience, and persistence.

A rural undergraduate campus in England: virtue from opportunity and necessity.

The implementation of new curriculum at Keele University Medical School, UK has made heavy use of general practice as a locus for learning. This has necessitated a substantial expansion in the School's teaching network. The School's hinterland includes a large rural area with a number of excellent general practices and associated community hospitals that, to date, have been unable to teach undergraduates because of their inaccessibility. This article describes how the School and its partners articulated a vision to establish a rural campus with an associated rural accommodation hub, and the challenges involved in establishing and sustaining the campus.

Are medical student results affected by allocation to different sites in a dispersed rural medical school?

INTRODUCTION: As medical education becomes more decentralised, and greater use is made of rural clinical schools and other dispersed sites, attention is being paid to the quality of the learning experiences across these sites. This article explores this issue by analysing the performance data of 4 cohorts of students in a dispersed clinical school model across 4 sites. The study is set in a newly established medical school in a regional area with a model of dispersed education, using data from the second to fifth cohorts to graduate from this school. METHODS: Summative assessment results of 4 graduating cohorts were examined over the final 2 years of the course. Two analyses were conducted: an analysis of variance of mean scores in both years across the 4 sites; and an analysis of the effect of moving to different clinical schools on the students' rank order of performance by use of the Kruskal-Wallis test. RESULTS: Analysis revealed no significant difference in the mean scores of the students studying at each site, and no significant differences overall in the median ranking across the years. Some small changes in the relative ranking of students were noticed, and workplace-based assessment scores in the final year were higher than the examination-based scores in the previous year. CONCLUSIONS: The choice of clinical school site for the final 2 years of an undergraduate rural medical school appears to have no effect on mean assessment scores and only a minor effect on the rank order of student scores. Workplace-based assessment produces higher scores but also has little effect on student rank order. Further studies are necessary to replicate these findings in other settings and demonstrate that student learning experiences in rural sites, while popular with students, translate into required learning outcomes, as measured by summative assessments.

Developing the accredited postgraduate assessment program for Fellowship of the Australian College of Rural and Remote Medicine.

INTRODUCTION: Accreditation of the Australian College of Rural and Remote Medicine (ACRRM) as a standards and training provider, by the Australian Medical Council (AMC) in 2007, is the first time in the world that a peak professional organisation for rural and remote medical education has been formally recognised. As a consequence, the Australian Government provided rural and remote medicine with formal recognition under Medicare as a generalist discipline. This accreditation was based on the ability of ACRRM to meet the AMC's guidelines for its training and assessment program. METHODS: The methodology was a six-step process that included: developing an assessment blueprint and a classification scheme; identifying an assessment model; choosing innovative summative and formative assessment methods that met the needs of rural and remote located medical practitioner candidates; 21 rural doctors and academics developing the assessment items as part of a week-long writing workshop; investigating the feasibility of purchasing assessment items; and 48 rural candidates piloting three of the assessment items to ensure they would meet the guidelines for national accreditation. RESULTS: The project resulted in an innovative formative and summative assessment program that occurs throughout 4 years of vocational training, using innovative, reliable, valid and acceptable methods with educational impact. The piloting process occurred for 3 of the 6 assessment tools. Structured Assessment Using Multiple Patient Scenarios (StAMPS) is a new assessment method developed as part of this project. The StAMPS pilot found that it was reliable, with a generalisability coefficient of >0.76 and was a valid, acceptable and feasible assessment tool with desired educational impact. The multiple choice question (MCQ) examination pilot found that the applied clinical nature of the questions and their wide range of scenarios proved a very acceptable examination to the profession. The web based in-training assessment examination pilot revealed that it would serve well as a formative process until ACRRM can further develop their MCQ database. CONCLUSIONS: The ACRRM assessment program breaks new ground for assessing rural and remote doctors in Australia, and provides new evidence regarding how a comprehensive and contemporary assessment system can work within a postgraduate medical setting.

Selecting performance assessment methods for experienced physicians.

BACKGROUND: While much is now known about how to assess the competence of medical practitioners in a controlled environment, less is known about how to measure the performance in practice of experienced doctors working in their own environments. The performance of doctors depends increasingly on how well they function in teams and how well the health care system around them functions. METHODS: This paper reflects the combined experiences of a group of experienced education researchers and the results of literature searches on performance assessment methods. CONCLUSION: Measurement of competence is different to measurement of performance. Components of performance could be re-conceptualised within a different domain structure. Assessment methods may be of a different utility to that in competence assessment and, indeed, of different utility according to the purpose of the assessment. An exploration of the utility of potential performance assessment methods suggests significant gaps that indicate priority areas for research and development.

Is insight important? measuring capacity to change performance.

BACKGROUND: Some doctors who perform poorly appear not to be aware of how their performance compares with accepted practice. The way that professionals maintain their existing expertise and acquire new knowledge and skills - that is, maintain their 'currency' of practice - requires a capacity to change. This capacity to change probably requires the individual doctor to possess insight into his or her performance as well as motivation to change. There may be a range of levels of insight in different individuals. At some point this reaches a level which is inadequate for effective self-regulation. Insight and performance may be critically related and there are instances where increasing insight in the presence of decreasing performance can also cause difficulties. OBJECTIVE: This paper presents an exploration into the nature of insight, its relationship to professional performance and its measurement as part of performance, reflecting the combined experiences of a group of experienced education researchers and the results of literature searches on insight and performance. CONCLUSION: There may be individuals in whom insight is so lacking that they are beyond remediation. If there is a dichotomy between adequate and inadequate levels of insight, testing this could be a cost-effective way of determining where efforts for remediation should be focussed.

Setting performance standards for medical practice: a theoretical framework.

BACKGROUND: The assessment of performance in the real world of medical practice is now widely accepted as the goal of assessment at the postgraduate level. This is largely a validity issue, as it is recognised that tests of knowledge and in clinical simulations cannot on their own really measure how medical practitioners function in the broader health care system. However, the development of standards for performance-based assessment is not as well understood as in competency assessment, where simulations can more readily reflect narrower issues of knowledge and skills. This paper proposes a theoretical framework for the development of standards that reflect the more complex world in which experienced medical practitioners work. METHODS: The paper reflects the combined experiences of a group of education researchers and the results of literature searches that included identifying current health system data sources that might contribute information to the measurement of standards. CONCLUSION: Standards that reflect the complexity of medical practice may best be developed through an "expert systems" analysis of clinical conditions for which desired health care outcomes reflect the contribution of several health professionals within a complex, three-dimensional, contextual model. Examples of the model are provided, but further work is needed to test validity and measurability.

From science to application: the development of an intervention package.

Many community-based organizations and health departments want to implement HIV prevention interventions with scientifically demonstrated effectiveness. The Replicating Effective Programs (REP) project supported researchers in developing intervention packages designed to help prevention partners replicate effective programs in their settings. Intervention packages convey the intervention's foundation, components, and methods and are one part of a larger system needed to transfer research-based HIV prevention technology to service providers. Implementation packages were developed using a multistage process. The original researchers drafted the materials, advisory groups reviewed the packages, and adopting agencies used the materials in trial runs. The advisory groups and adopting agencies recommended extensive use of examples, thorough explanations about the rationale for each intervention component, explicit representation of people of color in the materials, clear statements about the intended audience(s), and an easy-to-use and visually appealing format. Packages were revised based on these recommendations and the outcomes of the trial runs.

Transfer of research-based HIV prevention interventions to community service providers: fidelity and adaptation.

HIV prevention research interventions usually follow protocols with specific procedures. If a community-delivered intervention uses the same procedures with the same populations as those in the original research, the behavior change effects should be similar. However, community-based providers may not replicate an intervention exactly as it was conducted in the effectiveness study. Adaptation may be needed to better meet the needs of the clients, community, or organization. We propose that interventions can be defined in terms of core elements likely to be responsible for effectiveness. These core elements cannot be changed without fundamentally changing the intervention, whereas other characteristics may be modified without altering effectiveness. HIV prevention researchers and service providers can collaborate to develop interventions that not only are effective but can also be successfully implemented by service organizations. If researchers actively involve service providers and community members in intervention planning, technology transfer goals can be better achieved.

Is Australian rural practice changing? Findings from the National Rural General Practice Study.

The National Rural General Practice Study (NRGPS) was the first comprehensive national study covering rural and remote general practitioners throughout Australia. It was undertaken in 1996-1997 and drew on data from existing sources such as the Australian Bureau of Statistics and the Australian Institute of Health and Welfare, together with a postal survey of general practitioners in rural and remote areas. There was a 75% response rate to the survey, which covered professional issues, personal and social issues, personal background, patient issues, recruitment and retention programs and changing health services. Overall, the study findings confirmed those of previous individual State-based studies in the early 1990s and showed that there had been some changes since those previous studies. In particular, access to continuing medical education has improved, the rural medical workforce appears to be ageing, the proportion of women rural doctors is increasing and the projected length of stay in rural practice is decreasing. Whereas in the early 1990s the projection for rural doctor numbers was continuing decline, the NRGPS projected overall numbers in rural practice as staying approximately the same over the next 5 years. In the light of these trends, the challenge is to implement targeted initiatives that improve the recruitment and retention of rural and remote general practitioners.

Mobilizing young gay and bisexual men for HIV prevention: a two-community study.

BACKGROUND: To develop, implement and evaluate a community-level HIV prevention program (the Mpowerment Project) for young gay men in two US communities. DESIGN: Using a multiple baseline design, a cohort of young gay men was recruited independently of the intervention in each community and surveyed twice (1 year apart) regarding sexual risk behavior and psychosexual variables. The intervention was then implemented sequentially in each community. The cohorts were resurveyed immediately post- and 1-year post-intervention. Since there were few differences between the two communities, data were pooled to increase statistical power. INTERVENTION: The program had four components: peer outreach, whereby young gay men encouraged other men to engage in safer sex; peer-led small groups; a publicity campaign; and a young men's center. RESULTS: Baseline rates of unprotected anal intercourse were stable. Following intervention, the proportion of men who engaged in unprotected anal intercourse decreased from 38.3 to 30.9% (-19.3% relative reduction), with a reduction from 19.2 to 13.6% (-29.2% relative reduction) with non-primary partners, and a reduction from 57.7 to 41.8% (-27.6% relative reduction) with boyfriends. Reductions were sustained 1 year later with non-primary partners, but mixed results were found regarding sex with boyfriends. CONCLUSIONS: Mobilizing young gay men to support each other about safer sex is an effective approach to HIV prevention, but programs must be sustained. To reach young gay men, HIV prevention activities must be embedded into the satisfaction of needs for social and community belonging.

Analysis by training status of performance in the certification examination for Australian family doctors.

OBJECTIVES: Following recent changes to eligibility requirements for recognition as a general practitioner in Australia, an increasing proportion of candidates select the non-training, or practice-eligible, route to the RACGP Fellowship Examination, without the benefit of formal general practice training. DESIGN: Pass rates for the training and practice-eligible cohorts from the 1996 and 1997 examinations were determined and compared. SETTING: Royal Australian College of General Practitioners (RACGP). SUBJECTS: General practitioner candidates for the RACGP. RESULTS: The analysis of the 1996 and 1997 examinations showed that the overall pass rate and individual examination segment scores of the practice-eligible route candidates were lower than those for candidates from the training route. CONCLUSIONS: The observed difference in performance is more probably due to different preparation for the examination than any systematic bias against particular groups of candidates. All candidates for the examination would benefit from exposure to the general practice curriculum and supervised experience in Australian general practice.