Evaluation of recruitment and selection for specialty training in public health: interim results of a prospective cohort study to measure the predictive validity of the selection process.

BACKGROUND: The recruitment process for public health specialty training includes an assessment centre (AC) with three components, Rust Advanced Numerical Reasoning Appraisal (RANRA), Watson-Glaser Critical Thinking Appraisal (WGCT) and a Situation Judgement Test (SJT), which determines invitation to a selection centre (SC). The scores are combined into a total recruitment (TR) score that determines the offers of appointment. METHODS: A prospective cohort study using anonymous record linkage to investigate the association between applicant's scores in the recruitment process and registrar's progress through training measured by results of Membership Faculty Public Health (MFPH) examinations and outcomes of the Annual Review of Competence Progression (ARCP). RESULTS: Higher scores in RANRA, WGCT, AC, SC and TR were all significantly associated with higher adjusted odds of passing Part A MFPH exam at the first attempt. Higher scores in AC, SC and TR were significantly associated with passing Part B exam at the first attempt. Higher scores in SJT, AC and SC were significantly associated with satisfactory ARCP outcomes. CONCLUSIONS: The current UK national recruitment and selection process for public health specialty training has good predictive validity. The individual components of the process are testing different skills and abilities and together they are providing additive value.

A new competency model for general practice: implications for selection, training, and careers.

BACKGROUND: Recent structural and policy changes in the UK health service have significantly changed the job responsibilities for the GP role. AIM: To replicate a previous job analysis study to examine the relevance of current competency domains and selection criteria for doctors entering training. DESIGN AND METHOD: A multisource, multimethod approach comprising three phases: (1) stakeholder consultation (n = 205) using interviews, focus groups and behavioural observation of practising GPs; (2) a validation questionnaire based on results from phase 1 (n = 1082); followed by (3) an expert panel (n = 6) to review and confirm the final competency domains. RESULTS: Eleven competency domains were identified, which extends previous research findings. A new domain was identified called Leading for Continuing Improvement. Results show that, Empathy and Perspective Taking, Communication Skills, Clinical Knowledge and Expertise, and Professional Integrity are currently rated the most important domains. Results indicate a significant increase in ratings of importance for each domain in the future (P<0.001), except for Communication Skills and Empathy and Perspective Taking, which consistently remain high. CONCLUSION: The breadth of competencies required for GPs has increased significantly. GPs are now required to resolve competing tensions to be effective in their role, such as maintaining a patient focus while overseeing commissioning, with a potential ethical conflict between these aspects. Selection criteria remain largely unchanged but with increased priority in some domains (for example, Effective Teamworking). However, there is an urgent need to review the training provision arrangements to reflect the greater breadth of competencies now required.

Evaluating cognitive ability, knowledge tests and situational judgement tests for postgraduate selection.

OBJECTIVES: This study aimed to evaluate the validity and utility of and candidate reactions towards cognitive ability tests, and current selection methods, including a clinical problem-solving test (CPST) and a situational judgement test (SJT), for postgraduate selection. METHODS; This was an exploratory, longitudinal study to evaluate the validities of two cognitive ability tests (measuring general intelligence) compared with current selection tests, including a CPST and an SJT, in predicting performance at a subsequent selection centre (SC). Candidate reactions were evaluated immediately after test administration to examine face validity. Data were collected from candidates applying for entry into training in UK general practice (GP) during the 2009 recruitment process. Participants were junior doctors (n = 260). The mean age of participants was 30.9 years and 53.1% were female. Outcome measures were participants' scores on three job simulation exercises at the SC. RESULTS: Findings indicate that all tests measure overlapping constructs. Both the CPST and SJT independently predicted more variance than the cognitive ability test measuring non-verbal mental ability. The other cognitive ability test (measuring verbal, numerical and diagrammatic reasoning) had a predictive value similar to that of the CPST and added significant incremental validity in predicting performance on job simulations in an SC. The best single predictor of performance at the SC was the SJT. Candidate reactions were more positive towards the CPST and SJT than the cognitive ability tests. CONCLUSIONS: In terms of operational validity and candidate acceptance, the combination of the current CPST and SJT proved to be the most effective administration of tests in predicting selection outcomes. In terms of construct validity, the SJT measures procedural knowledge in addition to aspects of declarative knowledge and fluid abilities and is the best single predictor of performance in the SC. Further research should consider the validity of the tests in this study in predicting subsequent performance in training.

Evaluation of GP appraiser development centres using emotional intelligence: can positive learning outcomes be transferred to practice?

This paper describes a longitudinal evaluation of six pilot medical appraiser development centres (ADCs) with GPs held between February and September 2009 in the Kent, Surrey and Sussex Deanery. The ADCs were developed using traditional development centre methods and incorporated the concept of emotional intelligence (El). Initial evaluation results have shown positive short-term outcomes relating to appraiser skills and self-confidence as well as transfer of learning. This paper extends this earlier evaluation by looking at appraiser and appraisee feedback approximately one year after the ADCs using a validated framework for training evaluation. We discuss the long-term affective, cognitive and behavioural learning outcomes and the impact participation in the ADCs has on the broader healthcare system, including effects upon patient care and safety. Limitations of the current project and opportunities for future research are discussed.

Using emotional intelligence to facilitate strengthened appraiser development.

This paper describes a new approach to developing strengthened medical appraiser skills prior to the introduction of medical revalidation. We describe how we extended previous skills-based models and competency checklists to produce a behavioural model of effective appraiser performance. Development centre (DC) methods were used to produce a one-day workshop to encourage appraisers to reflect on their current level of ability and to identify and address additional required skills through observation, practice and feedback. In describing the DC, we discuss the impact of using the concept of emotional intelligence (EI) to develop appraiser skills and improve self-awareness. This aimed to support appraisers to effectively influence appraisees' continual professional development (CPD) and review appraisee practice through audit, significant events and patient and colleague feedback, with the ultimate aim of improving patient care. Finally, we provide initial evaluation data for our DC approach.

"Trust me...": psychological and behavioral predictors of perceived physician empathy.

A sociocognitive model of distal and proximal predictors of empathic judgments was tested among 100 physicians. The authors hypothesized that physician perceived control would affect empathy ratings via physician communication style. Specifically, physicians with high perceived control would use more open communication and be rated as more empathic. Physicians with low perceived control would use a controlling communication style and be rated as less empathic. Physicians completed a medical attribution questionnaire prior to a structured patient consultation exercise, during which patients and assessors rated physician empathy. The exercise was audiotaped, transcribed, and content analyzed for verbal behaviors. Support was found for the hypotheses; however, patients, but not medical assessors, associated empathy with reassurance and provision of medical information.