In-course and career outcomes predicted by medical school selection procedures based on personal qualities.

This paper reports a 40-year follow-up of 57 graduates from the initial intake to an Australian medical school who were selected on the basis of either academic criteria alone or desirable personal qualities as assessed by non-cognitive tests and an interview (with a third small group satisfying both criteria). Both students and teaching staff have remained blind to the basis for selection until the present day. Analysis of their under- and post-graduate careers indicates that 'academic' entrants were more likely to complete an intercalated BMedSci degree and to become specialists, while 'personal qualities' entrants were more likely to graduate with honours, become GPs, and win higher degrees after graduation. However, gender more significantly predicted these outcomes, with female results similar to 'personal qualities' entrants and males' similar to 'academic.' The results are interpreted with reference to the aims and structure of the Newcastle medicine curriculum.

Why is it so hard to consider personal qualities when selecting medical students?

Having 'good' doctors is important to everybody. How to select medical students better has been discussed repeatedly for more than seventy years, implying that prevailing methods could be improved. A significant body of research exists about selection methods and their application in medicine. Yet most medical schools world-wide continue to use prior academic performance and cognitive ability as their major criteria for selection, with minor or no consideration of personal qualities and interpersonal skills (possibly assuming they will develop naturally during training and practice). We describe the main methods available for assessing personal qualities of applicants to medical school and have attempted to identify some reasons and systemic disincentives working against their adoption.

Item Response Theory Analysis of the Big Five Questionnaire for Children-Short Form (BFC-SF): A Self-Report Measure of Personality in Children Aged 11-12 Years.

Prior investigations indicate that the five core personality dimensions (the "Big Five") are measurable by middle childhood. The aim of this research was to examine the psychometric properties of a short-form self-report measure of the Big Five personality dimensions in children that would be suitable for administration online in large population-based studies. Twenty-five questionnaire items in English, derived from the 65-item Big Five Questionnaire for Children in Italian (Barbaranelli, Caprara, Rabasca, & Pastorelli, 2003), were completed online by 27,415 Australian children in Year 6 (mean age 11.92 years). An item response theory approach evaluated the psychometric properties and resolved a 20-item short-form questionnaire. Exploratory and confirmatory factor analyses supported the Big Five structure. Construct validity was demonstrated via correlations between Big Five scores and the Strengths and Difficulties Questionnaire subscales (Goodman, 2001). The 20 items provide a brief, reliable, and valid child self-report measure of the Big Five personality dimensions.

Reliability of personality and values tests: The effects of "high stakes" selection conditions, and of four years in medical school.

This study investigated if scores on tests of personal qualities are affected by whether they will determine selection decisions ("high stakes") or not; and whether they are stable for individuals and groups across a four-year medical course. Two tests, one assessing values and one assessing components of personality, were administered either at the same time as a medical university entrance exam (first cohort; N = 216), or after entry was confirmed (second cohort; N = 142). Both cohorts took the tests again after four years of medical school. Analysis of variance was used to compare group mean scores and interactions, and correlation coefficients to measure temporal reliability. The high stakes cohort initially presented themselves in a significantly more positive light on the personality test. After four years of medical school scores on both tests changed significantly, towards more communitarian values and less empathic attitudes. Thus, personality scores were affected by both the conditions under which the initial tests were conducted and by the passage of time, but values only by the passage of time. Before and after scores were significantly correlated.

The 2015 Middle Childhood Survey (MCS) of mental health and well-being at age 11 years in an Australian population cohort.

PURPOSE: The Middle Childhood Survey (MCS) was designed as a computerised self-report assessment of children's mental health and well-being at approximately 11 years of age, conducted with a population cohort of 87 026 children being studied longitudinally within the New South Wales (NSW) Child Development Study. PARTICIPANTS: School Principals provided written consent for teachers to administer the MCS in class to year 6 students at 829 NSW schools (35.0% of eligible schools). Parent or child opt-outs from participation were received for 4.3% of children, and MCS data obtained from 27 808 children (mean age 11.5 years, SD 0.5; 49.5% female), representing 85.9% of students at participating schools. FINDINGS TO DATE: Demographic characteristics of participating schools and children are representative of the NSW population. Children completed items measuring Social Integration, Prosocial Behaviour, Peer Relationship Problems, Supportive Relationships (at Home, School and in the Community), Empathy, Emotional Symptoms, Conduct Problems, Aggression, Attention, Inhibitory Control, Hyperactivity-Inattention, Total Difficulties (internalising and externalising psychopathology), Perceptual Sensitivity, Psychotic-Like Experiences, Personality, Self-esteem, Daytime Sleepiness and Connection to Nature. Distributions of responses on each item and construct demarcate competencies and vulnerabilities within the population: most children report mental health and well-being, but the population distribution spanned the full range of possible scores on every construct. FUTURE PLANS: Multiagency, intergenerational linkage of the MCS data with health, education, child protection, justice and early childhood development records took place late in 2016. Linked data were used to elucidate patterns of risk and protection across early and middle child development, and these data will provide a foundation for future record linkages in the cohort that will track mental and physical health, social and educational/occupational outcomes into adolescence and early adulthood.

Have we got the selection process right? The validity of selection tools for predicting academic performance in the first year of undergraduate medicine.

This article was migrated. The article was marked as recommended. Content: There remains much debate over the 'best' method for selecting students in to medicine. This study aimed to assess the predictive validity of four different selection tools with academic performance outcomes in first-year undergraduate medical students. Methods: Regression analyses were conducted between admission scores on previous academic performance - the Australian Tertiary Admission Rank (ATAR), the Undergraduate Medicine and Health Sciences Admission Test (UMAT), Multiple-Mini Interview (MMI) and the Personal Qualities Assessment (PQA) with student performance in first-year assessments of Multiple Choice Questions, Short Answer Questions, Objective Structured Clinical Examinations (OSCE) and Problem-Based Learning (PBL) Tutor ratings in four cohorts of students (N = 604, 90%). Results: All four selection tools were found to have significant predictive associations with one or more measures of student performance in Year One of undergraduate medicine. UMAT, ATAR and MMI scores consistently predicted first year performance on a number of outcomes. ATAR was the only selection tool to predict the likelihood of making satisfactory progress overall. Conclusions: All four selection tools play a contributing role in predicting academic performance in first year medical students. Further research is required to assess the validity of selection tools in predicting performance in the later years of medicine.

Potential predictors of psychological distress and well-being in medical students: a cross-sectional pilot study.

PURPOSE: Research has consistently found that the proportion of medical students who experience high levels of psychological distress is significantly greater than that found in the general population. The aim of our research was to assess the levels of psychological distress more extensively than has been done before, and to determine likely predictors of distress and well-being. SUBJECTS AND METHODS: In 2013, students from an Australian undergraduate medical school (n=127) completed a questionnaire that recorded general demographics, hours per week spent studying, in paid work, volunteer work, and physical exercise; past and current physical and mental health, social support, substance use, measures of psychological distress (Kessler Psychological Distress Scale, depression, anxiety, stress, burnout); and personality traits. RESULTS: Females were found to have higher levels of psychological distress than males. However, in regression analysis, the effect of sex was reduced to nonsignificance when other variables were included as predictors of psychological distress. The most consistent significant predictors of our 20 indicators of psychological distress were social support and the personality traits of emotional resilience and self-control. CONCLUSION: The findings suggest that emotional resilience skills training embedded into the medical school curriculum could reduce psychological distress among medical students.

New South Wales Child Development Study (NSW-CDS): an Australian multiagency, multigenerational, longitudinal record linkage study.

PURPOSE: The initial aim of this multiagency, multigenerational record linkage study is to identify childhood profiles of developmental vulnerability and resilience, and to identify the determinants of these profiles. The eventual aim is to identify risk and protective factors for later childhood-onset and adolescent-onset mental health problems, and other adverse social outcomes, using subsequent waves of record linkage. The research will assist in informing the development of public policy and intervention guidelines to help prevent or mitigate adverse long-term health and social outcomes. PARTICIPANTS: The study comprises a population cohort of 87,026 children in the Australian State of New South Wales (NSW). The cohort was defined by entry into the first year of full-time schooling in NSW in 2009, at which time class teachers completed the Australian Early Development Census (AEDC) on each child (with 99.7% coverage in NSW). The AEDC data have been linked to the children's birth, health, school and child protection records for the period from birth to school entry, and to the health and criminal records of their parents, as well as mortality databases. FINDINGS TO DATE: Descriptive data summarising sex, geographic and socioeconomic distributions, and linkage rates for the various administrative databases are presented. Child data are summarised, and the mental health and criminal records data of the children's parents are provided. FUTURE PLANS: In 2015, at age 11 years, a self-report mental health survey was administered to the cohort in collaboration with government, independent and Catholic primary school sectors. A second record linkage, spanning birth to age 11 years, will be undertaken to link this survey data with the aforementioned administrative databases. This will enable a further identification of putative risk and protective factors for adverse mental health and other outcomes in adolescence, which can then be tested in subsequent record linkages.

Predictors of professional behaviour and academic outcomes in a UK medical school: A longitudinal cohort study.

BACKGROUND: Over the past 70 years, there has been a recurring debate in the literature and in the popular press about how best to select medical students. This implies that we are still not getting it right: either some students are unsuited to medicine or the graduating doctors are considered unsatisfactory, or both. AIM: To determine whether particular variables at the point of selection might distinguish those more likely to become satisfactory professional doctors, by following a complete intake cohort of students throughout medical school and analysing all the data used for the students' selection, their performance on a range of other potential selection tests, academic and clinical assessments throughout their studies, and records of professional behaviour covering the entire five years of the course. METHODS: A longitudinal database captured the following anonymised information for every student (n = 146) admitted in 2007 to the Hull York Medical School (HYMS) in the UK: demographic data (age, sex, citizenship); performance in each component of the selection procedure; performance in some other possible selection instruments (cognitive and non-cognitive psychometric tests); professional behaviour in tutorials and in other clinical settings; academic performance, clinical and communication skills at summative assessments throughout; professional behaviour lapses monitored routinely as part of the fitness-to-practise procedures. Correlations were sought between predictor variables and criterion variables chosen to demonstrate the full range of course outcomes from failure to complete the course to graduation with honours, and to reveal clinical and professional strengths and weaknesses. RESULTS: Student demography was found to be an important predictor of outcomes, with females, younger students and British citizens performing better overall. The selection variable "HYMS academic score", based on prior academic performance, was a significant predictor of components of Year 4 written and Year 5 clinical examinations. Some cognitive subtest scores from the UK Clinical Aptitude Test (UKCAT) and the UKCAT total score were also significant predictors of the same components, and a unique predictor of the Year 5 written examination. A number of the non-cognitive tests were significant independent predictors of Years 4 and 5 clinical performance, and of lapses in professional behaviour. First- and second-year tutor ratings were significant predictors of all outcomes, both desirable and undesirable. Performance in Years 1 and 2 written exams did not predict performance in Year 4 but did generally predict Year 5 written and clinical performance. CONCLUSIONS: Measures of a range of relevant selection attributes and personal qualities can predict intermediate and end of course achievements in academic, clinical and professional behaviour domains. In this study HYMS academic score, some UKCAT subtest scores and the total UKCAT score, and some non-cognitive tests completed at the outset of studies, together predicted outcomes most comprehensively. Tutor evaluation of students early in the course also identified the more and less successful students in the three domains of academic, clinical and professional performance. These results may be helpful in informing the future development of selection tools.

Trial use of the Personal Qualities Assessment (PQA) in the entrance examination of a Japanese medical university: similarities to the results in western countries.

BACKGROUND: The Personal Qualities Assessment (PQA), developed by the University of Newcastle, Australia to assess the aptitude of future medical professionals, has been used in Western countries. PURPOSES: The objective was to investigate whether the PQA is appropriate for Japanese medical school applicants. METHODS: Two of the PQA tests, Libertarian-Dual-Communitarian moral orientations (Mojac) and Narcissism, Aloofness, Confidence, and Empathy (NACE), were translated into Japanese, and administered at the Tokyo Women's Medical University entrance examinations from 2007 to 2009. RESULTS: The distributions of the applicants' Mojac and NACE scores were close to the normal distribution, and the mean scores did not exhibit a large difference from those in Western countries. The only significant difference was that the mean score of the NACE test was slightly lower than the Western norm. CONCLUSIONS: The translated PQA tests may be appropriate for use with Japanese applicants, though further research considering cultural differences is required.

Comparing the cognitive, personality and moral characteristics of high school and graduate medical entrants to the Tehran University of Medical Sciences in Iran.

BACKGROUND: Tehran University of Medical Sciences has two streams of medical student admission: an established high school entry (HSE) route and an experimental graduate entry (GE) route. AIM: To compare the cognitive skills, personality traits and moral characteristics of HSE and GE students admitted to this university. METHODS: The personal qualities assessment tool (PQA; www.pqa.net.au ) was translated from English to Persian and then back-translated. Afterwards 35 individuals from the GE and 109 individuals from the 2007 to 2008 HSE completed the test. The results were compared by t-test and Chi-square. RESULTS: The HSE students showed significantly higher ability in the cognitive skills tests (p < 0.001). They were also more libertarian (p = 0.022), but had lower ability to confront stress and unpleasant events (p < 0.001), and had lower self-awareness and self-control (p < 0.001). CONCLUSION: On the basis of their personal qualities, the GE students had more self-control and strength when coping with stress than the HSE students, but the latter had superior cognitive abilities. Hence it may be useful to include cognitive tests in GE students' entry exam and include tests of personal qualities to exclude those with unsuitable characteristics.

Using personal qualities assessment to measure the moral orientation and personal qualities of medical students in a non-Western culture.

How to select candidates with appropriate personal qualities for medical school is an important issue. This study examined the psychometric properties and group differences of the Personal Qualities Assessment (PQA) to test the feasibility of using it as a tool to assess the medical school applicants in a non-Western culture. Seven hundred forty-six medical students in Taiwan completed two psychometric measures: Mojac to assess moral orientation and NACE to assess four aspects of interpersonal relationships. Thirty-one students completed the tests twice to establish test-retest reliability. A subsample of 127 students also completed a measure of the "Big Five" personality traits to examine the construct validity of these scales. Both Mojac and NACE had acceptable internal consistency and test-retest reliability. Conceptually, coherent and significant relationships were observed between test components and between the NACE and Big Five. NACE but not Mojac varied significantly between different sociodemographic groups. Both tests demonstrated acceptable psychometric properties. However, the predictive validity of PQA requires future studies.

Can personal qualities of medical students predict in-course examination success and professional behaviour? An exploratory prospective cohort study.

BACKGROUND: Over two-thirds of UK medical schools are augmenting their selection procedures for medical students by using the United Kingdom Clinical Aptitude Test (UKCAT), which employs tests of cognitive and non-cognitive personal qualities, but clear evidence of the tests' predictive validity is lacking. This study explores whether academic performance and professional behaviours that are important in a health professional context can be predicted by these measures, when taken before or very early in the medical course. METHODS: This prospective cohort study follows the progress of the entire student cohort who entered Hull York Medical School in September 2007, having taken the UKCAT cognitive tests in 2006 and the non-cognitive tests a year later. This paper reports on the students' first and second academic years of study. The main outcome measures were regular, repeated tutor assessment of individual students' interpersonal skills and professional behaviour, and annual examination performance in the three domains of recall and application of knowledge, evaluation of data, and communication and practical clinical skills. The relationships between non-cognitive test scores, cognitive test scores, tutor assessments and examination results were explored using the Pearson product-moment correlations for each group of data; the data for students obtaining the top and bottom 20% of the summative examination results were compared using Analysis of Variance. RESULTS: Personal qualities measured by non-cognitive tests showed a number of statistically significant relationships with ratings of behaviour made by tutors, with performance in each year's objective structured clinical examinations (OSCEs), and with themed written summative examination marks in each year. Cognitive ability scores were also significantly related to each year's examination results, but seldom to professional behaviours. The top 20% of examination achievers could be differentiated from the bottom 20% on both non-cognitive and cognitive measures. CONCLUSIONS: This study shows numerous significant relationships between both cognitive and non-cognitive test scores, academic examination scores and indicators of professional behaviours in medical students. This suggests that measurement of non-cognitive personal qualities in applicants to medical school could make a useful contribution to selection and admission decisions. Further research is required in larger representative groups, and with more refined predictor measures and behavioural assessment methods, to establish beyond doubt the incremental validity of such measures over conventional cognitive assessments.

Using personality as a predictor of diet induced weight loss and weight management.

BACKGROUND: A major challenge for successful weight management is tailoring weight loss programs to individual needs. The aim of this study was to investigate whether personality traits could be used to match individuals to a compatible weight loss program that would maximize weight loss. METHOD: Two different weight loss trials were conducted, both with a weight loss greater than 5% the measure of success. Fifty-four individuals, BMI 30-40 kg/m2, either followed a slow, healthy eating weight loss diet (HEWLD) of 5000-6000 kJ/day for 12 weeks (n = 22), or a fast, very low energy diet (VLED) of 3000 kJ/day for 4 weeks (n = 32). Anthropometric measurements were recorded at baseline, at the end of the weight loss period and, for VLED, at the end of 10 weeks of weight maintenance. Personality traits were measured at baseline using the Tangney Self Control Scale plus 3 of the scales from the Five Factor Model - Neuroticism, Conscientiousness and Extraversion. RESULTS: The percentage weight loss was significantly greater in VLED (-7.38%) compared to HEWLD (-4.11%), (p < 0.001). Weight loss in HEWLD was positively correlated with Anxiety, a facet of Neuroticism. Weight loss in VLED was positively correlated with Neuroticism (r = 0.5, p < 0.01), and negatively correlated with Dutifulness and Discipline, facets of Conscientiousness, (p < 0.05 for both). No link was observed between weight loss and the personality trait, Self Control, in either HEWLD or VLED. CONCLUSION: The personality factor, Neuroticism, was linked to successful weight loss (that is ≥ 5%) with a particular weight loss treatment, suggesting that there is a potential to use measures of personality to identify appropriate weight loss/management strategies for individuals. TRIAL REGISTRATION: Australia and New Zealand Clinical Trials Register (ANZCTR): ACTRN12611000716965.

Predictive validity of the personal qualities assessment for selection of medical students in Scotland.

BACKGROUND: The Personal Qualities Assessment (PQA) was developed to enhance medical student selection by measuring a range of non-cognitive attributes in the applicants to medical school. Applicants to the five Scottish medical schools were invited to pilot the test in 2001 and 2002. AIMS: To evaluate the predictive validity of PQA for selecting medical students. METHODS: A longitudinal cohort study was conducted in which PQA scores were compared with senior year medical school performance. RESULTS: Consent to access performance markers was obtained from 626 students (61.6% of 1017 entrants in 2002-2003). Linkable Foundation Year (4th) rankings were available for 411 (66%) students and objective structured clinical examination (OSCE) rankings for 335 (54%) of those consenting. Both samples were representative of the original cohort. No significant correlations were detected between separate elements of the PQA assessment and student performance. However, using the algorithm advocated by Powis et al. those defined as 'non-extreme' (<±1.5 SD from the cohort mean scores; SD, standard deviation) character types on the involved-detached and on the libertarian-communitarian moral orientation scales were ranked higher in OSCEs (average of 7.5% or 25 out of 335, p = 0.049). CONCLUSIONS: This study was limited by high attrition and basic outcome markers which are insensitive to relevant non-cognitive characteristics. However, it is the largest currently available study of predictive validity for the PQA assessment. There was one finding of significance: that those students who were identified by PQA as 'not extreme' on the two personal characteristics scales performed better in an OSCE measure of professionalism. Futures studies are required since psychometric testing for both cognitive and non-cognitive attributes are increasingly used in admission process and these should include more and better measures of professionalism against which to correlate non-cognitive traits.

A comprehensive model for the selection of medical students.

BACKGROUND: Medical schools have a need to select their students from an excess of applicants. Selection procedures have evolved piecemeal: Academic thresholds have risen, written tests have been incorporated and interview protocols are developed. AIM: To develop and offer for critical review and, ultimately, present for adoption by medical schools, an evidence-based and defensible model for medical student selection. METHODS: We have described here a comprehensive model for selecting medical students which is grounded on the theoretical and empirical selection and assessment literature, and has been shaped by our own research and experience. RESULTS: The model includes the following selection criteria: Informed self-selection, academic achievement, general cognitive ability (GCA) and aspects of personality and interpersonal skills. A psychometrically robust procedure by which cognitive and non-cognitive test scores can be used to make selection decisions is described. Using de-identified data (n = 1000) from actual selection procedures, we demonstrate how the model and the procedure can be used in practice. CONCLUSION: The model presented is based on a currently best-practice approach and uses measures and methods that maximise the probability of making accurate, fair and defensible selection decisions.

Graduate entry to medicine: widening psychological diversity.

BACKGROUND: At Nottingham University more than 95% of entrants to the traditional 5-year medical course are school leavers. Since 2003 we have admitted graduate entrants (GEM) to a shortened (4-year) course to 'widen access to students from more disadvantaged backgrounds'. We have recently shown that the GEM course widens academic and socio-demographic diversity of the medical student population. This study explored whether GEM students also bring psychological diversity and whether this could be beneficial. METHODS: We studied: a) 217 and 96 applicants to the Nottingham 5- and 4-year courses respectively, applying in the 2002-3 UCAS cycle, and, b) 246 school leavers starting the 5-year course and 39 graduate entrants to the 4-year course in October 2003. The psychological profiles of the two groups of applicants and two groups of entrants were compared using their performance in the Goldberg 'Big 5' Personality test, the Personal Qualities Assessment (PQA; measuring interpersonal traits and interpersonal values), and the Lovibond and Lovibond measure of depression, anxiety and stress. For the comparison of the Entrants we excluded the 33 school leavers and seven graduates who took the tests as Applicants.Statistical analyses were undertaken using SPSS software (version 16.0). RESULTS: Graduate applicants compared to school leaver applicants were significantly more conscientious, more confident, more self controlled, more communitarian in moral orientation and less anxious. Only one of these differences was preserved in the entrants with graduates being less anxious. However, the graduate entrants were significantly less empathetic and conscientious than the school leavers. CONCLUSION: This study has shown that school leaver and graduate entrants to medical school differ in some psychological characteristics. However, if confirmed in other studies and if they were manifest in the extreme, not all the traits brought by graduates would be desirable for someone aiming for a medical career.