Subject(s)
Disasters , Internship and Residency , Mentally Ill Persons , Patient Transfer/organization & administration , Psychiatry/education , Arkansas , Emergency Services, Psychiatric/organization & administration , Hospitals, Veterans/organization & administration , Humans , Patient Care Planning/organization & administration , Patient Care Team/organization & administrationABSTRACT
Residency educators are identifying approaches to measure resident competence. Portfolios are well suited since they require work already completed as part of patient care where competency must be demonstrated. This paper describes assessment of the reliability and validity of portfolios in a psychiatry residency program. This was a cross-sectional study across 4years of residency education. Using guidelines, 18 residents assembled portfolios containing five entries chosen from 13 skills. Trained raters scored the portfolios. Residents and faculty were interviewed about their perceptions. Generalizability results indicated five entries and two raters were sufficient for relative decisions. Six entries or a third rater would be sufficient for absolute decisions. Portfolio scores tended to improve with years of training and correlated with psychiatric knowledge but not clinical performance. Residents and faculty identified benefits to assembling a portfolio. Portfolios incorporate tasks embedded in the residency to provide evidence of resident competency. The results support that the score is reliable and valid.
Subject(s)
Clinical Competence , Educational Measurement/methods , Internship and Residency , Cross-Sectional Studies , Humans , Reproducibility of ResultsABSTRACT
OBJECTIVE: To determine that portfolios, useable by any program, can provide needed evidence of resident performance within the ACGME general competencies. METHODS: Eighteen residents constructed portfolios with selected entries from thirteen psychiatric skills. Two raters assessed whether entries reflected resident performance within the general competencies. They indicated no (0), some (1), or definite evidence (2) for each competency. Median scores were reported by competency and psychiatric skill. RESULTS: Eighty entries were rated: 100% of the psychiatric skills reflected some evidence for medical knowledge; 92% provided some evidence for patient care, communication and interpersonal skills, and professionalism; 77% provided some evidence for systems-based practice, and 31% provided some evidence for practice-based learning. Ten of the 13 skills reflected evidence for at least five of the competencies. CONCLUSIONS: Any combination of five entries reflects all competencies with the exception of practice-based learning. This deficit can be corrected with revision of portfolio guidelines.
Subject(s)
Internship and Residency , Professional Competence , Psychiatry/education , Adolescent , Communication , Education , Humans , Interpersonal Relations , Patient Care/standards , Physician-Patient Relations , Surveys and QuestionnairesABSTRACT
OBJECTIVE: Since Engel introduced the biopsychosocial model, it has been extensively examined. The authors expect psychiatrists to formulate cases using the biopsychosocial model. However, resident psychiatrists' ability to generate formulations using this model has received little attention. METHODS: The authors evaluated resident biopsychosocial formulations using biopsychosocial scores from trained, blinded raters across four institutions. Second, the authors determined if an intervention could improve biopsychosocial formulation. DESIGN: This study included non-experimental and pre-post components using resident portfolio scores to measure biopsychosocial. PARTICIPANTS/SETTING: Residents from four postgraduate years (PGY) in four different programs participated. In one institution, faculty made a concerted effort to improve biopsychosocial formulation. There were 33 entries in 2000-2001 and 46 entries in 2001-2002. RESULTS: Using the combined data from all institutions, no PGY level averaged a rating of 3.0 (competent) in either year. In the institution implementing an intervention, a significant improvement was noted. CONCLUSION: This pilot study indicates that we can improve resident competency in this area.
Subject(s)
Internship and Residency , Models, Biological , Models, Psychological , Patient Care Planning , Psychiatry/education , Psychology/education , Research Design , Clinical Competence , Curriculum , Documentation/methods , Faculty, Medical , Humans , Pilot Projects , Program Evaluation , Social EnvironmentABSTRACT
The authors describe the development process for a portfolio that programs may use for evaluating psychiatry residents. A committee carefully developed the portfolio requirements for 13 necessary skills in psychiatry. Psychiatrists were trained to rate portfolios. Review of the development and training process suggested that portfolios provided information both on individual residents and on the residency program itself. The process requires orientation of both residents and faculty, attention to time demands, and availability of examples and resources.
