Translation, cultural adaptation and linguistic validation of the postgraduate hospital educational environment measure into Arabic.

BACKGROUND: Assessment of the clinical learning environment (CLE) is an essential step that teaching hospitals routinely undertake to ensure the environment is conducive, learning-oriented and supportive of junior doctors' education. The Postgraduate Hospital Educational Environment Measure (PHEEM) is an internationally recognized tool for assessing the CLE with evidence of high reliability and validity. Translation of PHEEM into other languages such as Spanish, Japanese and Persian enabled wider adoption of the instrument in the world. However, in Syria and other Arabic countries, a validated Arabic translation of PHEEM is still not available, making it difficult to adopt it and use it in Arabic contexts. This study aims to translate and culturally adapt the PHEEM from English into Arabic. METHODS: This study followed the structured translation and validation process guideline proposed by Sousa & Rojjanasrirat 2010. First, the PHEEM went through forward translation by three translators, then reconciled with the aid of a fourth translator. Afterwards, two professional bicultural and bilingual translators conducted back translation into English and compared it with the original version. This formed the Pre-final Version (PFV) which was then pretested for clarity on a sample of medical residents in Damascus, Syria. Following appropriate modifications, the PFV was sent to a panel of experts for a comprehensive review of language clarity and to assess content validity. RESULTS: A total of thirty-five medical residents were recruited. Ten items with language clarity issues were identified and modified according to the elicited suggestions. Thereafter, the modified PFV was presented to ten subject experts who identified three items in need of revision. The item-content Validity Index (CVI) was over 0.78 for all of the 40 items; the calculated scale-CVI was 0.945. DISCUSSION: This study provided the first linguistically valid Arabic translation of the widely used PHEEM inventory. The next step is to conduct a full psychometric analysis of the Arabic PHEEM to provide further evidence of validity and reliability.

Acceptability and feasibility of an evaluation table to assess the competency of general medicine interns during ambulatory rotations in Brest.

BACKGROUND: General practitioner interns need to acquire the expected clinical, communication, personal and professional competencies. Internship evaluations use qualitative evaluation tables to assess competency acquisition. However, there is no standardised evaluation table used in France. Some faculties use the exhaustive, precise, and manageable Exceler evaluation tool. We aim to evaluate opinions of General practice interns in Brest about the acceptability and feasibility of using the Exceler evaluation tool to monitor competency acquisition during internships. METHODS: This qualitative study used intern focus groups. Six-open ended questions with optional follow-up questions were asked. Cards from the Dixit® game were used to guide and facilitate discussion. Open, axial, then integrative analysis of the verbatim was performed. RESULTS: This is the first study to evaluate intern opinions about GP internship evaluations using focus groups. Participants felt that the quality of existing evaluations was insufficient, and it was difficult to monitor their progress. Adapting evaluations to individual profiles and backgrounds seemed necessary. Exceler appeared to be a possible solution due to its content validity, flexibility of use and accessibility. However, there were comments about possible modifications. CONCLUSIONS: Analysing opinions of tutors, supervisors and other practice centers could help identify potential barriers and reveal solutions to facilitate its implementation and use. TRIAL REGISTRATION: Not applicable.

Resident-led improvement project to screen for primary hyperaldosteronism in patients with resistant hypertension in an outpatient clinic.

Clinical practice guidelines recommend screening for primary hyperaldosteronism (PH) in patients with resistant hypertension. However, screening rates are low in the outpatient setting. We sought to increase screening rates for PH in patients with resistant hypertension in our Veterans Affairs (VA) outpatient resident physician clinic, with the goal of improving blood pressure control. Patients with possible resistant hypertension were identified through a VA Primary Care Almanac Metric query, with subsequent chart review for resistant hypertension criteria. Three sequential patient-directed cycles were implemented using rapid cycle improvement methodology during a weekly dedicated resident quality improvement half-day. In the first cycle, patients with resistant hypertension had preclinic PH screening labs ordered and were scheduled in the clinic for hypertension follow-up. In the second cycle, patients without screening labs completed were called to confirm medication adherence and counselled to screen for PH. In the third cycle, patients with positive screening labs were called to discuss mineralocorticoid receptor antagonist (MRA) initiation and possible endocrinology referral. Of 97 patients initially identified, 58 (60%) were found to have resistant hypertension while 39 had pseudoresistant hypertension from medication non-adherence. Of the 58 with resistant hypertension, 44 had not previously been screened for PH while 14 (24%) had already been screened or were already taking an MRA. Our screening rate for PH in resistant hypertension patients increased from 24% at the start of the project to 84% (37/44) after two cycles. Of the 37 tested, 24% (9/37) screened positive for PH, and 5 patients were started on MRAs. This resident-led quality improvement project demonstrated that a focused intervention process can improve PH identification and treatment.

Using Natural Language Processing to Evaluate the Quality of Supervisor Narrative Comments in Competency-Based Medical Education.

PURPOSE: Learner development and promotion rely heavily on narrative assessment comments, but narrative assessment quality is rarely evaluated in medical education. Educators have developed tools such as the Quality of Assessment for Learning (QuAL) tool to evaluate the quality of narrative assessment comments; however, scoring the comments generated in medical education assessment programs is time intensive. The authors developed a natural language processing (NLP) model for applying the QuAL score to narrative supervisor comments. METHOD: Samples of 2,500 Entrustable Professional Activities assessments were randomly extracted and deidentified from the McMaster (1,250 comments) and Saskatchewan (1,250 comments) emergency medicine (EM) residency training programs during the 2019-2020 academic year. Comments were rated using the QuAL score by 25 EM faculty members and 25 EM residents. The results were used to develop and test an NLP model to predict the overall QuAL score and QuAL subscores. RESULTS: All 50 raters completed the rating exercise. Approximately 50% of the comments had perfect agreement on the QuAL score, with the remaining resolved by the study authors. Creating a meaningful suggestion for improvement was the key differentiator between high- and moderate-quality feedback. The overall QuAL model predicted the exact human-rated score or 1 point above or below it in 87% of instances. Overall model performance was excellent, especially regarding the subtasks on suggestions for improvement and the link between resident performance and improvement suggestions, which achieved 85% and 82% balanced accuracies, respectively. CONCLUSIONS: This model could save considerable time for programs that want to rate the quality of supervisor comments, with the potential to automatically score a large volume of comments. This model could be used to provide faculty with real-time feedback or as a tool to quantify and track the quality of assessment comments at faculty, rotation, program, or institution levels.

The efficacy of ultrasonography on learning curve and inter-subject performance variability in radial artery cannulation in standardized training for residents: a randomized controlled trial.

AIM: To evaluate the effect of ultrasound (US) on learning curve and inter-subject performance variability of residents in radial artery cannulation. MATERIAL AND METHODS: Twenty non-anesthesiology residents who received standardized training in an anesthesiology department were selected and divided into two groups: anatomy group or US group. After training of relevant anatomy, US recognition and puncture skill, residents selected 10 patients either under US or anatomical localization performing radial artery catheterization. The number and time of successful cases of catheterization were recorded, success rate of first attempt and catheterization, as well as the total success rate of catheterization were calculated. The learning curve and inter-subject performance variability of residents were also calculated. Complications and the residents' satisfaction for teaching and self-confidence before puncture were also recorded. RESULTS: Compared to the anatomy group, total success rate and the success rate at first attempt were higher in US-guided group (88% vs. 57%, 94% vs. 81%). The average performance time in the US group was significantly less (2.9±0.8 min vs. 4.2±2.1 min) and the mean number of attempts was 1.6, while 2.6 for the anatomy group. With performing cases increasing, the average puncture time of residents in the US group decreased by 19s, while 14s in the anatomy group. More local hematoma occurred in the anatomy group. The satisfaction and confidence degree of residents were higher in US group ([98.5±6.5] vs [68.5±7.3], [90.2±8.6] vs [56.3±5.5]). CONCLUSION: US can significantly shorten the learning curve, reduce the inter-subject performance variability, improve the first attempt and total success rate of radial artery catheterization for non-anesthesiology residents.

Competency-based postgraduate training: Mentoring and monitoring using entrustable professional activities with descriptive rubrics for objectivity- A step beyond Dreyfus.

Background At Sri Balaji Vidyapeeth, a competency-based learning and training (CoBaLT©) model for postgraduate (PG) curriculum, within the regulatory norms, was implemented in 2016 after adequate faculty development programmes. This focused on well-defined outcomes. Methods A review of the outcomes was undertaken in 2018 as part of internal quality assurance receiving feedback from all stakeholders, viz. students, alumni and faculty. Recent publications were also reviewed. A major problem identified was lack of clarity in definition of levels leading to subjectivity in assessment while grading. It was also noted that the process needed to be aligned with the programme outcomes. Further refinements were, therefore, made to align and objectivize formative assessment using entrustable professional activities (EPA) with the aid of descriptive rubrics of sub-competencies and milestones. Addition of detailed rubrics for objectivity takes assessment a step beyond the Dreyfus model, aligning overall to the programme outcomes. Results Achievement of appropriate grades in EPAs by individual candidates ensures entrustability in professional activities by the time of completion of the PG programme. The modification was found more transparent and objective with reference to grading by the teachers and more conducive to reflections by the residents on their performance and how to improve it. Conclusions The use of descriptive rubrics along with EPAs brings transparency and plays a key role as an objective assessment tool, which can lend direction to individual resident learning and entrustability. This is an important component of outcome-based education.

Impact of Rater Training on Residents Technical Skill Assessments: A Randomized Trial.

OBJECTIVE: The ACS/APDS Resident Skills Curriculum's Objective Structured Assessment of Technical Skills (OSATS) consists of task-specific checklists and a global rating scale (GRS) completed by raters. Prior work demonstrated a need for rater training. This study evaluates the impact of a rater-training curriculum on scoring discrimination, consistency, and validity for handsewn bowel anastomosis (HBA) and vascular anastomosis (VA). DESIGN/ METHODS: A rater training video model was developed, which included a GRS orientation and anchoring performances representing the range of potential scores. Faculty raters were randomized to rater training or no rater training and were asked to score videos of resident HBA/VA. Consensus scores were assigned to each video using a modified Delphi process (Gold Score). Trained and untrained scores were analyzed for discrimination and score spread and compared to the Gold Score for relative agreement. RESULTS: Eight general and eight vascular surgery faculty were randomized to score 24 HBA/VA videos. Rater training increased rater discrimination and decreased rating scale shrinkage for both VA (mean trained score: 2.83, variance 1.88; mean untrained score: 3.1, variance 1.14, p = 0.007) and HBA (mean trained score: 3.52, variance 1.44; mean untrained score: 3.42, variance 0.96, p = 0.033). On validity analyses, a comparison between each rater group vs Gold Score revealed a moderate training impact for VA, trained κ=0.65 vs untrained κ=0.57 and no impact for HBA, R1 κ = 0.71 vs R2 κ = 0.73. CONCLUSION: A rater-training curriculum improved raters' ability to differentiate performance levels and use a wider range of the scoring scale. However, despite rater training, there was persistent disagreement between faculty GRS scores with no groups reaching the agreement threshold for formative assessment. If technical skill exams are incorporated into high stakes assessments, consensus ratings via a standard setting process are likely a more valid option than individual faculty ratings.

Improving Resident Knowledge of Ergonomics Guidelines.

OBJECTIVE: To improve resident knowledge of ergonomics guidelines for surgery, vaginal deliveries and repairs, and documentation. PROJECT SUMMARY: We consulted with a licensed occupational therapist at our institution regarding our difficulties with maintaining proper ergonomics while operating, performing deliveries, and documenting. We conducted two separate sessions: one in the obstetrics workroom regarding techniques to improve the ergonomics of our documentation and one in the operating room and labor and delivery unit to address techniques to avoid injury and promote long-term wellness. The sessions were conducted during morning report at our institution, and the obstetrics and gynecology teams attended both sessions. The sessions were available by videoconference for the entire residency program. Handouts were created to present basic ergonomics guidelines and were provided at the conclusion of the sessions. The handouts summarized the information provided during the sessions and included recommendations for adjustments that could be made in the workroom, operating room, and delivery room. The sessions were conducted during ACOG Wellness Week. OUTCOME: Adjustments were made in the obstetric and gynecology team workrooms to improve ergonomic function. Sessions improved resident knowledge of ergonomics recommendations for vaginal deliveries. Recommendations included guidelines for arm and shoulder position, tucked chins, and appropriate bed height (sitting vs standing). Adjustments were made in the workroom to position the top of the monitor just below eye level and arm's length away, and chairs were adjusted so that the keyboard and mouse height were just below elbow height. Residents were encouraged to keep arms and wrists in a relaxed, neutral position and to sit all the way back in the chair with back supported and feet firmly on the floor. Residents improved their knowledge of ergonomics guidelines and increased awareness of posture and positioning both on the labor and delivery unit and in the operating room. RELEVANCE TO WOMENS HEALTH OR PHYSICIANS IN PRACTICE: Work-related musculoskeletal disorders are prevalent among surgeons and can have a significant effect on productivity and career longevity. If we are able to implement evidence-based guidelines developed by high-quality ergonomics research, we can potentially protect obstetricians and gynecologists from injury and improve overall wellness.

[Feedback on the first three years of application of the reform of the third cycle in oncology]. / La phase socle en oncologie : retour d'expérience sur les trois premières années d'application de la réforme du troisième cycle.

Since the establishment of the reform of medical studies' third cycle in 2017, the first two residency semesters define the "phase socle" whose objective is to provide the basic knowledge of the specialty. We have carried out a declarative survey, submitted in 2020 to all French residents in Oncology whose "phase socle" had taken place during the first 3 years of the reform. The main objectives of this survey were to evaluate the theoretical teaching of oncology as well as the practical hospital training provided during this phase. The response rate was 44% (among 355 residents, 155 answered). In terms of theoretical training, the level of satisfaction with the national teaching courses of the Collège National des Enseignants en Cancérologie and the distant learning courses on the SIDES-NG platform was considered satisfactory (average visual analog scale of 6.7/10 and 5.7/10, respectively). There was greater heterogeneity in the organization of local courses, of which only 50% of base phase residents benefited. In terms of practical training, the training value of the medical oncology and radiation oncology residencies was good (visual analogue scale 7.9/10 and 6.7/10, respectively), with educational objectives adapted to the base phase, but with a greater workload for medical oncology. This study provides feedback that shows the success of this reform in oncology. It also offers suggestions, which could be the basis to improve the formation of oncology residents.

Impact of Bedside Rounding on Attending Teaching Evaluations.

OBJECTIVE: To examine associations between bedside rounding (BSR) and other rounding strategies (ORS) with resident evaluations of teaching attendings and self-reported attending characteristics. METHODS: Faculty from three academic medical centers who attended resident teaching services for ≥4 weeks during the 2018-2019 academic year were invited to complete a survey about personal and rounding characteristics. The survey instrument was iteratively developed to assess rounding strategy as well as factors that could affect choosing one rounding strategy over another. Survey results and teaching evaluation scores were linked, then deidentified and analyzed in aggregate. Included evaluation items assessed resident perceptions of autonomy, time management, professionalism, and teaching effectiveness, as well as a composite score (the numeric average of each attending's scores for all of the items at his or her institution). BSR was defined as spending >50% of rounding time in patients' rooms with the team. Hallway rounding and conference room rounding were combined into the ORS category and defined as >50% of rounding time in these settings. All of the scores were normalized to a 10-point scale to allow aggregation across sites. RESULTS: A total of 105 attendings were invited to participate, and 65 (62%) completed the survey. None of the resident evaluation scores significantly differed based on rounding strategy. Composite scores were similar for BSR and ORS (difference of <0.1 on a 10-point scale). Spearman correlation coefficients identified no statistically significant correlation between rounding strategy and evaluation scores. An exploratory analysis of variance model identified no single factor that was significantly associated with composite teaching scores (P > 0.45 for all) or the domains of teaching efficacy, professionalism, or autonomy (P > 0.13 for all). Having a formal educational role was significantly associated with better evaluation scores for time management, and the number of lectures delivered per year approached statistical significance for the same domain. CONCLUSIONS: Conducting BSR did not significantly affect resident evaluations of teaching attendings. Resident perception of teaching effectiveness based on rounding strategy should be neither a motivator nor a barrier to widespread institution of BSR.