Experience of caring for patients with COVID-19 and educational achievement among Japanese resident physicians: a nationwide survey with general medicine in-training examination.

BACKGROUND: As healthcare professional trainees, resident physicians are expected to help with COVID-19 care in various ways. Many resident physicians worldwide have cared for COVID-19 patients despite the increased risk of burnout. However, few studies have examined the experience with COVID-19 care among resident physicians and its effects on competency achievement regarding clinical basics and COVID-19 patient care. METHOD: This nationwide, cross-sectional Japanese study used a clinical training environment questionnaire for resident physicians (PGY-1 and - 2) in 593 teaching hospitals during the General Medicine In-Training Examination in January 2021. The General Medicine In-Training Examination questions comprised four categories (medical interviews and professionalism; symptomatology and clinical reasoning; physical examination and clinical procedures; and disease knowledge) and a COVID-19-related question. We examined the COVID-19 care experience and its relationship with the General Medicine In-Training Examination score, adjusting for resident and hospital variables. RESULTS: Of the 6,049 resident physicians, 2,841 (47.0%) had no experience caring for patients with COVID-19 during 2020. Total and categorical General Medicine In-Training Examination scores were not different irrespective of the experience with COVID-19 patient care. For the COVID-19-related question, residents with experience in COVID-19 care showed a significant increase in correct response by 2.6% (95% confidence interval, 0.3-4.9%; p = 0.028). CONCLUSIONS: The resident physicians' COVID-19 care experience was associated with better achievement of COVID-19-related competency without reducing clinical basics. However, approximately half of the residents missed the critical experience of caring for patients during this unparalleled pandemic in Japan.

Hospital Use of a Web-Based Clinical Knowledge Support System and In-Training Examination Performance Among Postgraduate Resident Physicians in Japan: Nationwide Observational Study.

Background: The relationship between educational outcomes and the use of web-based clinical knowledge support systems in teaching hospitals remains unknown in Japan. A previous study on this topic could have been affected by recall bias because of the use of a self-reported questionnaire. Objective: We aimed to explore the relationship between the use of the Wolters Kluwer UpToDate clinical knowledge support system in teaching hospitals and residents' General Medicine In-Training Examination (GM-ITE) scores. In this study, we objectively evaluated the relationship between the total number of UpToDate hospital use logs and the GM-ITE scores. Methods: This nationwide cross-sectional study included postgraduate year-1 and -2 residents who had taken the examination in the 2020 academic year. Hospital-level information was obtained from published web pages, and UpToDate hospital use logs were provided by Wolters Kluwer. We evaluated the relationship between the total number of UpToDate hospital use logs and residents' GM-ITE scores. We analyzed 215 teaching hospitals with at least 5 GM-ITE examinees and hospital use logs from 2017 to 2019. Results: The study population consisted of 3013 residents from 215 teaching hospitals with at least 5 GM-ITE examinees and web-based resource use log data from 2017 to 2019. High-use hospital residents had significantly higher GM-ITE scores than low-use hospital residents (mean 26.9, SD 2.0 vs mean 26.2, SD 2.3; P=.009; Cohen d=0.35, 95% CI 0.08-0.62). The GM-ITE scores were significantly correlated with the total number of hospital use logs (Pearson r=0.28; P<.001). The multilevel analysis revealed a positive association between the total number of logs divided by the number of hospital physicians and the GM-ITE scores (estimated coefficient=0.36, 95% CI 0.14-0.59; P=.001). Conclusions: The findings suggest that the development of residents' clinical reasoning abilities through UpToDate is associated with high GM-ITE scores. Thus, higher use of UpToDate may lead physicians and residents in high-use hospitals to increase the implementation of evidence-based medicine, leading to high educational outcomes.

Development of a Clinical Simulation Video to Evaluate Multiple Domains of Clinical Competence: Cross-Sectional Study.

BACKGROUND: Medical students in Japan undergo a 2-year postgraduate residency program to acquire clinical knowledge and general medical skills. The General Medicine In-Training Examination (GM-ITE) assesses postgraduate residents' clinical knowledge. A clinical simulation video (CSV) may assess learners' interpersonal abilities. OBJECTIVE: This study aimed to evaluate the relationship between GM-ITE scores and resident physicians' diagnostic skills by having them watch a CSV and to explore resident physicians' perceptions of the CSV's realism, educational value, and impact on their motivation to learn. METHODS: The participants included 56 postgraduate medical residents who took the GM-ITE between January 21 and January 28, 2021; watched the CSV; and then provided a diagnosis. The CSV and GM-ITE scores were compared, and the validity of the simulations was examined using discrimination indices, wherein ≥0.20 indicated high discriminatory power and >0.40 indicated a very good measure of the subject's qualifications. Additionally, we administered an anonymous questionnaire to ascertain participants' views on the realism and educational value of the CSV and its impact on their motivation to learn. RESULTS: Of the 56 participants, 6 (11%) provided the correct diagnosis, and all were from the second postgraduate year. All domains indicated high discriminatory power. The (anonymous) follow-up responses indicated that the CSV format was more suitable than the conventional GM-ITE for assessing clinical competence. The anonymous survey revealed that 12 (52%) participants found the CSV format more suitable than the GM-ITE for assessing clinical competence, 18 (78%) affirmed the realism of the video simulation, and 17 (74%) indicated that the experience increased their motivation to learn. CONCLUSIONS: The findings indicated that CSV modules simulating real-world clinical examinations were successful in assessing examinees' clinical competence across multiple domains. The study demonstrated that the CSV not only augmented the assessment of diagnostic skills but also positively impacted learners' motivation, suggesting a multifaceted role for simulation in medical education.

Association between regional quota program in medical schools and practical clinical competency based on General Medicine In-Training Examination score: a nationwide cross-sectional study of resident physicians in Japan.

PURPOSE: A regional quota program (RQP) was introduced in Japan to ameliorate the urban-rural imbalance of physicians. Despite concerns about the low learning abilities of RQP graduates, the relationship between the RQP and practical clinical competency after initiating clinical residency has not been evaluated. METHODS: We conducted a nationwide cross-sectional study to assess the association between the RQP and practical clinical competency based on General Medicine In-Training Examination (GM-ITE) scores. We compared the overall and category GM-ITE results between RQP graduates and other resident physicians. The relationship between the RQP and scores was examined using multilevel linear regression analysis. RESULTS: There were 4978 other resident physicians and 1119 RQP graduates out of 6097 participants from 593 training hospitals. Being younger; preferring internal, general, or emergency medicine; managing fewer inpatients; and having fewer ER shifts were all characteristics of RQP graduates. In multilevel multivariable linear regression analysis, there was no significant association between RQP graduates and total GM-ITE scores (coefficient: 0.26; 95% confidence interval: -0.09, 0.61; P = .15). The associations of RQP graduates with GM-ITE scores in each category and specialty were not clinically relevant. However, in the same multivariable model, the analysis did reveal that total GM-ITE scores demonstrated strong positive associations with younger age and GM preference, both of which were significantly common in RQP graduates. CONCLUSION: Practical clinical competency evaluated based on the GM-ITE score showed no clinically relevant differences between RQP graduates and other resident physicians. Key messages What is already known on this topic Many countries offer unique admission processes to medical schools and special undergraduate programs to increase the supply of physicians in rural areas. Concerns have been raised about the motivation, learning capabilities, and academic performance of the program graduates. What this study adds This nationwide cross-sectional study in Japan revealed clinical competency based on the scores from the General Medicine In-Training Examination showed no clinically relevant differences between graduates of regional quota programs and other resident physicians. How this study might affect research, practice, or policy The study provides evidence to support the Japanese regional quota program from the perspective of clinical competency after initiating clinical practice.

Association between prolonged weekly duty hours and self-study time among residents: a cross-sectional study.

PURPOSE: In 2024, the Japanese government will enforce a maximum 80-hour weekly duty hours (DHs) regulation for medical residents. Although this reduction in weekly DHs could increase the self-study time (SST) of these residents, the relationship between these two variables remains unclear. The aim of the study was to investigate the relationship between the SST and DHs of residents in Japan. METHODS: In this nationwide cross-sectional study, the subjects were candidates of the General Medicine In-Training Examination in the 2020 academic year. We administered questionnaires and categorically asked questions regarding daily SST and weekly DHs during the training period. To account for hospital variability, proportional odds regression models with generalized estimating equations were used to analyse the association between SST and DHs. RESULTS: Of the surveyed 6117 residents, 32.0% were female, 49.1% were postgraduate year-1 residents, 83.8% were affiliated with community hospitals, and 19.9% worked for ≥80 hours/week. Multivariable analysis revealed that residents working ≥80 hours/week spent more time on self-study than those working 60-70 hours/week. Conversely, residents who worked <50 hours/week spent less time on self-study than those who worked 60-70 hours/week. The factors associated with longer SST were sex, postgraduate year, career aspiration for internal medicine, affiliation with community hospitals, academic involvement, and well-being. CONCLUSION: Residents with long DHs had longer SSTs than residents with short DHs. Future DH restrictions may not increase but rather decrease resident SST. Effective measures to encourage self-study are required, as DH restrictions may shorten SST.

Impact of duty hours on competency-related knowledge acquisition among community hospital residents.

Background: The effect of duty hour (DH) restrictions on postgraduate residents' acquisition of clinical competencies is unclear. We evaluated the relationship between DHs and competency-related knowledge acquisition using the General Medicine In-training Examination (GM-ITE). Methods: We conducted a multicenter, cross-sectional study of community hospital residents among 2019 GM-ITE examinees. Self-reported average DHs per week were classified into five DH categories and the competency domains were classified into four areas: symptomatology and clinical reasoning (CR), physical examination and clinical procedure (PP), medical interview and professionalism (MP), and disease knowledge (DK). The association between these scores and DHs was examined using random-intercept linear models with and without adjustment for confounding factors. Results: We included 4753 participants in the analyses. Of these, 31% were women, and 49.1% were in the postgraduate year (PGY) 2. Mean CR and MP scores were lower among residents in Category 1 (<50 h) than in residents in Category 3 (≥60 and <70 h; reference group). Mean DK scores were lower among residents in Categories 1 and 2 (≥50 and <60 h) than in the reference group. PGY-2 residents in Categories 1 and 2 had lower CR scores than those in Category 3; however, PGY-1 residents in Category 5 showed higher scores. Conclusions: The relationship between DHs and each competency area is not strictly linear. The acquisition of knowledge of physical examination and clinical procedures skills in particular may not be related to DHs.

Impact of the resident duty hours on in-training examination score: A nationwide study in Japan.

PURPOSE: The relationship between duty hours (DH) and the performance of postgraduate residents is needed to establish appropriate DH limits. This study explores their relationship using the General Medicine In-training Examination (GM-ITE). MATERIALS AND METHODS: In this cross-sectional study, GM-ITE examinees of 2019 had participated. We analyzed data from the examination and questionnaire, including DH per week (eight categories). We examined the association between DH and GM-ITE score, using random-intercept linear models with and without adjustments. RESULTS: Five thousand five hundred and ninety-three participants (50.7% PGY-1, 31.6% female, 10.0% university hospitals) were included. Mean GM-ITE scores were lower among residents in Category 2 (45-50 h; mean score difference, -1.05; p < 0.001) and Category 4 (55-60 h; -0.63; p = 0.008) compared with residents in Category 5 (60-65 h; Reference). PGY-2 residents in Categories 2-4 had lower GM-ITE scores compared to those in Category 5. University residents in Category 1 and Category 5 showed a large mean difference (-3.43; p = 0.01). CONCLUSIONS: DH <60-65 h per week was independently associated with lower resident performance, but more DH did not improve performance. DH of 60-65 h per week may be the optimal balance for a resident's education and well-being.

Validation of the General Medicine in-Training Examination Using the Professional and Linguistic Assessments Board Examination Among Postgraduate Residents in Japan.

PURPOSE: In Japan, the General Medicine In-training Examination (GM-ITE) was developed by a non-profit organization in 2012. The GM-ITE aimed to assess the general clinical knowledge among residents and to improve the training programs; however, it has not been sufficiently validated and is not used for high-stake decision-making. This study examined the association between GM-ITE and another test measure, the Professional and Linguistic Assessments Board (PLAB) 1 examination. METHODS: Ninety-seven residents who completed the GM-ITE in fiscal year 2019 were recruited and took the PLAB 1 examination in Japanese. The association between two tests was assessed using the Pearson product-moment statistics. The discrimination indexes were also assessed for each question. RESULTS: A total of 91 residents at 17 teaching hospitals were finally included in the analysis, of whom 69 (75.8%) were women and 59 (64.8%) were postgraduate second year residents. All the participants were affiliated with community hospitals. Positive correlations were demonstrated between the GM-ITE and the PLAB scores (r = 0.58, p < 0.001). The correlations between the PLAB score and the scores in GM-ITE categories were as follows: symptomatology/clinical reasoning (r = 0.54, p < 0.001), physical examination/procedure (r = 0.38, p < 0.001), medical interview/professionalism (r = 0.25, p < 0.001), and disease knowledge (r = 0.36, p < 0.001). The mean discrimination index of each question of the GM-ITE (mean ± SD; 0.23 ± 0.15) was higher than that of the PLAB (0.16 ± 0.16; p = 0.004). CONCLUSION: This study demonstrates incremental validity evidence of the GM-ITE to assess the clinical knowledge acquisition. The results indicate that GM-ITE can be widely used to improve resident education in Japan.

Difference in the general medicine in-training examination score between community-based hospitals and university hospitals: a cross-sectional study based on 15,188 Japanese resident physicians.

BACKGROUND: The general medicine in-training examination (GM-ITE) is designed to objectively evaluate the postgraduate clinical competencies (PGY) 1 and 2 residents in Japan. Although the total GM-ITE scores tended to be lower in PGY-1 and PGY-2 residents in university hospitals than those in community-based hospitals, the most divergent areas of essential clinical competencies have not yet been revealed. METHODS: We conducted a nationwide, multicenter, cross-sectional study in Japan, using the GM-ITE to compare university and community-based hospitals in the four areas of basic clinical knowledge". Specifically, "medical interview and professionalism," "symptomatology and clinical reasoning," "physical examination and clinical procedures," and "disease knowledge" were assessed. RESULTS: We found no significant difference in "medical interview and professionalism" scores between the community-based and university hospital residents. However, significant differences were found in the remaining three areas. A 1.28-point difference (95% confidence interval: 0.96-1.59) in "physical examination and clinical procedures" in PGY-1 residents was found; this area alone accounts for approximately half of the difference in total score. CONCLUSIONS: The standardization of junior residency programs and the general clinical education programs in Japan should be promoted and will improve the overall training that our residents receive. This is especially needed in categories where university hospitals have low scores, such as "physical examination and clinical procedures."

Impact of general medicine rotation training on the in-training examination scores of 11, 244 Japanese resident physicians: a Nationwide multi-center cross-sectional study.

BACKGROUND: Although general medicine (GM) faculty in Japanese medical schools have an important role in educating medical students, the importance of residents' rotation training in GM in postgraduate education has not been sufficiently recognized in Japan. To evaluate the relationship between the rotation of resident physicians in the GM department and their In-Training Examination score. METHODS: This study is a nationwide multi-center cross-sectional study in Japan. Participants of this study are Japanese junior resident physicians [postgraduate year (PGY)-1 and PGY-2] who took the General Medicine In-Training Examination (GM-ITE) in fiscal years 2016 to 2018 at least once (n = 11,244). The numbers of participating hospitals in the GM-ITE were 381, 459, and 503 in 2016, 2017, and 2018.The GM-ITE score consisted of four categories (medical interview/professionalism, symptomatology/clinical reasoning, physical examination/procedure, and disease knowledge). We evaluated relationship between educational environment (including hospital information) and the GM-ITE score. RESULTS: A total of 4464 (39.7%) residents experienced GM department rotation training. Residents who rotated had higher total scores than residents who did not rotate (38.1 ± 12.1, 36.8 ± 11.7, and 36.5 ± 11.5 for residents who experienced GM rotation training, those who did not experience this training in hospitals with a GM department, and those who did not experience GM rotation training in hospitals without a GM department, p = 0.0038). The association between GM rotation and competency remained after multivariable adjustment in the multilevel model: the score difference between GM rotation training residents and non-GM rotation residents in hospitals without a GM department was estimated as 1.18 (standard error, 0.30, p = 0.0001), which was approximately half of the standard deviation of random effects due to hospital variation (estimated as 2.00). CONCLUSIONS: GM rotation training improved the GM-ITE score of residents and should be considered mandatory for junior residents in Japan.

Awareness of Diagnostic Error among Japanese Residents: a Nationwide Study.

BACKGROUND: Residents' understanding of diagnostic error may differ between countries. We sought to explore the relationship between diagnostic error knowledge and self-study, clinical knowledge, and experience. METHODS: Our nationwide study involved postgraduate year 1 and 2 (PGY-1 and -2) Japanese residents. The Diagnostic Error Knowledge Assessment Test (D-KAT) and General Medicine In-Training Examination (GM-ITE) were administered at the end of the 2014 academic year. D-KAT scores were compared with the benchmark scores of US residents. Associations between D-KAT score and gender, PGY, emergency department (ED) rotations per month, mean number of inpatients handled at any given time, and mean daily minutes of self-study were also analyzed, both with and without adjusting for GM-ITE scores. Student's t test was used for comparisons with linear mixed models and structural equation models (SEM) to explore associations with D-KAT or GM-ITE scores. RESULTS: The mean D-KAT score among Japanese PGY-2 residents was significantly lower than that of their US PGY-2 counterparts (6.2 vs. 8.3, p < 0.001). GM-ITE scores correlated with ED rotations (≥6 rotations: 2.14; 0.16-4.13; p = 0.03), inpatient caseloads (5-9 patients: 1.79; 0.82-2.76; p < 0.001), and average daily minutes of self-study (≥91 min: 2.05; 0.56-3.53; p = 0.01). SEM revealed that D-KAT scores were directly associated with GM-ITE scores (ß = 0.37, 95% CI: 0.34-0.41) and indirectly associated with ED rotations (ß = 0.06, 95% CI: 0.02-0.10), inpatient caseload (ß = 0.04, 95% CI: 0.003-0.08), and average daily minutes of study (ß = 0.13, 95% CI: 0.09-0.17). CONCLUSIONS: Knowledge regarding diagnostic error among Japanese residents was poor compared with that among US residents. D-KAT scores correlated strongly with GM-ITE scores, and the latter scores were positively associated with a greater number of ED rotations, larger caseload (though only up to 15 patients), and more time spent studying.