5. The 2022 Revision of the Model Core Curriculum for Medical Education in Japan and the Common Achievement Test Being Made an Official Requirement / Participatory Clinical Clerkship / 医学教育

Following the revision of the structure and content of the Model Core Curriculum for Medical Education to be more outcome-based and the legal status of the medical practice performed by medical students in the clinical clerkship, we have revised the Guideline for Participatory Clinical Clerkship. The following items were revised or newly described : significance of enhancing the participatory clinical clerkship, scope of medical practice, confidentiality, patient consent, patient consultation and support service, objectives of the clinical clerkship, simulation education, departments where the clinical clerkship is conducted, assessment in the clinical practice setting, CC-EPOC, and entrustable professional activities. A foundation has been established to promote seamless undergraduate and postgraduate medical education. However, future work is needed to examine the specific level of performance expected at the end of the clinical clerkship and department-specific clinical practice goals and educational strategies.

Measuring students learning approaches through achievement: structural validity of SLAT-Thinking / Medindo Abordagens de Aprendizagem pelo Desempenho: Validade Estrutural do TAP-Pensamento / Midiendo los Enfoques de Aprendizaje mediante el Desempeño: Validez Estructural del TAP-Pensamiento

The field of learning approach studies measures its constructs exclusively using self-report instruments. Recently created, SLAT-Thinking is the first test to measure performance approaches in a task that requires the respondent to identify the author's thinking in a given text. This paper presents the first evidence on the structural validity of SLAT-Thinking. A sample of 622 higher education students was randomly divided into training sample and test sample. Two models were tested in the training sample through confirmatory factor analysis of items, resulting in a final model. This model presented configural, metric, and scalar invariance when comparing the training and test sample. The results indicate that SLAT-Thinking reliably measures three levels of learning approaches: superficial, intermediate-deep, deep. The measurement of levels generates information not previously identified by the area, bringing conceptual implications for the construct.

O campo de estudos em abordagens de aprendizagem mensura exclusivamente seus construtos usando instrumentos de autorrelato. Criado recentemente, o TAP-Pensamento é o primeiro teste a mensurar abordagens via o desempenho em uma tarefa que demanda ao respondente identificar o pensamento do autor em determinado texto. O presente artigo apresenta as primeiras evidências sobre a validade estrutural do TAP-Pensamento. Uma amostra de 622 estudantes do ensino superior foi dividida aleatoriamente em amostra-treino e amostra-teste. Dois modelos foram testados na amostra-treino, via análise fatorial confirmatória de itens, resultando em um modelo final. Este modelo apresentou invariância configural, métrica, e escalar, ao se comparar a amostra treino e teste. Os resultados indicam que o TAP-Pensamento mensura de forma confiável três níveis de abordagens de aprendizagem: superficial, intermediária-profunda, profunda. A mensuração de níveis gera informações anteriormente não identificadas pela área, trazendo implicações conceituais para o construto.

El campo de estudios sobre los enfoques de aprendizaje mide sus constructos exclusivamente utilizando instrumentos de autorreporte. Recientemente creado, el TAP-Pensamiento es el primer test que mide los enfoques a través del desempeño de una tarea que demanda que la persona identifique el pensamiento del autor en un texto determinado. Este artículo presenta las primeras evidencias sobre la validez estructural del TAP-Pensamiento. Una muestra de 622 estudiantes de educación superior fue dividida aleatoriamente en muestra-entrenamiento y muestra-prueba. Dos modelos fueron testados en la muestra de entrenamiento a través del análisis factorial confirmatorio de ítems, lo que resultó en un modelo final. Este modelo presentó invarianza configural, métrica y escalar al comparar las muestras de entrenamiento y prueba. Los resultados indican que el TAP-Pensamiento mide de manera confiable tres niveles de enfoques de aprendizaje: superficial, intermedio-profundo y profundo. La medición por niveles genera información no identificada previamente por el área, trayendo implicaciones conceptuales para el constructo.

Present undergraduate medical education with connection to Postgraduate education / 医学教育

Medical education at college is the very first step of life-long learning as a medical doctor. Curricular reforms in Japan took place in the early 21st century, and can be exampled by the development of a model core curriculum, the emergence of the CAT (common achievement test) examination, the development of clinical clerkship and so on. The International accreditation of medical schools has just started spring of 2017. It highlights outcome-based education, which accelerates the connection of undergraduate education with postgraduate training.

Evidências de validade do subteste de leitura do teste de desempenho escolar / Evidence of validity of the reading subtest of school achievement test / Evidencias de validad del subtest de lectura del test de desempeño escolar

O teste de desempenho escolar é um instrumento psicopedagógico desenvolvido para crianças da 1ª à 6ª série, cujo objetivo é avaliar de maneira ampla a aprendizagem por meio de três subtestes: leitura, escrita e aritmética. A proposta deste estudo foi avaliar as propriedades psicométricas do subteste de leitura, utilizando a análise de eixos principais e a teoria de resposta ao item para avaliar a estrutura interna, a dificuldade e a discriminação dos itens. A amostra foi composta de dados oriundos de pesquisas realizadas em quatro Estados brasileiros (N = 1.831). Os resultados indicaram que o subteste de leitura apresenta evidências de unidimensionalidade, avalia com mais precisão estudantes com níveis baixos e médios de habilidade e discrimina estudantes apenas até a 3ª série. Esses resultados indicam que o subteste mostra evidências de validade, mas requer refinamento.

The school achievement test is a psychopedagogical instrument for children from 1st to 6th grade, assessing broadly learning through three subtests: reading, writing and arithmetic. The goal of this study was to evaluate the psychometric properties of the reading subtest, using principal axes analysis and item response theory to evaluate the internal structure, items difficulty and discrimination. The sample consisted of data from surveys conducted in four Brazilian states (N = 1831). Results indicated that the reading subtest presents evidence of unidimensionality, more accurately assess students with low and medium skill, and discriminates students only through third grade. These results indicate that subtest presents evidence of validity, but requires refinement.

Test de desempeño escolar es un instrumento psicológico y pedagógico desarrollado para los niños de 1º a 6º grado, evaluando de manera general el aprendizaje por tres subtests: lectura, escrita y aritmética. El objetivo de este estudio fue evaluar las propiedades psicométricas del subtest de lectura mediante el análisis de los ejes principales y la teoría de respuesta al ítem al evaluar la estructura interna, la dificultad y la discriminación de materiales. La muestra consistió en datos de encuestas realizadas en cuatro estados de Brasil (N = 1.831). Los resultados indicaron que el subtest de lectura presenta evidencia de unidimensionalidad, evalua con mayor precisión a los estudiantes con baja y media capacidad y discrimina a los estudiantes sólo hasta el tercer grado. Estos resultados indican que el subtest tiene evidencias de validez, pero requiere de refinación.

Avaliação dos subtestes de leitura e escrita do teste de desempenho escolar através da Teoria de Resposta ao Item / Assessment of the reading and writing subtests of the school achievement test through Item Response Theory

O Teste de Desempenho Escolar (TDE) avalia a aprendizagem escolar através de três subtestes: leitura, escrita e aritmética. O objetivo deste estudo foi conhecer quais partes do continuum de habilidades são medidas pelos subtestes de leitura e escrita e a quantidade de informação fornecida, utilizando a Teoria de Resposta ao Item (TRI). A amostra foi composta de 1850 crianças. Os resultados indicaram que o subteste escrita mensura precisamente níveis médios de habilidade e menos satisfatoriamente níveis baixos e altos. O subteste leitura revelou ter discriminação apropriada para níveis baixos e médios de habilidade. As partes do continuum de habilidade que estão sendo medidas adequadamente estão fornecendo alta quantidade de informação, demonstrando que os subtestes leitura e escrita estão funcionando bem. (AU)

The School Achievement Test (SAT) assesses learning through three subtests: reading, writing and arithmetic. The goal of this study was to know which parts of the ability continuum are measured by the subtests as well as the amount of information provided in the reading and writing subtests using the Item Response Theory (IRT). The sample consisted of 1850 children. Results indicated that the writing subtest precisely measures average levels of ability and less satisfactorily low and high levels. The reading subtest revealed to have adequate discrimination for low and average levels of ability. The parts of the ability continuum which are adequately measured provide high amounts of information, indicating that reading and writing subtests are doing well. (AU)

Teste de desempenho escolar: um estudo de levantamento / Test for school achievement: a survey study / Prueba de rendimiento académico: un estudio de recopilación

O Teste de Desempenho Escolar (TDE), de 1994, é um instrumento psicométrico de aplicação individual que avalia de forma ampla as capacidades fundamentais para o desempenho escolar em três áreas específicas: leitura, escrita e aritmética, voltado para 1ª a 6ª séries do Ensino Fundamental. O objetivo do presente estudo foi realizar um levantamento das publicações científicas que utilizaram o TDE de 1994 a 2011. Foi realizada busca nas bases de dados nacionais e internacionais. Os dados de 222 publicações foram analisados, e observou-se um aumento gradativo do número de publicações, em especial no biênio 2008-9 com um total de 73 publicações científicas. Verificou-se que o teste segue apresentando relevância, assim como a importância de atualizar o TDE, tendo em vista que o instrumento é amplamente utilizado em todo o país, porém possui normas desatualizadas e não está adequado à nova realidade de ensino brasileiro (Ensino Fundamental de nove anos)...

The test for School Achievement (TDE), published in 1994, is a psychometric instrument that evaluates basic school achievement skills in three areas: reading, writing and arithmetics, from 1st to 6th grades. The goal of the present study was to make a survey of the scientific publications that used the TDE from 1994 to 2011. National and international databases were included. Data from 222 publications was analyzed, and there was a gradual increase in the number of publications, especially in the 2008-09 biennium, with 73 scientific publications. It was found that the TDE is still very relevant, as well as the importance of updating the TDE, due to its wide usage around the country, but outdated norms and inadequacy to the current reality of Brazilian education (basic education of nine years)...

La Prueba de Rendimiento Académico (TDE), 1994, es un instrumento psicométrico para la aplicación individual que evalúa amplias capacidades críticas para el rendimiento escolar en tres áreas específicas: lectura, escritura y aritmética, creado para los estudiantes de primero a sexto grados de la enseñanza primaria. El objetivo de este estudio fue examinar las publicaciones científicas que utilizaron el TDE 1994 hasta 2011. Se realizó una búsqueda en las bases de datos nacionales e internacionales. Los datos de 222 publicaciones se analizaron, y hubo un aumento gradual en el número de publicaciones, sobre todo en los años 2008-9 un total de 73 publicaciones científicas. Se encontró que la prueba sigue mostrando interés, así como la importancia de la actualización de la TDE, ya que el instrumento es ampliamente utilizado en todo el país, pero tiene leyes obsoletas y no se ajusta a la nueva realidad de la educación brasileña (escuela primaria nueve años)...

The effect of confidence value on students’ self-assessment / 医学教育

Introduction: We examined whether an intervention in students’ self-assessment causes behavioral changes in how they make their assessments.<br>Method: Students taking part in problem-based learning were divided into 2 groups. The students of Group 1 were asked to evaluate themselves with a four-step process, whereas the students of Group 2 students were asked to evaluated themselves and to indicate how confident they were that their self-assessments were consistent with assessments by their instructors.<br>Results: We observed a significant difference in self-assessment patterns between the groups. Students in Group 1 overestimated their abilities, whereas students in Group 2 underestimated their abilities. However, when we compared students’ self-evaluations and their grades in lecture courses, we found that students with low grades were more likely to overestimate their ability than were students with high grades, regardless of whether they had stated how confident they were in their assessments.<br>Discussion: By considering their degree of confidence that their self-assessments agreed with assessments by their instructors, students might show more careful assessment behavior. However, students with low grades require a greater degree of individual guidance before exhibiting behavioral changes.

Study of correlations between the examinees' results on the basic objective structured clinical examination (OSCE), computer-based testing, and the advanced OSCE / 医学教育

Studying the correlation between the results of the Common Achievement Test (basic objective structured clinical examination [OSCE] and computer-based testing [CBT]) and the results of advanced OSCE in the same examinees is necessary to improve clinical clerkships and to establish the role of each examination.<BR>1) Ninety-seven students took the Common Achievement Test CBT and the basic OSCE in their fourth year and took the advanced OSCE in their sixth year. The Common Achievement Test basic OSCE were composed of interview, chest, abdomen, neurology, and head and neck stations, and the CBT included blocks 1-4 and 5-6. The advanced OSCE had 3 scenarios in the chest station, 2 in the abdomen station, and 4 in the neurology station. Each scenario had 3 items. We also examined the pass analysis on the basis of these 3 items in the advanced OSCE.<BR>2) The results of the Common Achievement Test basic OSCE interview did not correlate significantly with the results of the interview station of the advanced OSCE. The results of physical examinations in the Common Achievement Test basic OSCE, excluding the chest, did not correlate significantly with the results of the physical examination stations of the advanced OSCE.<BR>3) In the advanced OSCE, an adequate medical interview is necessary for the physical examination. An adequate physical examination is also necessary for answering the written tests (e.g., differential diagnosis). 4) Students should have sufficient basic knowledge and motivation for successfully performing clinical clerkships, because the correlations among the results of the advanced OSCE interview, the OSCE total score, the Common Achievement Test basic OSCE and CBT score were extremely strong.<BR>5) The advanced OSCE, the Common Achievement Test basic OSCE, and CBT have distinct characteristics and roles.

Avaliação do desempenho acadêmico de alunos do ensino fundamental em Vitória, Espírito Santo / Achievement performance assessment of elementary school students in Vitória, Espírito Santo, Brazil

A aquisição de habilidades acadêmicas tem sido de grande interesse para a Psicologia e para a Educação, pois problemas nessas áreas podem afetar o processo de escolarização e produzir um baixo senso de auto-eficácia para aprender. Esta pesquisa avaliou as habilidades de leitura, escrita e aritmética de alunos do ensino fundamental de Vitória, Espírito Santo, num intervalo de um ano. Participaram 172 estudantes de segunda à quinta série de uma escola pública com progressão continuada (90 alunas e 82 alunos; faixa etária: 8 a 19 anos), avaliados pelo Teste de Desempenho Escolar. Nas duas avaliações, prevaleceram a classificação inferior (1ª: 56,4 por cento; 2ª: 59,3 por cento) e o melhor desempenho feminino. Na segunda avaliação, aumentou significativamente a média de acertos no Teste de Desempenho Escolar (1ª: 95,05; 2ª: 103,2 pontos), porém não alterou a classificação. Comparando as séries nas duas avaliações, observaram-se diferenças significativas entre as séries com intervalo de dois anos. Os resultados revelam dificuldades na aquisição das habilidades acadêmicas avaliadas.

Academic abilities acquisition has been focused by Psychology and Education, as problems in these areas can affect schooling process and produce a low self-efficacy learning sense. This research assessed children's reading, writing and arithmetic abilities, in an Elementary School of Vitória, Espírito Santo, Brazil during one year. 172 (One hundred and seventy two) students from the elementary course had participated: public school children from the 2nd to 5th grade in the context of the educational continuous progression (90 girls and 82 boys; ages: 8-19 years old) had been assessed by the School Achievement Test. In both assessments, the lowest classification prevailed (1st: 56.4 percent; 2nd: 59.3 percent) and women got the best performance. Correct answers increased significantly in the 2nd application (1st: 95.05; 2nd: 103.2), however, the lowest classification hasn't change. Comparing the grades in these two applications, significant differences between the grades over an interval of two years were observed. The results disclose the difficulties in those academic abilities acquisition.

Analysis of the Questionarre for Nationwide Common Achievement Test System for Entering Clinical Clerkship in Japan / 医学教育

Nationwide common achievement test system for entering clinical clerkship will formally start from December 2005. Before the start, four times trial examination were carried out. Based on these trials, some problems for the test sytem were pointed out. We have analysed the problems proposed to induce more suitable examination system.

Statistical Analysis with the Item-Response Theory of the First Trial of the Computer-Based Nationwide Common Achievement Test in Medicine / 医学教育

Data from the first trial of the computer-based nationwide common achievement test in medicine, carried out from February through July in 2002, were analyzed to evaluate the applicability of the item-response theory. The trial test was designed to cover 6 areas of the core curriculum and included a total of 2791 items. For each area, 3 to 40 items were chosen randomly and administered to 5693 students in the fourth to sixth years; the responses of 5676 of these students were analyzed with specifically designed computer systems. Each student was presented with 100 items. The itemresponse patterns were analyzed with a 3-parameter logistic model (item discrimination, item difficulty, and guessing parameter). The main findings were: 1) Item difficulty and the percentage of correct answers were strongly correlated (r=-0.969to-0.982). 2) Item discrimination and the point-biserial correlation were moderately strongly correlated (r=0.304 to 0.511). 3) The estimated abilities and the percentage of correct answers were strongly correlated (r=0.810 to 0.945). 4) The mean ability increased with school year. 5) The correlation coefficients among the 6 curriculum area ability scores were less than 0.6. Because the nationwide common achievement test was designed to randomly present items to each student, the item-response theory can be used to adjust the differences among test sets. The first trial test was designed without considering the item-response theory, but the second trial test was administered with a design better suited for comparison. Results of an analysis of the second trial will be reported soon.

Effect of Workshops for Preparing Multiple-Choice Questions for Computer-Based Testing Used in the Nationwide Common Examination Before Clinical Clerkships / 医学教育

Computer-based testing (CBT) has been used in Japan since 2002 to assess medical students' basic and clinical medical knowledge, based on the model core-curriculum, before they start clinical clerkships. For effective CBT, multiplechoice questions must accurately assess the knowledge of students. Questions for CBT are submitted by all medical schools in Japan. However, only 40% of questions are chosen for CBT and used at random; the other 60% of questions are rejected because of poor quality. Toimprove the ability of medical staff to devise questions, workshops were held at 30 medical schools. The acceptance rate of questions from schools where workshops were held was significantly increased. The workshops were extremely effbctive for improving the quality of questions.

Report of the First Trial of Nationwide Common Achievement Test-Computer-Based Testing in Medicine / 医学教育

The first trial of common achievement test-computer-based testing was held from January through August 2002. The number of examinees was 5, 693, of whom 5, 676 were analyzed. Single-best-answer, five-choice questions were used. The highest score was 92 points, the lowest score was 19 points, and the average score was 55.9±10.2 points (standard deviation). Scores were distributed normally. The test sets did not differ significantly in difficulty, although test-set items differed for each student. The percentage of correct answers, the ∅-coefficient, and the point-biserial correlation coefficient were calculated for each category of the model core curriculum. The percentage of correct answers was highest in category A of the model core curriculum, and percentages of correct answers were similar in categories B, C, D, E, and F. The ∅-coefficient and the correlation coefficient were low in categories A and F and were highest in category C. Although the percentage of correct answers in this trial was lower than expected, many test items had discriminatory power. The Test Items Evaluation Subcommittee is now evaluating test items, determining pool items, and revising new test items for the second trial and expect to compile a useful item bank.

Problems in Preparing Multiple Choice Questions for Computer-based Testing Used in the Nationwide Common Examination before Clinical Clerkships / 医学教育

In 2002, Japanese medical students began computer-based testing (CBT) to assess their basic and clinical medical knowledge, based on the model core-curriculum, before starting clinical clerkships. Of 9, 919 multiple choice questions submitted by 80 medical schools, 2, 791 were used for CBT and 7, 128 were rejected. To improve the quality of future CBT, we analyzed why questions were rejected. The most commons reasons were difficulty, length, and inappropriate choice of answers. A training course may be needed to improve the ability of medical school staff to devise questions.