Performance of ChatGPT on questions from the Brazilian College of Radiology annual resident evaluation test / Desempenho do ChatGPT nas questões da avaliação anual de residentes do Colégio Brasileiro de Radiologia

Abstract Objective: To test the performance of ChatGPT on radiology questions formulated by the Colégio Brasileiro de Radiologia (CBR, Brazilian College of Radiology), evaluating its failures and successes. Materials and Methods: 165 questions from the CBR annual resident assessment (2018, 2019, and 2022) were presented to ChatGPT. For statistical analysis, the questions were divided by the type of cognitive skills assessed (lower or higher order), by topic (physics or clinical), by subspecialty, by style (description of a clinical finding or sign, clinical management of a case, application of a concept, calculation/classification of findings, correlations between diseases, or anatomy), and by target academic year (all, second/third year, or third year only). Results: ChatGPT answered 88 (53.3%) of the questions correctly. It performed significantly better on the questions assessing lower-order cognitive skills than on those assessing higher-order cognitive skills, providing the correct answer on 38 (64.4%) of 59 questions and on only 50 (47.2%) of 106 questions, respectively (p = 0.01). The accuracy rate was significantly higher for physics questions than for clinical questions, correct answers being provided for 18 (90.0%) of 20 physics questions and for 70 (48.3%) of 145 clinical questions (p = 0.02). There was no significant difference in performance among the subspecialties or among the academic years (p > 0.05). Conclusion: Even without dedicated training in this field, ChatGPT demonstrates reasonable performance, albeit still insufficient for approval, on radiology questions formulated by the CBR.

Resumo Objetivo: Testar o desempenho do ChatGPT em questões de radiologia formuladas pelo Colégio Brasileiro de Radiologia (CBR), avaliando seus erros e acertos. Materiais e Métodos: 165 questões da avaliação anual dos residentes do CBR (2018, 2019 e 2022) foram apresentadas ao ChatGPT. Elas foram divididas, para análise estatística, em questões que avaliavam habilidades cognitivas de ordem superior ou inferior e de acordo com a subespecialidade, o tipo da questão (descrição de um achado clínico ou sinal, manejo clínico de um doente, aplicação de um conceito, cálculo ou classificação dos achados descritos, associação entre doenças ou anatomia) e o ano da residência (R1, R2 ou R3). Resultados: O ChatGPT acertou 53,3% das questões (88/165). Houve diferença estatística entre o desempenho em questões de ordem cognitiva inferior (64,4%; 38/59) e superior (47,2%; 50/106) (p = 0,01). Houve maior índice de acertos em física (90,0%; 18/20) do que em questões clínicas (48,3%; 70/145) (p = 0,02). Não houve diferença significativa de desempenho entre subespecialidades ou ano de residência (p > 0,05). Conclusão: Mesmo sem treinamento dedicado a essa área, o ChatGPT apresenta desempenho razoável, mas ainda insuficiente para aprovação, em questões de radiologia formuladas pelo CBR.

The use of e-ASPECTS in acute stroke care: validation of method performance compared to the performance of specialists / O uso do e-ASPECTS no AVC agudo: validação da performance do método comparada com a performance dos especialistas

ABSTRACT Background: The rapid evaluation of non-contrast-enhanced computed tomography (NCCT) brain scans in patients with anterior stroke symptoms saves time and favors optimal and prompt treatment. e-ASPECTS is a tool that automatically calculates the Alberta Stroke Program Early CT Score (ASPECTS) values, leading to a more accurate and timely image evaluation. Objective: To determine the ability of e-ASPECTS in differentiating images with and without injury. Methods: One-hundred sixteen patients admitted to a stroke unit in a Brazilian tertiary hospital underwent a CT scan at admission and at least one control brain imaging (NCCT or magnetic resonance imaging - MRI) 24 hours after admission. ASPECTS evaluation was performed by three neuroradiologists, three neurologists, and three neurology residents, all blinded to the symptoms and the injury side. The scores were compared to the ground truth, and an ASPECTS score was provided by two independent non blinded evaluators. Sensitivity and specificity were analyzed, and receiver operating characteristic curves, Bland-Altman plots with mean error score, and Matthews correlation coefficients (MCCs) were obtained for ASPECTS scores, assuming values equal to 10 for images without injury and values other than 10 for images with ischemic injury. Results: e-ASPECTS demonstrated similar performance to that of neuroradiologists and neurologists, with an area under the curve of 0.78 and an MCC value of 0.48 in the dichotomous analysis. The sensitivity and specificity of e-ASPECTS were 75% and 73%, respectively. Conclusion: e-ASPECTS is a validated and reliable tool for determining early signs of ischemia in NCCT.

RESUMO Introdução: A avaliação rápida da tomografia de crânio sem contraste (TCSC) em pacientes com AVC de circulação anterior economiza tempo e permite um tratamento rápido e otimizado. O e-ASPECTS é um software que calcula automaticamente os valores do ASPECTS e permite uma avaliação da imagem mais precisa e ágil. Objetivo: Determinar a habilidade do e-ASPECTS em diferenciar imagens com e sem lesão. Métodos: Cento e dezesseis pacientes admitidos em uma unidade de AVC de um hospital terciário brasileiro foram submetidos a uma TCSC na admissão e pelo menos uma imagem de controle (TC ou Ressonância de Crânio) 24 horas após a admissão. A avaliação do ASPECTS foi realizada por três neurorradiologistas, três neurologistas e três residentes em neurologia, todos cegados para os sintomas e para o lado da lesão. Os valores foram comparados ao ground truth (GT) e uma pontuação ASPECTS foi obtida por dois avaliadores independentes não cegos. Análise da sensibilidade e especificidade, características das curvas ROC, gráficos de Bland-Altman com média de escore de erro e coeficientes de correlação de Matthews (CCM) foram realizados para os valores de ASPECTS, assumindo valores iguais a 10 como imagens sem lesões e valores diferentes de 10 como imagens com alguma lesão isquêmica. Resultados: o e-ASPECTS demonstrou uma performance similar aos neurorradiologistas e neurologistas, com uma área sob a curva de 0,78 e um valor de CCM de 0,48 na análise dicotômica. Sensibilidade e especificidade do e-ASPECTS foram, respectivamente, 75 e 73%. Conclusão: O e-ASPECTS é uma ferramenta confiável e validada para determinar sinais precoces de isquemia nas TCSC.

Leprosy and hepatitis B coinfection in southern Brazil

To investigate the association of leprosy with hepatitis B virus (HBV) infection, as yet unknown for South Brazil, we assessed hepatitis B virus coinfection in 199 South Brazilian leprosy patients (119 lepromatous, 15 tuberculoid, 30 borderline, 12 undetermined and 23 unspecified) and in 681 matched blood donors by screening for the hepatitis B virus markers HBSAg and anti-HBc, using ELISA. Positive samples were retested and anti-HBc+ only samples were tested for the hepatitis B surface antibody (anti-HBs). There was a strong association between leprosy and hepatitis B virus infection (OR = 9.8, 95% CI = 6.4–14.7; p = 0.004·E−30), as well as an association between HBV infection and lepromatous leprosy, compared to other forms (OR = 2.4, 95% CI = 1.2–4.8; p = 0.017). We also found that confinement due to leprosy was associated with hepatitis B virus infection (OR = 3.9, 95% CI = 2.1–7.4; p = 0.015·E−3). Leprosy patients are susceptible to develop hepatitis B virus infection, especially lepromatous. Institutionalized patients, who probably present a stronger Th2 response, have higher risk of being exposed to hepatitis B virus. This clearly emphasizes the need for special care to leprosy patients in preventing hepatitis B virus coinfection in South Brazil.