RESUMO
BACKGROUND: Artificial intelligence (AI)-based tools can reshape healthcare practice. This includes ChatGPT which is considered among the most popular AI-based conversational models. Nevertheless, the performance of different versions of ChatGPT needs further evaluation in different settings to assess its reliability and credibility in various healthcare-related tasks. Therefore, the current study aimed to assess the performance of the freely available ChatGPT-3.5 and the paid version ChatGPT-4 in 10 different diagnostic clinical microbiology case scenarios. METHODS: The current study followed the METRICS (Model, Evaluation, Timing/Transparency, Range/Randomization, Individual factors, Count, Specificity of the prompts/language) checklist for standardization of the design and reporting of AI-based studies in healthcare. The models tested on December 3, 2023 included ChatGPT-3.5 and ChatGPT-4 and the evaluation of the ChatGPT-generated content was based on the CLEAR tool (Completeness, Lack of false information, Evidence support, Appropriateness, and Relevance) assessed on a 5-point Likert scale with a range of the CLEAR scores of 1-5. ChatGPT output was evaluated by two raters independently and the inter-rater agreement was based on the Cohen's κ statistic. Ten diagnostic clinical microbiology laboratory case scenarios were created in the English language by three microbiologists at diverse levels of expertise following an internal discussion of common cases observed in Jordan. The range of topics included bacteriology, mycology, parasitology, and virology cases. Specific prompts were tailored based on the CLEAR tool and a new session was selected following prompting each case scenario. RESULTS: The Cohen's κ values for the five CLEAR items were 0.351-0.737 for ChatGPT-3.5 and 0.294-0.701 for ChatGPT-4 indicating fair to good agreement and suitability for analysis. Based on the average CLEAR scores, ChatGPT-4 outperformed ChatGPT-3.5 (mean: 2.64±1.06 vs. 3.21±1.05, P=.012, t-test). The performance of each model varied based on the CLEAR items, with the lowest performance for the "Relevance" item (2.15±0.71 for ChatGPT-3.5 and 2.65±1.16 for ChatGPT-4). A statistically significant difference upon assessing the performance per each CLEAR item was only seen in ChatGPT-4 with the best performance in "Completeness", "Lack of false information", and "Evidence support" (P=0.043). The lowest level of performance for both models was observed with antimicrobial susceptibility testing (AST) queries while the highest level of performance was seen in bacterial and mycologic identification. CONCLUSIONS: Assessment of ChatGPT performance across different diagnostic clinical microbiology case scenarios showed that ChatGPT-4 outperformed ChatGPT-3.5. The performance of ChatGPT demonstrated noticeable variability depending on the specific topic evaluated. A primary shortcoming of both ChatGPT models was the tendency to generate irrelevant content lacking the needed focus. Although the overall ChatGPT performance in these diagnostic microbiology case scenarios might be described as "above average" at best, there remains a significant potential for improvement, considering the identified limitations and unsatisfactory results in a few cases.
RESUMO
The ongoing multi-country human monkeypox (HMPX) outbreak was declared as a public health emergency of international concern. Considering the key role of healthcare workers (HCWs) in mitigating the HMPX outbreak, we aimed to assess their level of knowledge and their confidence in diagnosis and management of the disease, besides the assessment of their attitude towards emerging virus infections from a conspiracy point of view. An online survey was distributed among HCWs in Jordan, a Middle Eastern country, during May−July 2022 using a questionnaire published in a previous study among university students in health schools in Jordan. The study sample comprised 606 HCWs, with about two-thirds being either physicians (n = 204, 33.7%) or nurses (n = 190, 31.4%). Four out of the 11 HMPX knowledge items had <50% correct responses with only 33.3% of the study respondents having previous knowledge that vaccination is available to prevent HMPX. A majority of study respondents (n = 356, 58.7%) strongly agreed, agreed or somewhat agreed that the spread of HMPX is related to a role of male homosexuals. Confidence in the ability of diagnosis based on the available monkeypox virus diagnostic tests was reported by 50.2% of the respondents, while the confidence levels were lower for the ability to manage (38.9%) and to diagnose (38.0%) HMPX cases based on their current level of knowledge and skills. Higher confidence levels for HMPX diagnosis and management were found among physicians compared to nurses. The endorsement of conspiracy beliefs about virus emergence was associated with lower HMPX knowledge, the belief in the role of male homosexuals in HMPX spread, and with lower diagnosis and management confidence levels. The current study highlighted the gaps in knowledge regarding HMPX among HCWs in Jordan as well as the lack of confidence to diagnose and manage cases among physicians and nurses. Raising the awareness about the disease is needed urgently considering the rapid escalation in the number of cases worldwide with reported cases in the Middle East. The attitude towards male homosexuals' role in HMPX spread necessitates proper intervention measures to prevent stigma and discrimination among this risk group. The adoption of conspiratorial beliefs regarding virus emergence was widely prevalent and this issue needs to be addressed with proper and accurate knowledge considering its potential harmful impact.
RESUMO
Vaccination to prevent influenza virus infection and to lessen its severity is recommended among healthcare workers (HCWs). Health professionals have a higher risk of exposure to viruses and could transmit the influenza virus to vulnerable patients who are prone to severe disease and mortality. The aim of the current study was to evaluate the levels of influenza vaccine acceptance and uptake as well as its determinants, among Jordanian HCWs over the last influenza season of 2021/2022. This study was based on a self-administered electronic survey that was distributed in March 2022. Psychological determinants of influenza vaccine acceptance and vaccine conspiracy beliefs were assessed using the previously validated 5C scale questionnaire (confidence, complacency, constraints, calculation and collective responsibility) and the vaccine conspiracy beliefs scale. The study sample comprised a total of 1218 HCWs: nurses (n = 412, 33.8%), physicians (n = 367, 30.1%), medical technicians (n = 182, 14.9%), pharmacists (n = 161, 13.2%) and dentists (n = 87, 7.1%), among others. About two-thirds of the study sample expressed willingness to receive influenza vaccination if provided free of charge (n = 807, 66.3%), whereas less than one-third were willing to pay for the vaccine (n = 388, 31.9%). The self-reported uptake of the influenza vaccine in the last influenza season was 62.8%. The following factors were significantly associated with higher acceptance of influenza vaccination if provided freely, as opposed to vaccine hesitancy/rejection: male sex; physicians and dentists among HCW categories; higher confidence and collective responsibility; and lower complacency, constraints and calculation. Higher influenza vaccine uptake was significantly correlated with nurses and physicians among HCW categories, older age, a higher monthly income, higher confidence and collective responsibility, lower complacency and constraints and lower embrace of general vaccine conspiracy beliefs. The results of the current study can provide helpful clues to improve influenza vaccine coverage among HCWs in Jordan. Consequently, this can help to protect vulnerable patient groups and reserve valuable resources in healthcare settings. Psychological determinants appeared to be the most significant factors for vaccine acceptance and uptake, whereas the embrace of general vaccine conspiracy beliefs was associated with lower rates of influenza vaccine uptake, which should be considered in educational and interventional measures aiming to promote influenza vaccination.
RESUMO
Background and Objective: The recent multi-country outbreak of human monkeypox (HMPX) in non-endemic regions poses an emerging public health concern. University students in health schools/faculties represent a core knowledgeable group that can be helpful to study from a public health point of view. As future healthcare workers, assessment of their knowledge and attitude towards emerging zoonotic viral infections can be helpful to assess their taught material and courses with potential improvement if gaps in knowledge were identified. Therefore, we aimed to evaluate the level of HMPX knowledge, conspiracy beliefs regarding emerging virus infections, as well as their associated determinants among university students studying Medicine, Nursing, Dentistry, Pharmacy, Medical Laboratory Sciences, and Rehabilitation in Jordanian health schools/faculties. In addition, we sought to evaluate the correlation between HMPX knowledge and the extent of holding conspiracy beliefs regarding emerging viral infection. Materials and Methods: A convenient sample of university students was obtained through an electronic survey distributed in late May 2022 using the chain-referral approach. Assessment of HMPX knowledge and general attitude towards emerging virus infections was based on survey items adopted from previously published literature. Results: The study sample comprised 615 students with a mean age of 20 years and a majority of females (432, 70.2%) and medical students (n = 351, 57.1%). Out of eleven monkeypox knowledge items, three were identified correctly by >70% of the respondents. Only 26.2% of the respondents (n = 161) knew that vaccination to prevent monkeypox is available. Age was significantly associated with better HMPX knowledge for a majority of items. Older age, females, and affiliation to non-medical schools/faculties were associated with harboring higher levels of conspiracy beliefs regarding emerging virus infections. Our data also indicate that lower levels of HMPX knowledge were associated with higher levels of conspiracy beliefs. Conclusion: The current study pointed to generally unsatisfactory levels of knowledge regarding the emerging HMPX among university students in Jordanian health schools/faculties. Conspiracy beliefs regarding emerging virus infections were widely prevalent, and its potential detrimental impact on health behavior should be evaluated in future studies.
