Your browser doesn't support javascript.
Show: 20 | 50 | 100
Results 1 - 5 de 5
Filter
1.
J Biomed Inform ; 121: 103865, 2021 09.
Article in English | MEDLINE | ID: covidwho-1300864

ABSTRACT

We present an overview of the TREC-COVID Challenge, an information retrieval (IR) shared task to evaluate search on scientific literature related to COVID-19. The goals of TREC-COVID include the construction of a pandemic search test collection and the evaluation of IR methods for COVID-19. The challenge was conducted over five rounds from April to July 2020, with participation from 92 unique teams and 556 individual submissions. A total of 50 topics (sets of related queries) were used in the evaluation, starting at 30 topics for Round 1 and adding 5 new topics per round to target emerging topics at that state of the still-emerging pandemic. This paper provides a comprehensive overview of the structure and results of TREC-COVID. Specifically, the paper provides details on the background, task structure, topic structure, corpus, participation, pooling, assessment, judgments, results, top-performing systems, lessons learned, and benchmark datasets.


Subject(s)
COVID-19 , Pandemics , Humans , Information Storage and Retrieval , SARS-CoV-2
2.
Yearb Med Inform ; 30(1): 13-16, 2021 Aug.
Article in English | MEDLINE | ID: covidwho-1196870

ABSTRACT

BACKGROUND: On December 16, 2020 representatives of the International Medical Informatics Association (IMIA), a Non-Governmental Organization in official relations with the World Health Organization (WHO), along with its International Academy for Health Sciences Informatics (IAHSI), held an open dialogue with WHO Director General (WHO DG) Tedros Adhanom Ghebreyesus about the opportunities and challenges of digital health during the COVID-19 global pandemic. OBJECTIVES: The aim of this paper is to report the outcomes of the dialogue and discussions with more than 200 participants representing different civil society organizations (CSOs). METHODS: The dialogue was held in form of a webinar. After an initial address of the WHO DG, short presentations by the panelists, and live discussions between panelists, the WHO DG and WHO representatives took place. The audience was able to post questions in written. These written discussions were saved with participants' consent and summarized in this paper. RESULTS: The main themes that were brought up by the audience for discussion were: (a) opportunities and challenges in general; (b) ethics and artificial intelligence; (c) digital divide; (d) education. Proposed actions included the development of a roadmap based on the lessons learned from the COVID-19 pandemic. CONCLUSIONS: Decision making by policy makers needs to be evidence-based and health informatics research should be used to support decisions surrounding digital health, and we further propose next steps in the collaboration between IMIA and WHO such as future engagement in the World Health Assembly.


Subject(s)
Biomedical Technology , COVID-19 , Health Information Exchange , Medical Informatics , Telemedicine , World Health Organization , Artificial Intelligence , Global Health , Humans , Interinstitutional Relations , Medical Informatics/education , Medical Informatics/organization & administration , Societies, Medical , World Health Organization/organization & administration
3.
Yearb Med Inform ; 30(1): 8-12, 2021 Aug.
Article in English | MEDLINE | ID: covidwho-1196869

ABSTRACT

OBJECTIVES: To summarize the major activities of the International Academy of Health Sciences Informatics (IAHSI) in the 2020 time period and to welcome its 2020 Class of Fellows. METHOD: Report from the members of the Academy's Board. RESULTS: Due to the SARS-CoV-2 pandemic, both Plenary meetings in 2020 had to be organized as virtual meetings. Scientific discussions, focusing on mobilizing computable biomedical knowledge and on data standards and interoperability formed major parts of these meetings. A statement on the use of informatics in pandemic situations was elaborated and sent to the World Health Organization. A panel on data standards and interoperability started its work. 34 Fellows were welcomed in the 2020 Class of Fellows so that the Academy now consists of 179 members. CONCLUSIONS: There was a shift from supporting to strategic activities in the Academy's work. After having achieved organizational stability, the Academy can now focus on its strategic work and so on its main objective.


Subject(s)
Academies and Institutes/organization & administration , Medical Informatics , Global Health , National Academy of Sciences, U.S. , United States
4.
J Biomed Inform ; 117: 103745, 2021 05.
Article in English | MEDLINE | ID: covidwho-1163986

ABSTRACT

The COVID-19 pandemic has resulted in a rapidly growing quantity of scientific publications from journal articles, preprints, and other sources. The TREC-COVID Challenge was created to evaluate information retrieval (IR) methods and systems for this quickly expanding corpus. Using the COVID-19 Open Research Dataset (CORD-19), several dozen research teams participated in over 5 rounds of the TREC-COVID Challenge. While previous work has compared IR techniques used on other test collections, there are no studies that have analyzed the methods used by participants in the TREC-COVID Challenge. We manually reviewed team run reports from Rounds 2 and 5, extracted features from the documented methodologies, and used a univariate and multivariate regression-based analysis to identify features associated with higher retrieval performance. We observed that fine-tuning datasets with relevance judgments, MS-MARCO, and CORD-19 document vectors was associated with improved performance in Round 2 but not in Round 5. Though the relatively decreased heterogeneity of runs in Round 5 may explain the lack of significance in that round, fine-tuning has been found to improve search performance in previous challenge evaluations by improving a system's ability to map relevant queries and phrases to documents. Furthermore, term expansion was associated with improvement in system performance, and the use of the narrative field in the TREC-COVID topics was associated with decreased system performance in both rounds. These findings emphasize the need for clear queries in search. While our study has some limitations in its generalizability and scope of techniques analyzed, we identified some IR techniques that may be useful in building search systems for COVID-19 using the TREC-COVID test collections.


Subject(s)
COVID-19 , Information Storage and Retrieval , Pandemics , Humans , Multivariate Analysis , SARS-CoV-2
5.
J Am Med Inform Assoc ; 27(9): 1431-1436, 2020 07 01.
Article in English | MEDLINE | ID: covidwho-165244

ABSTRACT

TREC-COVID is an information retrieval (IR) shared task initiated to support clinicians and clinical research during the COVID-19 pandemic. IR for pandemics breaks many normal assumptions, which can be seen by examining 9 important basic IR research questions related to pandemic situations. TREC-COVID differs from traditional IR shared task evaluations with special considerations for the expected users, IR modality considerations, topic development, participant requirements, assessment process, relevance criteria, evaluation metrics, iteration process, projected timeline, and the implications of data use as a post-task test collection. This article describes how all these were addressed for the particular requirements of developing IR systems under a pandemic situation. Finally, initial participation numbers are also provided, which demonstrate the tremendous interest the IR community has in this effort.


Subject(s)
Betacoronavirus , Coronavirus Infections , Information Storage and Retrieval , Pandemics , Pneumonia, Viral , COVID-19 , Humans , Information Storage and Retrieval/methods , SARS-CoV-2 , Search Engine
SELECTION OF CITATIONS
SEARCH DETAIL
...