RESUMO
Background: Understanding complex biological pathways, including gene-gene interactions and gene regulatory networks, is critical for exploring disease mechanisms and drug development. Manual literature curation of biological pathways is useful but cannot keep up with the exponential growth of the literature. Large-scale language models (LLMs), notable for their vast parameter sizes and comprehensive training on extensive text corpora, have great potential in automated text mining of biological pathways. Method: This study assesses the effectiveness of 21 LLMs, including both API-based models and open-source models. The evaluation focused on two key aspects: gene regulatory relations (specifically, 'activation', 'inhibition', and 'phosphorylation') and KEGG pathway component recognition. The performance of these models was analyzed using statistical metrics such as precision, recall, F1 scores, and the Jaccard similarity index. Results: Our results indicated a significant disparity in model performance. Among the API-based models, ChatGPT-4 and Claude-Pro showed superior performance, with an F1 score of 0.4448 and 0.4386 for the gene regulatory relation prediction, and a Jaccard similarity index of 0.2778 and 0.2657 for the KEGG pathway prediction, respectively. Open-source models lagged their API-based counterparts, where Falcon-180b-chat and llama1-7b led with the highest performance in gene regulatory relations (F1 of 0.2787 and 0.1923, respectively) and KEGG pathway recognition (Jaccard similarity index of 0.2237 and 0. 2207, respectively). Conclusion: LLMs are valuable in biomedical research, especially in gene network analysis and pathway mapping. However, their effectiveness varies, necessitating careful model selection. This work also provided a case study and insight into using LLMs as knowledge graphs.
RESUMO
The second conference of the Nigerian Bioinformatics and Genomics Network (NBGN21) was held from October 11 to October 13, 2021. The event was organized by the Nigerian Bioinformatics and Genomics Network. A 1-day genomic analysis workshop on genome-wide association study and polygenic risk score analysis was organized as part of the conference. It was organized primarily as a research capacity building initiative to empower Nigerian researchers to take a leading role in this cutting-edge field of genomic data science. The theme of the conference was "Leveraging Bioinformatics and Genomics for the attainments of the Sustainable Development Goals." The conference used a hybrid approach-virtual and in-person. It served as a platform to bring together 235 registered participants mainly from Nigeria and virtually, from all over the world. NBGN21 had four keynote speakers and four leading Nigerian scientists received awards for their contributions to genomics and bioinformatics development in Nigeria. A total of 100 travel fellowships were awarded to delegates within Nigeria. A major topic of discussion was the application of bioinformatics and genomics in the achievement of the Sustainable Development Goals (SDG3-Good Health and Well-Being, SDG4-Quality Education, and SDG 15-Life on Land [Biodiversity]). In closing, most of the NBGN21 conference participants were interviewed and interestingly they agreed that bioinformatics and genomic analysis of African genomes are vital in identifying population-specific genetic variants that confer susceptibility to different diseases that are endemic in Africa. The knowledge of this can empower African healthcare systems and governments for timely intervention, thereby enhancing good health and well-being.
