GenSLMs: Genome-scale language models reveal SARS-CoV-2 evolutionary dynamics.

We seek to transform how new and emergent variants of pandemiccausing viruses, specifically SARS-CoV-2, are identified and classified. By adapting large language models (LLMs) for genomic data, we build genome-scale language models (GenSLMs) which can learn the evolutionary landscape of SARS-CoV-2 genomes. By pretraining on over 110 million prokaryotic gene sequences and finetuning a SARS-CoV-2-specific model on 1.5 million genomes, we show that GenSLMs can accurately and rapidly identify variants of concern. Thus, to our knowledge, GenSLMs represents one of the first whole genome scale foundation models which can generalize to other prediction tasks. We demonstrate scaling of GenSLMs on GPU-based supercomputers and AI-hardware accelerators utilizing 1.63 Zettaflops in training runs with a sustained performance of 121 PFLOPS in mixed precision and peak of 850 PFLOPS. We present initial scientific insights from examining GenSLMs in tracking evolutionary dynamics of SARS-CoV-2, paving the path to realizing this on large biological data.

#COVIDisAirborne: AI-Enabled Multiscale Computational Microscopy of Delta SARS-CoV-2 in a Respiratory Aerosol

We seek to completely revise current models of airborne transmission of respiratory viruses by providing never-before-seen atomic-level views of the SARS-CoV-2 virus within a respiratory aerosol. Our work dramatically extends the capabilities of multiscale computational microscopy to address the significant gaps that exist in current experimental methods, which are limited in their ability to interrogate aerosols at the atomic/molecular level and thus ob-scure our understanding of airborne transmission. We demonstrate how our integrated data-driven platform provides a new way of exploring the composition, structure, and dynamics of aerosols and aerosolized viruses, while driving simulation method development along several important axes. We present a series of initial scientific discoveries for the SARS-CoV-2 Delta variant, noting that the full scientific impact of this work has yet to be realized. ACM Reference FormatAbigail Dommer1{dagger}, Lorenzo Casalino1{dagger}, Fiona Kearns1{dagger}, Mia Rosenfeld1, Nicholas Wauer1, Surl-Hee Ahn1, John Russo,2 Sofia Oliveira3, Clare Morris1, AnthonyBogetti4, AndaTrifan5,6, Alexander Brace5,7, TerraSztain1,8, Austin Clyde5,7, Heng Ma5, Chakra Chennubhotla4, Hyungro Lee9, Matteo Turilli9, Syma Khalid10, Teresa Tamayo-Mendoza11, Matthew Welborn11, Anders Christensen11, Daniel G. A. Smith11, Zhuoran Qiao12, Sai Krishna Sirumalla11, Michael OConnor11, Frederick Manby11, Anima Anandkumar12,13, David Hardy6, James Phillips6, Abraham Stern13, Josh Romero13, David Clark13, Mitchell Dorrell14, Tom Maiden14, Lei Huang15, John McCalpin15, Christo- pherWoods3, Alan Gray13, MattWilliams3, Bryan Barker16, HarindaRajapaksha16, Richard Pitts16, Tom Gibbs13, John Stone6, Daniel Zuckerman2*, Adrian Mulholland3*, Thomas MillerIII11,12*, ShantenuJha9*, Arvind Ramanathan5*, Lillian Chong4*, Rommie Amaro1*. 2021. #COVIDisAirborne: AI-Enabled Multiscale Computational Microscopy ofDeltaSARS-CoV-2 in a Respiratory Aerosol. In Supercomputing 21: International Conference for High Perfor-mance Computing, Networking, Storage, and Analysis. ACM, New York, NY, USA, 14 pages. https://doi.org/finalDOI