Evaluation and optimization of sequence-based gene regulatory deep learning models.

Neural networks have emerged as immensely powerful tools in predicting functional genomic regions, notably evidenced by recent successes in deciphering gene regulatory logic. However, a systematic evaluation of how model architectures and training strategies impact genomics model performance is lacking. To address this gap, we held a DREAM Challenge where competitors trained models on a dataset of millions of random promoter DNA sequences and corresponding expression levels, experimentally determined in yeast, to best capture the relationship between regulatory DNA and gene expression. For a robust evaluation of the models, we designed a comprehensive suite of benchmarks encompassing various sequence types. While some benchmarks produced similar results across the top-performing models, others differed substantially. All top-performing models used neural networks, but diverged in architectures and novel training strategies, tailored to genomics sequence data. To dissect how architectural and training choices impact performance, we developed the Prix Fixe framework to divide any given model into logically equivalent building blocks. We tested all possible combinations for the top three models and observed performance improvements for each. The DREAM Challenge models not only achieved state-of-the-art results on our comprehensive yeast dataset but also consistently surpassed existing benchmarks on Drosophila and human genomic datasets. Overall, we demonstrate that high-quality gold-standard genomics datasets can drive significant progress in model development.

Proformer: a hybrid macaron transformer model predicts expression values from promoter sequences.

The breakthrough high-throughput measurement of the cis-regulatory activity of millions of randomly generated promoters provides an unprecedented opportunity to systematically decode the cis-regulatory logic that determines the expression values. We developed an end-to-end transformer encoder architecture named Proformer to predict the expression values from DNA sequences. Proformer used a Macaron-like Transformer encoder architecture, where two half-step feed forward (FFN) layers were placed at the beginning and the end of each encoder block, and a separable 1D convolution layer was inserted after the first FFN layer and in front of the multi-head attention layer. The sliding k-mers from one-hot encoded sequences were mapped onto a continuous embedding, combined with the learned positional embedding and strand embedding (forward strand vs. reverse complemented strand) as the sequence input. Moreover, Proformer introduced multiple expression heads with mask filling to prevent the transformer models from collapsing when training on relatively small amount of data. We empirically determined that this design had significantly better performance than the conventional design such as using the global pooling layer as the output layer for the regression task. These analyses support the notion that Proformer provides a novel method of learning and enhances our understanding of how cis-regulatory sequences determine the expression values.

MCHeart: Multi-Channel-Based Heart Signal Processing Scheme for Heart Noise Detection Using Deep Learning.

In this study, we constructed a model to predict abnormal cardiac sounds using a diverse set of auscultation data collected from various auscultation positions. Abnormal heart sounds were identified by extracting features such as peak intervals and noise characteristics during systole and diastole. Instead of using raw signal data, we transformed them into log-mel 2D spectrograms, which were employed as input variables for the CNN model. The advancement of our model involves integrating a deep learning architecture with feature extraction techniques based on existing knowledge of cardiac data. Specifically, we propose a multi-channel-based heart signal processing (MCHeart) scheme, which incorporates our proposed features into the deep learning model. Additionally, we introduce the ReLCNN model by applying residual blocks and MHA mechanisms to the LCNN architecture. By adding murmur features with a smoothing function and training the ReLCNN model, the weighted accuracy of the model increased from 79.6% to 83.6%, showing a performance improvement of approximately 4% point compared to the LCNN baseline model.

SeATAC: a tool for exploring the chromatin landscape and the role of pioneer factors.

Assay for Transposase-Accessible Chromatin with sequencing (ATAC-seq) reveals chromatin accessibility across the genome. Currently, no method specifically detects differential chromatin accessibility. Here, SeATAC uses a conditional variational autoencoder model to learn the latent representation of ATAC-seq V-plots and outperforms MACS2 and NucleoATAC on six separate tasks. Applying SeATAC to several pioneer factor-induced differentiation or reprogramming ATAC-seq datasets suggests that induction of these factors not only relaxes the closed chromatin but also decreases chromatin accessibility of 20% to 30% of their target sites. SeATAC is a novel tool to accurately reveal genomic regions with differential chromatin accessibility from ATAC-seq data.

FOXK1 regulates Wnt signalling to promote cardiogenesis.

AIMS: Congenital heart disease (CHD) is the most common genetic birth defect, which has considerable morbidity and mortality. We focused on deciphering key regulators that govern cardiac progenitors and cardiogenesis. FOXK1 is a forkhead/winged helix transcription factor known to regulate cell cycle kinetics and is restricted to mesodermal progenitors, somites, and heart. In the present study, we define an essential role for FOXK1 during cardiovascular development. METHODS AND RESULTS: We used the mouse embryoid body system to differentiate control and Foxk1 KO embryonic stem cells into mesodermal, cardiac progenitor cells and mature cardiac cells. Using flow cytometry, immunohistochemistry, cardiac beating, transcriptional and chromatin immunoprecipitation quantitative polymerase chain reaction assays, bulk RNA sequencing (RNAseq) and assay for transposase-accessible chromatin using sequencing (ATACseq) analyses, FOXK1 was observed to be an important regulator of cardiogenesis. Flow cytometry analyses revealed perturbed cardiogenesis in Foxk1 KO embryoid bodies (EBs). Bulk RNAseq analysis at two developmental stages showed a significant reduction of the cardiac molecular program in Foxk1 KO EBs compared to the control EBs. ATACseq analysis during EB differentiation demonstrated that the chromatin landscape nearby known important regulators of cardiogenesis was significantly relaxed in control EBs compared to Foxk1 KO EBs. Furthermore, we demonstrated that in the absence of FOXK1, cardiac differentiation was markedly impaired by assaying for cardiac Troponin T expression and cardiac contractility. We demonstrate that FOXK1 is an important regulator of cardiogenesis by repressing the Wnt/ß-catenin signalling pathway and thereby promoting differentiation. CONCLUSION: These results identify FOXK1 as an essential transcriptional and epigenetic regulator of cardiovascular development. Mechanistically, FOXK1 represses Wnt signalling to promote the development of cardiac progenitor cells.

ETV2 and VEZF1 interaction and regulation of the hematoendothelial lineage during embryogenesis.

Ets variant 2 (Etv2), a member of the Ets factor family, has an essential role in the formation of endothelial and hematopoietic cell lineages during embryonic development. The functional role of ETS transcription factors is, in part, dependent on the interacting proteins. There are relatively few studies exploring the coordinated interplay between ETV2 and its interacting proteins that regulate mesodermal lineage determination. In order to identify novel ETV2 interacting partners, a yeast two-hybrid analysis was performed and the C2H2 zinc finger transcription factor VEZF1 (vascular endothelial zinc finger 1) was identified as a binding factor, which was specifically expressed within the endothelium during vascular development. To confirm this interaction, co-immunoprecipitation and GST pull down assays demonstrated the direct interaction between ETV2 and VEZF1. During embryoid body differentiation, Etv2 achieved its peak expression at day 3.0 followed by rapid downregulation, on the other hand Vezf1 expression increased through day 6 of EB differentiation. We have previously shown that ETV2 potently activated Flt1 gene transcription. Using a Flt1 promoter-luciferase reporter assay, we demonstrated that VEZF1 co-activated the Flt1 promoter. Electrophoretic mobility shift assay and Chromatin immunoprecipitation established VEZF1 binding to the Flt1 promoter. Vezf1 knockout embryonic stem cells had downregulation of hematoendothelial marker genes when undergoing embryoid body mediated mesodermal differentiation whereas overexpression of VEZF1 induced the expression of hematoendothelial genes during differentiation. These current studies provide insight into the co-regulation of the hemato-endothelial lineage development via a co-operative interaction between ETV2 and VEZF1.

Etv2 regulates enhancer chromatin status to initiate Shh expression in the limb bud.

Sonic hedgehog (Shh) is essential for limb development, and the mechanisms that govern the propagation and maintenance of its expression has been well studied; however, the mechanisms that govern the initiation of Shh expression are incomplete. Here we report that ETV2 initiates Shh expression by changing the chromatin status of the developmental limb enhancer, ZRS. Etv2 expression precedes Shh in limb buds, and Etv2 inactivation prevents the opening of limb chromatin, including the ZRS, resulting in an absence of Shh expression. Etv2 overexpression in limb buds causes nucleosomal displacement at the ZRS, ectopic Shh expression, and polydactyly. Areas of nucleosome displacement coincide with ETS binding site clusters. ETV2 also functions as a transcriptional activator of ZRS and is antagonized by ETV4/5 repressors. Known human polydactyl mutations introduce novel ETV2 binding sites in the ZRS, suggesting that ETV2 dosage regulates ZRS activation. These studies identify ETV2 as a pioneer transcription factor (TF) regulating the onset of Shh expression, having both a chromatin regulatory role and a transcriptional activation role.

ETV2 functions as a pioneer factor to regulate and reprogram the endothelial lineage.

The vasculature is an essential organ for the delivery of blood and oxygen to all tissues of the body and is thus relevant to the treatment of ischaemic diseases, injury-induced regeneration and solid tumour growth. Previously, we demonstrated that ETV2 is an essential transcription factor for the development of cardiac, endothelial and haematopoietic lineages. Here we report that ETV2 functions as a pioneer factor that relaxes closed chromatin and regulates endothelial development. By comparing engineered embryonic stem cell differentiation and reprogramming models with multi-omics techniques, we demonstrated that ETV2 was able to bind nucleosomal DNA and recruit BRG1. BRG1 recruitment remodelled chromatin around endothelial genes and helped to maintain an open configuration, resulting in increased H3K27ac deposition. Collectively, these results will serve as a platform for the development of therapeutic initiatives directed towards cardiovascular diseases and solid tumours.

Single cell lineage reconstruction using distance-based algorithms and the R package, DCLEAR.

BACKGROUND: DCLEAR is an R package used for single cell lineage reconstruction. The advances of CRISPR-based gene editing technologies have enabled the prediction of cell lineage trees based on observed edited barcodes from each cell. However, the performance of existing reconstruction methods of cell lineage trees was not accessed until recently. In response to this problem, the Allen Institute hosted the Cell Lineage Reconstruction Dream Challenge in 2020 to crowdsource relevant knowledge from across the world. Our team won sub-challenges 2 and 3 in the challenge competition. RESULTS: The DCLEAR package contained the R codes, which was submitted in response to sub-challenges 2 and 3. Our method consists of two steps: (1) distance matrix estimation and (2) the tree reconstruction from the distance matrix. We proposed two novel methods for distance matrix estimation as outlined in the DCLEAR package. Using our method, we find that two of the more sophisticated distance methods display a substantially improved level of performance compared to the traditional Hamming distance method. DCLEAR is open source and freely available from R CRAN and from under the GNU General Public License, version 3. CONCLUSIONS: DCLEAR is a powerful resource for single cell lineage reconstruction.

Benchmarked approaches for reconstruction of in vitro cell lineages and in silico models of C. elegans and M. musculus developmental trees.

The recent advent of CRISPR and other molecular tools enabled the reconstruction of cell lineages based on induced DNA mutations and promises to solve the ones of more complex organisms. To date, no lineage reconstruction algorithms have been rigorously examined for their performance and robustness across dataset types and number of cells. To benchmark such methods, we decided to organize a DREAM challenge using in vitro experimental intMEMOIR recordings and in silico data for a C. elegans lineage tree of about 1,000 cells and a Mus musculus tree of 10,000 cells. Some of the 22 approaches submitted had excellent performance, but structural features of the trees prevented optimal reconstructions. Using smaller sub-trees as training sets proved to be a good approach for tuning algorithms to reconstruct larger trees. The simulation and reconstruction methods here generated delineate a potential way forward for solving larger cell lineage trees such as in mouse.

Humanized skeletal muscle in MYF5/MYOD/MYF6-null pig embryos.

Because post-mortem human skeletal muscle is not viable, autologous muscle grafts are typically required in tissue reconstruction after muscle loss due to disease or injury. However, the use of autologous tissue often leads to donor-site morbidity. Here, we show that intraspecies and interspecies chimaeric pig embryos lacking native skeletal muscle can be produced by deleting the MYF5, MYOD and MYF6 genes in the embryos via CRISPR, followed by somatic-cell nuclear transfer and the delivery of exogenous cells (porcine blastomeres or human induced pluripotent stem cells) via blastocyst complementation. The generated intraspecies chimaeras were viable and displayed normal histology, morphology and function. Human:pig chimaeras generated with TP53-null human induced pluripotent stem cells led to higher chimaerism efficiency, with embryos collected at embryonic days 20 and 27 containing humanized muscle, as confirmed by immunohistochemical and molecular analyses. Human:pig chimaeras may facilitate the production of exogenic organs for research and xenotransplantation.

ETV2 (Ets Variant Transcription Factor 2)-Rhoj Cascade Regulates Endothelial Progenitor Cell Migration During Embryogenesis.

OBJECTIVE: Endothelial progenitors migrate early during embryogenesis to form the primary vascular plexus. The regulatory mechanisms that govern their migration are not completely defined. Here, we describe a novel role for ETV2 (Ets variant transcription factor 2) in cell migration and provide evidence for an ETV2-Rhoj network as a mechanism responsible for this process. Approach and Results: Analysis of RNAseq datasets showed robust enrichment of migratory/motility pathways following overexpression of ETV2 during mesodermal differentiation. We then analyzed ETV2 chromatin immunoprecipitation-seq and assay for transposase accessible chromatin-seq datasets, which showed enrichment of chromatin immunoprecipitation-seq peaks with increased chromatin accessibility in migratory genes following overexpression of ETV2. Migratory assays showed that overexpression of ETV2 enhanced cell migration in mouse embryonic stem cells, embryoid bodies, and mouse embryonic fibroblasts. Knockout of Etv2 led to migratory defects of Etv2-EYFP+ angioblasts to their predefined regions of developing embryos relative to wild-type controls at embryonic day (E) 8.5, supporting its role during migration. Mechanistically, we showed that ETV2 binds the promoter region of Rhoj serving as an upstream regulator of cell migration. Single-cell RNAseq analysis of Etv2-EYFP+ sorted cells revealed coexpression of Etv2 and Rhoj in endothelial progenitors at E7.75 and E8.25. Overexpression of ETV2 led to a robust increase in Rhoj in both embryoid bodies and mouse embryonic fibroblasts, whereas, its expression was abolished in the Etv2 knockout embryoid bodies. Finally, shRNA-mediated knockdown of Rhoj resulted in migration defects, which were partially rescued by overexpression of ETV2. CONCLUSIONS: These results define an ETV2-Rhoj cascade, which is important for the regulation of endothelial progenitor cell migration.

IRX3/5 regulate mitotic chromatid segregation and limb bud shape.

Pattern formation is influenced by transcriptional regulation as well as by morphogenetic mechanisms that shape organ primordia, although factors that link these processes remain under-appreciated. Here we show that, apart from their established transcriptional roles in pattern formation, IRX3/5 help to shape the limb bud primordium by promoting the separation and intercalation of dividing mesodermal cells. Surprisingly, IRX3/5 are required for appropriate cell cycle progression and chromatid segregation during mitosis, possibly in a nontranscriptional manner. IRX3/5 associate with, promote the abundance of, and share overlapping functions with co-regulators of cell division such as the cohesin subunits SMC1, SMC3, NIPBL and CUX1. The findings imply that IRX3/5 coordinate early limb bud morphogenesis with skeletal pattern formation.

Stem Cell-Derived Cardiomyocytes and Beta-Adrenergic Receptor Blockade in Duchenne Muscular Dystrophy Cardiomyopathy.

BACKGROUND: Although cardiomyopathy has emerged as a leading cause of death in Duchenne muscular dystrophy (DMD), limited studies and therapies have emerged for dystrophic heart failure. OBJECTIVES: The purpose of this study was to model DMD cardiomyopathy using DMD patient-specific human induced pluripotent stem cell (hiPSC)-derived cardiomyocytes and to identify physiological changes and future drug therapies. METHODS: To explore and define therapies for DMD cardiomyopathy, the authors used DMD patient-specific hiPSC-derived cardiomyocytes to examine the physiological response to adrenergic agonists and ß-blocker treatment. The authors further examined these agents in vivo using wild-type and mdx mouse models. RESULTS: At baseline and following adrenergic stimulation, DMD hiPSC-derived cardiomyocytes had a significant increase in arrhythmic calcium traces compared to isogenic controls. Furthermore, these arrhythmias were significantly decreased with propranolol treatment. Using telemetry monitoring, the authors observed that mdx mice, which lack dystrophin, had an arrhythmic death when stimulated with isoproterenol; the lethal arrhythmias were rescued, in part, by propranolol pre-treatment. Using single-cell and bulk RNA sequencing (RNA-seq), the authors compared DMD and control hiPSC-derived cardiomyocytes, mdx mice, and control mice (in the presence or absence of propranolol and isoproterenol) and defined pathways that were perturbed under baseline conditions and pathways that were normalized after propranolol treatment in the mdx model. The authors also undertook transcriptome analysis of human DMD left ventricle samples and found that DMD hiPSC-derived cardiomyocytes have dysregulated pathways similar to the human DMD heart. The authors further determined that relatively few patients with DMD see a cardiovascular specialist or receive ß-blocker therapy. CONCLUSIONS: The results highlight mechanisms and therapeutic interventions from human to animal and back to human in the dystrophic heart. These results may serve as a prelude for an adequately powered clinical study that examines the impact of ß-blocker therapy in patients with dystrophinopathies.

Generation of human endothelium in pig embryos deficient in ETV2.

The scarcity of donor organs may be addressed in the future by using pigs to grow humanized organs with lower potential for immunological rejection after transplantation in humans. Previous studies have demonstrated that interspecies complementation of rodent blastocysts lacking a developmental regulatory gene can generate xenogeneic pancreas and kidney1,2. However, such organs contain host endothelium, a source of immune rejection. We used gene editing and somatic cell nuclear transfer to engineer porcine embryos deficient in ETV2, a master regulator of hematoendothelial lineages3-7. ETV2-null pig embryos lacked hematoendothelial lineages and were embryonic lethal. Blastocyst complementation with wild-type porcine blastomeres generated viable chimeric embryos whose hematoendothelial cells were entirely donor-derived. ETV2-null blastocysts were injected with human induced pluripotent stem cells (hiPSCs) or hiPSCs overexpressing the antiapoptotic factor BCL2, transferred to synchronized gilts and analyzed between embryonic day 17 and embryonic day 18. In these embryos, all endothelial cells were of human origin.

ETV2-null porcine embryos survive to post-implantation following incomplete enucleation.

Blind enucleation is used in porcine somatic cell nuclear transfer (SCNT) to remove the metaphase II (MII) spindle from the oocyte. Deviation of the MII spindle location, however, leads to incomplete enucleation (IE). Here, we report that the rate of complete enucleation (CE) using the blind method was 80.2 ± 1.7%, although this significantly increased when the polar body-MII deviation was minimized (≦45°). While it is established that IE embryos will not survive to full term, the effect of IE on early stage development is unknown. We have previously demonstrated in mice and pigs that ETV2 deletion results in embryonic lethality due to the lack of hematoendothelial lineages. We observed that ETV2-null cloned embryos derived from blindly and incompletely enucleated oocytes had both WT and mutant sequences at E18 and, using FISH analysis, we observed triploidy. We also compared SCNT embryos generated from either CE or intentionally IE oocytes using the spindle viewer system. We observed a higher in vitro blastocyst rate in the IE versus the CE-SCNT embryos (31.9 ± 3.2% vs 21.0 ± 2.1%). Based on known processes in normal fertilization, we infer that the IE-SCNT embryos extruded the haploid second PB after fusion with donor fibroblasts and formed a near-triploid aneuploid nucleus in each blastomere. These studies demonstrate the peri-implantation survival of residual haploid nuclei following IE and emphasize the need for complete enucleation especially for the analysis of SCNT embryos in the peri-implantation stage and will, further, impact the field of reverse xenotransplantation.

Etv2 transcriptionally regulates Yes1 and promotes cell proliferation during embryogenesis.

Etv2, an Ets-transcription factor, governs the specification of the earliest hemato-endothelial progenitors during embryogenesis. While the transcriptional networks during hemato-endothelial development have been well described, the mechanistic details are incompletely defined. In the present study, we described a new role for Etv2 as a regulator of cellular proliferation via Yes1 in mesodermal lineages. Analysis of an Etv2-ChIPseq dataset revealed significant enrichment of Etv2 peaks in the upstream regions of cell cycle regulatory genes relative to non-cell cycle genes. Our bulk-RNAseq analysis using the doxycycline-inducible Etv2 ES/EB system showed increased levels of cell cycle genes including E2f4 and Ccne1 as early as 6 h following Etv2 induction. Further, EdU-incorporation studies demonstrated that the induction of Etv2 resulted in a ~2.5-fold increase in cellular proliferation, supporting a proliferative role for Etv2 during differentiation. Next, we identified Yes1 as the top-ranked candidate that was expressed in Etv2-EYFP+ cells at E7.75 and E8.25 using single cell RNA-seq analysis. Doxycycline-mediated induction of Etv2 led to an increase in Yes1 transcripts in a dose-dependent fashion. In contrast, the level of Yes1 was reduced in Etv2 null embryoid bodies. Using bioinformatics algorithms, biochemical, and molecular biology techniques, we show that Etv2 binds to the promoter region of Yes1 and functions as a direct upstream transcriptional regulator of Yes1 during embryogenesis. These studies enhance our understanding of the mechanisms whereby Etv2 governs mesodermal fate decisions early during embryogenesis.

Sall4 regulates neuromesodermal progenitors and their descendants during body elongation in mouse embryos.

Bi-potential neuromesodermal progenitors (NMPs) produce both neural and paraxial mesodermal progenitors in the trunk and tail during vertebrate body elongation. We show that Sall4, a pluripotency-related transcription factor gene, has multiple roles in regulating NMPs and their descendants in post-gastrulation mouse embryos. Sall4 deletion using TCre caused body/tail truncation, reminiscent of early depletion of NMPs, suggesting a role of Sall4 in NMP maintenance. This phenotype became significant at the time of the trunk-to-tail transition, suggesting that Sall4 maintenance of NMPs enables tail formation. Sall4 mutants exhibit expanded neural and reduced mesodermal tissues, indicating a role of Sall4 in NMP differentiation balance. Mechanistically, we show that Sall4 promotion of WNT/ß-catenin signaling contributes to NMP maintenance and differentiation balance. RNA-Seq and SALL4 ChIP-Seq analyses support the notion that Sall4 regulates both mesodermal and neural development. Furthermore, in the mesodermal compartment, genes regulating presomitic mesoderm differentiation are downregulated in Sall4 mutants. In the neural compartment, we show that differentiation of NMPs towards post-mitotic neuron is accelerated in Sall4 mutants. Our results collectively provide evidence supporting the role of Sall4 in regulating NMPs and their descendants.