Compact Model Training by Low-Rank Projection With Energy Transfer.

Low-rankness plays an important role in traditional machine learning but is not so popular in deep learning. Most previous low-rank network compression methods compress networks by approximating pretrained models and retraining. However, the optimal solution in the Euclidean space may be quite different from the one with low-rank constraint. A well-pretrained model is not a good initialization for the model with low-rank constraints. Thus, the performance of a low-rank compressed network degrades significantly. Compared with other network compression methods such as pruning, low-rank methods attract less attention in recent years. In this article, we devise a new training method, low-rank projection with energy transfer (LRPET), that trains low-rank compressed networks from scratch and achieves competitive performance. We propose to alternately perform stochastic gradient descent training and projection of each weight matrix onto the corresponding low-rank manifold. Compared to retraining on the compact model, this enables full utilization of model capacity since solution space is relaxed back to Euclidean space after projection. The matrix energy (the sum of squares of singular values) reduction caused by projection is compensated by energy transfer. We uniformly transfer the energy of the pruned singular values to the remaining ones. We theoretically show that energy transfer eases the trend of gradient vanishing caused by projection. In modern networks, a batch normalization (BN) layer can be merged into the previous convolution layer for inference, thereby influencing the optimal low-rank approximation (LRA) of the previous layer. We propose BN rectification to cut off its effect on the optimal LRA, which further improves the performance. Comprehensive experiments on CIFAR-10 and ImageNet have justified that our method is superior to other low-rank compression methods and also outperforms recent state-of-the-art pruning methods. For object detection and semantic segmentation, our method still achieves good compression results. In addition, we combine LRPET with quantization and hashing methods and achieve even better compression than the original single method. We further apply it in Transformer-based models to demonstrate its transferability. Our code is available at https://github.com/BZQLin/LRPET.

Development and expansion of the CRISPR/Cas9 toolboxes for powerful genome engineering in yeast.

Yeasts represent a group of the microorganisms most frequently seen in biotechnology. Recently, the class 2 type II CRISPR system (CRISPR/Cas9) has become the principal toolbox for genome editing. By efficiently implementing genetic manipulations such as gene integration/knockout, base editor, and transcription regulation, the development of biotechnological applications in yeasts has been extensively promoted. The genome-level tools based on CRISPR/Cas9, used for screening and identifying functional genes/gene clusters, are also advancing. In general, CRISPR/Cas9-assisted editing tools have gradually become standardized and function as host-orthogonal genetic systems, which results in time-saving for strain engineering and biotechnological application processes. In this review, we summarize the key points of the basic elements in the CRISPR/Cas9 system, including Cas9 variants, guide RNA, donors, and effectors. With a focus on yeast, we have also introduced the development of various CRISPR/Cas9 systems and discussed their future possibilities.

Characterization of cross-species transcription and splicing from Penicillium to Saccharomyces cerevisiae.

Heterologous expression of eukaryotic gene clusters in yeast has been widely used for producing high-value chemicals and bioactive secondary metabolites. However, eukaryotic transcription cis-elements are still undercharacterized, and the cross-species expression mechanism remains poorly understood. Here we used the whole expression unit (including original promoter, terminator, and open reading frame with introns) of orotidine 5'-monophosphate decarboxylases from 14 Penicillium species as a showcase, and analyzed their cross-species expression in Saccharomyces cerevisiae. We found that pyrG promoters from the Penicillium species could drive URA3 expression in yeast, and that inefficient cross-species splicing of Penicillium introns might result in weak cross-species expression. Thus, this study demonstrates cross-species expression from Penicillium to yeast, and sheds light on the opportunities and challenges of cross-species expression of fungi expression units and gene clusters in yeast without refactoring for novel natural product discovery.

Expression of fungal biosynthetic gene clusters in S. cerevisiae for natural product discovery.

Fungi are well known for production of antibiotics and other bioactive secondary metabolites, that can be served as pharmaceuticals, therapeutic agents and industrially useful compounds. However, compared with the characterization of prokaryotic biosynthetic gene clusters (BGCs), less attention has been paid to evaluate fungal BGCs. This is partially because heterologous expression of eukaryotic gene constructs often requires replacement of original promoters and terminators, as well as removal of intron sequences, and this substantially slow down the workflow in natural product discovery. It is therefore of interest to investigate the possibility and effectiveness of heterologous expression and library screening of intact BGCs without refactoring in industrial friendly microbial cell factories, such as the yeast Saccharomyces cerevisiae. Here, we discuss the importance of developing new research directions on library screening of fungal BGCs in yeast without refactoring, followed by outlooking prominent opportunities and challenges for future advancement.

Bioprospecting Through Cloning of Whole Natural Product Biosynthetic Gene Clusters.

Since the discovery of penicillin, natural products and their derivatives have been a valuable resource for drug discovery. With recent development of genome mining approaches in the post-genome era, a great number of natural product biosynthetic gene clusters (BGCs) have been identified and these can potentially be exploited for the discovery of novel natural products that can find application as pharmaceuticals. Since many BGCs are silent or do not express in native hosts under laboratory conditions, heterologous expression of BGCs in genetically tractable hosts becomes an attractive route to activate these BGCs to discover the corresponding products. Here, we highlight recent achievements in cloning and discovery of natural product biosynthetic pathways via intact BGC capturing, and discuss the prospects of high-throughput and multiplexed cloning of rational-designed gene clusters in the future.

Metabolic engineering of Escherichia coli for poly(3-hydroxybutyrate) production via threonine bypass.

BACKGROUND: Poly(3-hydroxybutyrate) (PHB), have been considered to be good candidates for completely biodegradable polymers due to their similar mechanical properties to petroleum-derived polymers and complete biodegradability. Escherichia coli has been used to simulate the distribution of metabolic fluxes in recombinant E. coli producing poly(3-hydroxybutyrate) (PHB). Genome-scale metabolic network analysis can reveal unexpected metabolic engineering strategies to improve the production of biochemicals and biofuels. RESULTS: In this study, we reported the discovery of a new pathway called threonine bypass by flux balance analysis of the genome-scale metabolic model of E. coli. This pathway, mainly containing the reactions for threonine synthesis and degradation, can potentially increase the yield of PHB and other acetyl-CoA derived products by reutilizing the CO2 released at the pyruvate dehydrogenase step. To implement the threonine bypass for PHB production in E. coli, we deregulated the threonine and serine degradation pathway and enhanced the threonine synthesis, resulting in 2.23-fold improvement of PHB titer. Then, we overexpressed glyA to enhance the conversion of glycine to serine and activated transhydrogenase to generate NADPH required in the threonine bypass. CONCLUSIONS: The result strain TB17 (pBHR68) produced 6.82 g/L PHB with the yield of 0.36 g/g glucose in the shake flask fermentation and 35.92 g/L PHB with the yield of 0.23 g/g glucose in the fed-batch fermentation, which was almost 3.3-fold higher than the parent strain. The work outlined here shows that genome-scale metabolic network analysis can reveal novel metabolic engineering strategies for developing efficient microbial cell factories.

Metabolic engineering of Escherichia coli using CRISPR-Cas9 meditated genome editing.

Engineering cellular metabolism for improved production of valuable chemicals requires extensive modulation of bacterial genome to explore complex genetic spaces. Here, we report the development of a CRISPR-Cas9 based method for iterative genome editing and metabolic engineering of Escherichia coli. This system enables us to introduce various types of genomic modifications with near 100% editing efficiency and to introduce three mutations simultaneously. We also found that cells with intact mismatch repair system had reduced chance to escape CRISPR mediated cleavage and yielded increased editing efficiency. To demonstrate its potential, we used our method to integrate the ß-carotene synthetic pathway into the genome and to optimize the methylerythritol-phosphate (MEP) pathway and central metabolic pathways for ß-carotene overproduction. We collectively tested 33 genomic modifications and constructed more than 100 genetic variants for combinatorially exploring the metabolic landscape. Our best producer contained15 targeted mutations and produced 2.0 g/L ß-carotene in fed-batch fermentation.

Engineering of Serine-Deamination pathway, Entner-Doudoroff pathway and pyruvate dehydrogenase complex to improve poly(3-hydroxybutyrate) production in Escherichia coli.

BACKGROUND: Poly(3-hydroxybutyrate) (PHB), a biodegradable bio-plastic, is one of the most common homopolymer of polyhydroxyalkanoates (PHAs). PHB is synthesized by a variety of microorganisms as intracellular carbon and energy storage compounds in response to environmental stresses. Bio-based production of PHB from renewable feedstock is a promising and sustainable alternative to the petroleum-based chemical synthesis of plastics. In this study, a novel strategy was applied to improve the PHB biosynthesis from different carbon sources. RESULTS: In this research, we have constructed E. coli strains to produce PHB by engineering the Serine-Deamination (SD) pathway, the Entner-Doudoroff (ED) pathway, and the pyruvate dehydrogenase (PDH) complex. Firstly, co-overexpression of sdaA (encodes L-serine deaminase), L-serine biosynthesis genes and pgk (encodes phosphoglycerate kinase) activated the SD Pathway, and the resulting strain SD02 (pBHR68), harboring the PHB biosynthesis genes from Ralstonia eutropha, produced 4.86 g/L PHB using glucose as the sole carbon source, representing a 2.34-fold increase compared to the reference strain. In addition, activating the ED pathway together with overexpressing the PDH complex further increased the PHB production to 5.54 g/L with content of 81.1% CDW. The intracellular acetyl-CoA concentration and the [NADPH]/[NADP(+)] ratio were enhanced after the modification of SD pathway, ED pathway and the PDH complex. Meanwhile, these engineering strains also had a significant increase in PHB concentration and content when xylose or glycerol was used as carbon source. CONCLUSIONS: Significant levels of PHB biosynthesis from different kinds of carbon sources can be achieved by engineering the Serine-Deamination pathway, Entner-Doudoroff pathway and pyruvate dehydrogenase complex in E. coli JM109 harboring the PHB biosynthesis genes from Ralstonia eutropha. This work demonstrates a novel strategy for improving PHB production in E. coli. The strategy reported here should be useful for the bio-based production of PHB from renewable resources.

Improved poly(3-hydroxybutyrate) production in Escherichia coli by inactivation of cytochrome bd-II oxidase or/and NDH-II dehydrogenase in low efficient respiratory chains.

In order to redirect more carbon flux from TCA cycle into poly(3-hydroxybutyrate) (PHB) biosynthesis pathway via increasing respiratory efficiency, appB and ndh genes encoding cytochrome bd-II oxidase and NDH-II dehydrogenase were inactivated in Escherichia coli JM109/pBHR68. All appB or/and ndh knockout strains exhibited significantly increased PHB accumulation accompanying with increased NAD(P)H/NAD(P)(+) ratio and intracellular acetyl-CoA pool. Among them, the Δndh strain could accumulate up to 6.16g/L PHB from 20g/L glucose and 3.5g/L PHB from 20g/L xylose, respectively, a 1.76-fold and 3.43-fold increase compared to the wild-type control. The PHB production of this strain reached 28.23g/L in a 5-L fermentor study, which was 2.70-fold as much as that of the wild-type control. These results indicated that inactivating the cytochrome bd-II oxidase or/and NDH-II dehydrogenase of the aerobic respiratory chain is a simple and effective strategy to improve PHB biosynthesis in E. coli. To date, this is the first time to improve PHB production by inactivation of cytochrome bd-II oxidase or/and NDH-II dehydrogenase in low efficient respiratory chains.

Metabolic engineering of Escherichia coli for the production of riboflavin.

BACKGROUND: Riboflavin (vitamin B2), the precursor of the flavin cofactors flavin mononucleotide (FMN) and flavin adenine dinucleotide (FAD), is used commercially as an animal feed supplement and food colorant. E. coli is a robust host for various genetic manipulations and has been employed for efficient production of biofuels, polymers, amino acids, and bulk chemicals. Thus, the aim of this study was to understand the metabolic capacity of E. coli for the riboflavin production by modification of central metabolism, riboflavin biosynthesis pathway and optimization of the fermentation conditions. RESULTS: The basic producer RF01S, in which the riboflavin biosynthesis genes ribABDEC from E. coli were overexpressed under the control of the inducible trc promoter, could accumulate 229.1 mg/L of riboflavin. Further engineering was performed by examining the impact of expression of zwf (encodes glucose 6-phosphate dehydrogenase) and gnd (encodes 6-phosphogluconate dehydrogenase) from Corynebacterium glutamicum and pgl (encodes 6-phosphogluconolactonase) from E. coli on riboflavin production. Deleting pgi (encodes glucose-6-phosphate isomerase) and genes of Entner-Doudoroff (ED) pathway successfully redirected the carbon flux into the oxidative pentose phosphate pathway, and overexpressing the acs (encodes acetyl-CoA synthetase) reduced the acetate accumulation. These modifications increased riboflavin production to 585.2 mg/L. By further modulating the expression of ribF (encodes riboflavin kinase) for reducing the conversion of riboflavin to FMN in RF05S, the final engineering strain RF05S-M40 could produce 1036.1 mg/L riboflavin in LB medium at 37°C. After optimizing the fermentation conditions, strain RF05S-M40 produced 2702.8 mg/L riboflavin in the optimized semi-defined medium, which was a value nearly 12-fold higher than that of RF01S, with a yield of 137.5 mg riboflavin/g glucose. CONCLUSIONS: The engineered strain RF05S-M40 has the highest yield among all reported riboflavin production strains in shake flask culture. This work collectively demonstrates that E. coli has a potential to be a microbial cell factory for riboflavin bioproduction.

Multiplex iterative plasmid engineering for combinatorial optimization of metabolic pathways and diversification of protein coding sequences.

Engineering complex biological systems typically requires combinatorial optimization to achieve the desired functionality. Here, we present Multiplex Iterative Plasmid Engineering (MIPE), which is a highly efficient and customized method for combinatorial diversification of plasmid sequences. MIPE exploits ssDNA mediated λ Red recombineering for the introduction of mutations, allowing it to target several sites simultaneously and generate libraries of up to 10(7) sequences in one reaction. We also describe "restriction digestion mediated co-selection (RD CoS)", which enables MIPE to produce enhanced recombineering efficiencies with greatly simplified coselection procedures. To demonstrate this approach, we applied MIPE to fine-tune gene expression level in the 5-gene riboflavin biosynthetic pathway and successfully isolated a clone with 2.67-fold improved production in less than a week. We further demonstrated the ability of MIPE for highly multiplexed diversification of protein coding sequence by simultaneously targeting 23 codons scattered along the 750 bp sequence. We anticipate this method to benefit the optimization of diverse biological systems in synthetic biology and metabolic engineering.

Cooperative behavior in evolutionary snowdrift games with the unconditional imitation rule on regular lattices.

We study an evolutionary snowdrift game with the unconditional imitation updating rule on regular lattices. Detailed numerical simulations establish the structure of plateaus and discontinuous jumps of the equilibrium cooperation frequency f(c) as a function of the cost-to-benefit ratio r. By analyzing the stability of local configurations, it is found that the transitions occur at values of r at which there are changes in the ranking of the payoffs to the different local configurations of agents using different strategies. Nonmonotonic behavior of f(c)(r) at the intermediate range of r is analyzed in terms of the formation of blocks of agents using the cooperative strategy that are stabilized by agents inside the block due to the updating rule. For random initial condition with 50%-50% agents of different strategies randomly dispersed, cooperation persists in the whole range of r and the level of cooperation is higher than that in the well-mixed case in a wide range of r. These results are in sharp contrast to those based on the replicator updating rule. The sensitivity to initial states with different fractions of cooperative agents is also discussed. The results serve to illustrate that both the spatial structure and the updating rule are important in determining the level of cooperation in a competing population. When extreme initial states are used where there are very few agents of a strategy in a background of the opposite strategy, the result would depend on the stability of the clusters formed by the initially minority agents.

Competition between the catalyzed birth and death in the exchange-driven growth.

We propose a three-species ( A , B , and C ) exchange-driven aggregate growth model with competition between catalyzed birth and catalyzed death. In the system, exchange-driven aggregation occurs between any two aggregates of the same species with the size-dependent rate kernel Kn(k,j)=Knkj (n=1,2,3) , and, meanwhile, monomer birth and death of species A occur under the catalysis of species B and C with the catalyzed birth and catalyzed death rate kernels I(k,j)=Ikjv and J(k,j)=Jkjv , respectively. The kinetic behavior is investigated by means of the mean-field rate equation approach. The form of the aggregate size distribution ak(t) of species A is found to depend crucially on the competition between species- B -catalyzed birth of species A and species- C -catalyzed death of species A , as well as the exchange-driven growth. The results show that (i) when exchange-driven aggregation dominates the process, ak(t) satisfies the conventional scaling form; (ii) when catalyzed birth dominates the process, ak(t) takes the conventional or generalized scaling form; and (iii) when catalyzed death dominates the process, the aggregate size distribution of species A evolves only according to some modified scaling forms.

Mutually catalyzed birth of population and assets in exchange-driven growth.

We propose an exchange-driven aggregation growth model of population and assets with mutually catalyzed birth to study the interaction between the population and assets in their exchange-driven processes. In this model, monomer (or equivalently, individual) exchange occurs between any pair of aggregates of the same species (population or assets). The rate kernels of the exchanges of population and assets are K(k,l) = Kkl and L(k,l) = Lkl , respectively, at which one monomer migrates from an aggregate of size k to another of size l. Meanwhile, an aggregate of one species can yield a new monomer by the catalysis of an arbitrary aggregate of the other species. The rate kernel of asset-catalyzed population birth is I(k,l) = Iklmu [and that of population-catalyzed asset birth is J(k,l) = Jklnu], at which an aggregate of size k gains a monomer birth when it meets a catalyst aggregate of size l . The kinetic behaviors of the population and asset aggregates are solved based on the rate equations. The evolution of the aggregate size distributions of population and assets is found to fall into one of three categories for different parameters mu and nu: (i) population (asset) aggregates evolve according to the conventional scaling form in the case of mu < or = 0 (nu < or = 0), (ii) population (asset) aggregates evolve according to a modified scaling form in the case of nu = 0 and mu > 0 (mu = 0 and nu > 0 ), and (iii) both population and asset aggregates undergo gelation transitions at a finite time in the case of mu = nu > 0.

Experimental investigation of the scaling of columnar joints.

Columnar jointing is a fracture pattern common in igneous rocks in which cracks self-organize into a roughly hexagonal arrangement, leaving behind an ordered colonnade. We report observations of columnar jointing in a laboratory analog system, desiccated corn starch slurries. Using measurements of moisture density, evaporation rates, and fracture advance rates as evidence, we suggest that an advective-diffusive system is responsible for the rough scaling behavior of columnar joints. This theory explains the order of magnitude difference in scales between jointing in lavas and in starches. We investigated the scaling of average columnar cross-sectional areas due to the evaporation rate, the analog of the cooling rate of igneous columnar joints. We measured column areas in experiments where the evaporation rate depended on lamp height and time, in experiments where the evaporation rate was fixed using feedback methods, and in experiments where gelatin was added to vary the rheology of the starch. Our results suggest that the column area at a particular depth is related to both the current conditions, and hysteretically to the geometry of the pattern at previous depths. We argue that there exists a range of stable column scales allowed for any particular evaporation rate.

Migration-driven aggregate growth on scale-free networks.

We study the kinetics of migration-driven aggregate growth on completely connected scale-free networks. A reversible migration system is considered with the size-dependent rate kernel K(k; l/i;j) approximately k(u)i(v)(lj)(v), at which an i-mer aggregate located on the node with j links gains one monomer from a k-mer aggregate on the node with l links. The results show that the evolution behavior of the aggregate size distribution is drastically different from that for the corresponding same system in normal space. This model can be used to mimic some phenomena such as the distribution of city populations. Moreover, we verify our analytic results in good agreement with the data of the population distributions of all U.S. counties.

Kinetics of migration-driven aggregation processes on scale-free networks.

We propose a solvable model for the migration-driven aggregate growth on completely connected scale-free networks. A reversible migration system is considered with the produce rate kernel K(k;l|i;j) approximately k(u)i(upsilon)(lj)(nu) or the generalized kernel K(k;l|i;j) approximately (k(upsilon)i(omega)+k(omega)i(upsilon)(lj)(nu), at which an i-mer aggregate locating on the node with j links gains one monomer from a k-mer aggregate locating on the node with l links. It is found that the evolution behavior of the system depends crucially on the details of the rate kernel. In some cases, the aggregate size distribution approaches a scaling form and the typical size S(t,l) of the aggregates locating on the nodes with l links grows infinitely with time; while in other cases, a gelation transition may emerge in the system at a finite critical time. We also introduce a simplified model, in which the aggregates independently gain or lose one monomer at the rate I(1)(k;l)=I(2)(k;l) proportional to k(omega)l(nu), and find the similar results. Most intriguingly, these models exhibit that the evolution behavior of the total distribution of the aggregates with the same size is drastically different from that for the corresponding system in normal space. We test our analytical results with the population data of all counties in the U.S. during the past century and find good agreement between the theoretical predictions and the realistic data.

Kinetics of a migration-driven aggregation process with birth and death.

We propose an irreversible aggregation model driven by migration and birth-death processes with the symmetric migration rate kernel K(k;j)=K'(k;j)=Ikj(upsilon), and the birth rate J(1)k and death rate J(2)k proportional to the aggregate's size k. Based on the mean-field theory, we investigate the evolution behavior of the system through developing the scaling theory. The total mass M1 is reserved in the J(1)=J(2) case and increases exponentially with time in the J1>J2 case. In these cases, the long-time asymptotic behavior of the aggregate size distribution a(k)(t) always obeys the scaling law for the upsilon

Solvable aggregation model with monomer annihilation.

We propose a simple model in which irreversible aggregations occur between any two clusters of the same species and monomer annihilations occur between any two clusters of different species. We investigate the mean-field rate equation to analyze kinetics of the system under symmetrical initial conditions. In the constant-reaction-rate case, the cluster-mass distribution of either species approaches a conventional scaling form and both species survive finally; while for the system with a fast rate kernel, both species scale according to a modified form and no species can survive at the end.

Solvable n-species aggregation processes with joint annihilation.

We study the kinetic behavior of the aggregation-annihilation processes of an n-species (n> or =3) system, in which an irreversible aggregation reaction occurs between any two clusters of the same species and an irreversible complete annihilation reaction occurs only between one certain A(n) species and each of the other A(m) species (m=1,2,...n-1). Based on the mean-field theory, we investigate the rate equations of the processes to obtain the asymptotic solutions of the cluster-mass distributions in several different cases. The results show that the evolution behavior of the system depends crucially on the ratios of the equivalent aggregation rate of A(m) species and the aggregation rate of A(n) species to the annihilation rate. The cluster-mass distribution of each species always obeys a conventional scaling law or a modified one, and the scaling exponents depend only on the reaction rates for most cases. However, when both the equivalent aggregation rate of A(m) species and the aggregation rate of A(n) species are twice as large as the annihilation rate, the scaling exponents depend on the reaction rates as well as the initial concentrations.