ABSTRACT
In this paper, we present Vision Permutator, a conceptually simple and data efficient MLP-like architecture for visual recognition. By realizing the importance of the positional information carried by 2D feature representations, unlike recent MLP-like models that encode the spatial information along the flattened spatial dimensions, Vision Permutator separately encodes the feature representations along the height and width dimensions with linear projections. This allows Vision Permutator to capture long-range dependencies and meanwhile avoid the attention building process in transformers. The outputs are then aggregated in a mutually complementing manner to form expressive representations. We show that our Vision Permutators are formidable competitors to convolutional neural networks (CNNs) and vision transformers. Without the dependence on spatial convolutions or attention mechanisms, Vision Permutator achieves 81.5% top-1 accuracy on ImageNet without extra large-scale training data (e.g., ImageNet-22k) using only 25M learnable parameters, which is much better than most CNNs and vision transformers under the same model size constraint. When scaling up to 88M, it attains 83.2% top-1 accuracy, greatly improving the performance of recent state-of-the-art MLP-like networks for visual recognition. We hope this work could encourage research on rethinking the way of encoding spatial information and facilitate the development of MLP-like models. PyTorch/MindSpore/Jittor code is available at https://github.com/Andrew-Qibin/VisionPermutator.
ABSTRACT
Recently, Vision Transformers (ViTs) have been broadly explored in visual recognition. With low efficiency in encoding fine-level features, the performance of ViTs is still inferior to the state-of-the-art CNNs when trained from scratch on a midsize dataset like ImageNet. Through experimental analysis, we find it is because of two reasons: 1) the simple tokenization of input images fails to model the important local structure such as edges and lines, leading to low training sample efficiency; 2) the redundant attention backbone design of ViTs leads to limited feature richness for fixed computation budgets and limited training samples. To overcome such limitations, we present a new simple and generic architecture, termed Vision Outlooker (VOLO), which implements a novel outlook attention operation that dynamically conduct the local feature aggregation mechanism in a sliding window manner across the input image. Unlike self-attention that focuses on modeling global dependencies of local features at a coarse level, our outlook attention targets at encoding finer-level features, which is critical for recognition but ignored by self-attention. Outlook attention breaks the bottleneck of self-attention whose computation cost scales quadratically with the input spatial dimension, and thus is much more memory efficient. Compared to our Tokens-To-Token Vision Transformer (T2T-ViT), VOLO can more efficiently encode fine-level features that are essential for high-performance visual recognition. Experiments show that with only 26.6 M learnable parameters, VOLO achieves 84.2% top-1 accuracy on ImageNet-1 K without using extra training data, 2.7% better than T2T-ViT with a comparable number of parameters. When the model size is scaled up to 296 M parameters, its performance can be further improved to 87.1%, setting a new record for ImageNet-1 K classification. In addition, we also take the proposed VOLO as pretrained models and report superior performance on downstream tasks, such as semantic segmentation. Code is available at https://github.com/sail-sg/volo.
ABSTRACT
γ-Cyclodextrin glycosyltransferase (γ-CGTase, EC 2.4.1.19) is an essential enzyme required in the production of γ-cyclodextrin, which shows huge prospects in the food, medicine, materials, and chemical industries. In this study, γ-CGTase from Bacillus sp. G-825-6 STB17 was successfully cloned and expressed in Bacillus subtilis WB600. The final extracellular activity of γ-CGTase can reach 45.34 U/mL with the deletion of the signal peptide, which was about 11.3 times of the initial level of γ-CGTase secreted by the general pathway. By monitoring the whole cultivation process, secretion was divided into two stages, which were dominated by cell membrane changes and apoptosis. The measurement of lactate dehydrogenase and the results of fluorescence microscopy demonstrated that the cell membrane permeability changed significantly in the middle stage of fermentation, proving that it played a crucial role in the non-classical secretion of γ-CGTase. Furthermore, the addition of Triton X-100, a non-ionic surfactant, remarkably enhanced the secretion level of γ-CGTase by 21.5%, which was caused by the increase in cell membrane permeability. This work is the first to obtain the extracellular expression of CGTase via the non-classical secretion pathway in B. subtilis and provides a new strategy to enhance extracellular expression by regulating the cell membranes.
