Adding Before Pruning: Sparse Filter Fusion for Deep Convolutional Neural Networks via Auxiliary Attention.

Filter pruning is a significant feature selection technique to shrink the existing feature fusion schemes (especially on convolution calculation and model size), which helps to develop more efficient feature fusion models while maintaining state-of-the-art performance. In addition, it reduces the storage and computation requirements of deep neural networks (DNNs) and accelerates the inference process dramatically. Existing methods mainly rely on manual constraints such as normalization to select the filters. A typical pipeline comprises two stages: first pruning the original neural network and then fine-tuning the pruned model. However, choosing a manual criterion can be somehow tricky and stochastic. Moreover, directly regularizing and modifying filters in the pipeline suffer from being sensitive to the choice of hyperparameters, thus making the pruning procedure less robust. To address these challenges, we propose to handle the filter pruning issue through one stage: using an attention-based architecture that adaptively fuses the filter selection with filter learning in a unified network. Specifically, we present a pruning method named adding before pruning (ABP) to make the model focus on the filters of higher significance by training instead of man-made criteria such as norm, rank, etc. First, we add an auxiliary attention layer into the original model and set the significance scores in this layer to be binary. Furthermore, to propagate the gradients in the auxiliary attention layer, we design a specific gradient estimator and prove its effectiveness for convergence in the graph flow through mathematical derivation. In the end, to relieve the dependence on the complicated prior knowledge for designing the thresholding criterion, we simultaneously prune and train the filters to automatically eliminate network redundancy with recoverability. Extensive experimental results on the two typical image classification benchmarks, CIFAR-10 and ILSVRC-2012, illustrate that the proposed approach performs favorably against previous state-of-the-art filter pruning algorithms.

A Learning Framework for n-Bit Quantized Neural Networks Toward FPGAs.

The quantized neural network (QNN) is an efficient approach for network compression and can be widely used in the implementation of field-programmable gate arrays (FPGAs). This article proposes a novel learning framework for n -bit QNNs, whose weights are constrained to the power of two. To solve the gradient vanishing problem, we propose a reconstructed gradient function for QNNs in the back-propagation algorithm that can directly get the real gradient rather than estimating an approximate gradient of the expected loss. We also propose a novel QNN structure named n -BQ-NN, which uses shift operation to replace the multiply operation and is more suitable for the inference on FPGAs. Furthermore, we also design a shift vector processing element (SVPE) array to replace all 16-bit multiplications with SHIFT operations in convolution operation on FPGAs. We also carry out comparable experiments to evaluate our framework. The experimental results show that the quantized models of ResNet, DenseNet, and AlexNet through our learning framework can achieve almost the same accuracies with the original full-precision models. Moreover, when using our learning framework to train our n -BQ-NN from scratch, it can achieve state-of-the-art results compared with typical low-precision QNNs. Experiments on Xilinx ZCU102 platform show that our n -BQ-NN with our SVPE can execute 2.9 times faster than that with the vector processing element (VPE) in inference. As the SHIFT operation in our SVPE array will not consume digital signal processing (DSP) resources on FPGAs, the experiments have shown that the use of SVPE array also reduces average energy consumption to 68.7% of the VPE array with 16 bit.

Development and characterization of an enterovirus 71 (EV71) virus-like particles (VLPs) vaccine produced in Pichia pastoris.

Enterovirus 71 (EV71) is one of the major causative agents for hand, foot and mouth disease (HFMD) in children. Although there are three inactivated virus-based HFMD vaccines licensed in China, alternative approaches have been taken to produce an effective and safer vaccine that is easier to manufacture in large scale. Among these, a virus-like particles (VLPs) based EV71 vaccine is under active development. For this purpose, an efficient methodology for the production of EV71-VLPs by recombinant technology is needed. We here report the construction and expression of the P1 and 3C genes of EV71 in Pichia pastoris for producing VLP-based EV71 vaccine antigen with a high yield and simple manufacturing process. Based on codon-optimized P1 and 3C genes, EV71-VLPs were efficiently expressed in Pichia pastoris system, and the expression level reached 270 mg/L. Biochemical and biophysical analyses showed that the produced EV71-VLPs consisted of processed VP0, VP1, and VP3 present as ~35nm spherical particles. The immune response as a function of EV71-VLPs and adjuvant dose ratio was investigated for vaccine development. Immunization with EV71-VLPs of 1-5 µg/dose and adjuvant of 225 µg/dose induced robust neutralizing antibody responses in mice and provided effective protection against lethal challenge in both maternally transferred antibody and passive transfer protection mouse models. Therefore, the yeast produced EV71-VLPs antigen is a promising candidate for the development of a vaccine against HFMD.

The protective efficacy against Schistosoma japonicum infection by immunization with DNA vaccine and levamisole as adjuvant in mice.

Levamisole (LMS) as an adjuvant enhances cell-mediated immunity in DNA vaccination; we investigated the efficacy and liver immunopathology alleviation of a DNA vaccine, VR1012-SjGST-32, in a LMS formulation in the murine challenge model. Compared to controls, the VR1012-SjGST-32 plus LMS can reduce worm and egg burdens, as well as, immunopathological complications associated chronic inflammation significantly in liver, which were apparently associated with Th1-type response. Together, these results suggest that the LMS as a potential Schistosome DNA vaccine adjuvant can enhance both worm killing and disease prevention, which is possibly mediated through the induction of a strong Th1-dominant environment in immunized mice.

Vaccination of goats with 31 kDa and 32 kDa Schistosoma japonicum antigens by DNA priming and protein boosting.

Two Schistosoma japonicum vaccine candidate antigens Sj 31 and Sj 32, which have shown particular promise to induce protective immunity in mice, were used to immunize goats by using a DNA priming-protein boosting strategy in present work. DNA vaccine formulations of the two antigens (VRSj31 and VRSj32) were produced and injected intramuscularly twice at a 2-week interval and then recombinant proteins (rSj31 and rSj32) together with Freund Complete Adjuvant (FCA) were used to boost the goats. The experiment was repeated in different batche cercariae. A strong anamnestic antibody response was induced after boost. A significant reduction of liver egg counts and miracidial hatching was showed in both experiments. Significant protections against challenge infection were elicited with 31.6% of percentage reduction for worm recovery in the second experiment and 20.9% in the first experiment, respectively.

Identification and characterization of peptides mimicking the epitopes of metalloprotease of Schistosoma japonicum.

In an attempt to isolate and characterize peptides mimicking epitopes of metalloprotease and explore their immunological protection against Schistosoma japonicum (S. japonicum), polyclonal anti-metalloprotease sera was prepared to screen a 12-mer random peptide library to isolate phages binding specially to antisera IgG. Then, phage ELISA, animal immunization, DNA sequencing, Western blotting and enzymatic activity neutralizing analysis were used to characterize the selected phage clones. All of ten randomly picked clones were shown to be positive. Five peptides of different amino acid sequences deduced from DNA sequences were obtained and two of them (peptides 2 and 3) could induce significant reduction (31.0% and 31.8%, respectively) in worm burden and high reduction (52.6% and 54.9%, respectively) in liver eggs per gram (LEPG), while, unexpectedly, others (peptides 1, 4 and 5) could not elicit enough protection against infection of S. japonicum. Peptides 2 and 3 could be recognized by S. japonicum infected mouse sera (IMS) and could elicit neutralizing Abs. The results show that peptides 2 and 3 are antigenic and immunogenic. They are true mimics of epitopes of metalloprotease and useful as novel vaccine candidates against S. japonicum.

[Boost effect of recombinant IL-4 on protection of Schistosoma japonicum cathepsin B DNA vaccine in mice against the parasite].

OBJECTIVE: To investigate the enhancement effect of IL-4 expression plasmid on cathepsin B DNA vaccine of Schistosoma japonicum (Sj) in mice. METHODS: The recombinant IL-4 plasmid constructed by cloning PCR amplified product of murine IL-4 gene into eukaryotic expression vector pcDNA3.1 was co-injected intramuscularly with Sj cathepsin B expression plasmid DNA to mice as the test group. The other three groups of mice were set up as control including IL-4 expression plasmid, Sj cathepsin B expression plasmid and two vacant vector plasmids. The expression of IL-4 and cathepsin B was visualized by immunohistochemistry. Challenge infection in mice was carried out 3 weeks after the last vaccination and immune protection was assessed by worm and egg reduction rates. RESULTS: The recombinant mIL-4 plasmid and cathepsin B DNA vaccine were expressed in muscular cells of the vaccinated mice. Immunization with cathepsin B DNA plus recombinant mIL-4 plasmid yielded a 43.2 % of worm reduction rate and a 76.6% of egg reduction rate, showing a significant difference (P<0.01, P<0.05) compared with that of cathepsin B DNA vaccine alone. CONCLUSION: As an adjuvant, IL-4 DNA can improve the protective effect of cathepsin B DNA vaccine in mice against S. japonicum infection.

[Cloning and characterization of new genes of Schistosoma japonicum].

OBJECTIVE: To clone and characterize new genes of Schistosoma japonicum, Sj, and to provide efficient vaccine candidates. METHODS: Sj adult cDNA library was screened with rabbit sera raised against male worm soluble antigen. The inserted cDNA fragments from the positively selected clones were amplified with PCR and further sequenced, as well as characterized through internet NCBI GenBank software. RESULTS: Eleven positive clones were obtained and two were verified by GenBank as new, including a novel gene designated as Sj-P8 (GenBank accession No. AF517843) and a new partial cDNA of Sj myosin (GenBank accession No. AY770506). The two new genes encoded a transmembrane protein of 75 amino acids and a myosin protein fragment of 212 amino acids respectively. CONCLUSION: The newly obtained genes may provide useful information for the research on Sj vaccine.

Phage displaying peptides mimic schistosoma antigenic epitopes selected by rat natural antibodies and protective immunity induced by their immunization in mice.

AIM: To obtain the short peptides mimic antigenic epitopes selected by rat natural antibodies to schistosomes, and to explore their immunoprotection against schistosomiasis in mice. METHODS: Adults worm antigens (AWA) were analyzed by sodium dodecyl sulfate-polyacrylamide gel electrophoresis (SDS-PAGE) and enzyme-linked transferred immunoblotting methods with normal SD rat sera (NRS). The killing effects on schistosomula with fresh and heat-inactivated sera from SD rats were observed. Then the purified IgG from sera of SD rats was used to biopan a phage random peptide library and 20 randomly selected positive clones were detected by ELISA and 2 of them were sequenced. Sixty female mice were immunized thrice with positive phage clones (0, 2nd), 4th wk). Each mouse was challenged with 40 cercariae, and all mice were killed 42 d after challenge. The worms and the liver eggs were counted. RESULTS: NRS could specifically react to the molecules of 75,000, 47,000, 34,500 and 23,000 of AWA. Sera from SD rats showed that the mortality rate of schistosomula was 76.2%, and when the sera were heat-inactivated in vitro, the mortality rate was decreased to 41.0% after being cultured for 48 h. The specific phages bound to IgG were enriched about 300-folds after three rounds of biopanning. Twenty clones were detected by ELISA, 19 of them bound to the specific IgG of rat sera. Immunization with these epitopes was carried out in mice. Compared with the control groups, the mixture of two mimic peptides could induce 34.9% (P=0.000) worm reduction and 67.6% (P=0.000) total liver egg reduction in mice. Two different mimic peptides could respectively induce 31.0% (P=0.001), 14.5% (P=0.074) worm reduction and 61.2% (P=0.000), 35.7% (P=0.000) total liver egg reduction. The specific antibody could be induced by immunization of the mimic peptides, and the antibody titer in immunized mice reached more than 1:6,400 as detected by ELISA. CONCLUSION: Specific peptides mimic antigenic molecules can be obtained by biopanning the phage random peptide library and a partially protective immunity against schistosome infection can be stimulated by these phage epitopes in mice.

[Mucosal immunization of recombinant Schistosoma japonicum ferritin].

OBJECTIVE: To subclone and express the gene encoding Schistosoma japonicum ferritin (SjFer) and study its immune protection against challenge infection in mice vaccinated intranasally. METHODS: The SjFer gene was amplified by PCR, and subcloned into the N-terminal of intein 2 of the pTWIN1 vector. The positive recombinant was screened by PCR, restriction enzyme digestion and sequence analysis. The positive recombinant was transformed into E. coli ER2566. The soluble recombinant fusion protein (rSjFer-intein 2) was expressed in E. coli by induction of low IPTG concentration under low temperature, and analyzed by SDS-PAGE and Western blotting. Mice were immunized intranasally with rSjFer, using chitosan as adjuvant. Two weeks after the third vaccination, challenge infection with S. japonicum cercariae was carried out. Worms and eggs collected from the livers of mice were counted at 42 days after the challenge. Levels of specific antibodies were detected by ELISA before infection. RESULTS: SjFer was successfully subcloned into pTWIN1 vector and expressed in E. coli. In mice vaccinated intranasally with rSjFer and adjuvant chitosan, the worm reduction rate was 35.51% and the reduction rate of eggs per gram liver tissue (LEPG) was 52.17%. As compared with the control groups, levels of IgG, IgA in sera and SIgA in saliva increased significantly. CONCLUSION: The expressed rSjFer can induce partial protective immunity against S. japonicum infection in mice when they were vaccinated intranasally, with chitosan as adjuvant.

Schistosoma japonicum: isolation and identification of peptides mimicking ferritin epitopes from phage display library.

In an attempt to isolate and identify the antigenic epitopes on ferritin of Schistosoma japonicum (SjFer) and to test their protective potentiality against Schistosoma japonicum (S.j), polyclonal antisera against SjFer was prepared to screen a 12-mer random peptide library. Three rounds of biopanning were performed and resulted in an enrichment. Six peptides selected randomly from the third elute were all found to be positive by evaluating the binding to anti-SjFer sera by ELISA and Western blotting. Three amino acid sequences were deduced from the six phage clones by sequencing. When they were used to vaccinate mice, the three peptides could induce significant reduction in adult worms (26.7%, 20.4%, and 25.9%) as well as in liver eggs per gram (LEPG) (40.0%, 38.2%, and 40.8%). This result showed that three mimotopes on SjFer were obtained and they could induce significant protective efficacy against S.j.

[Studies on fractionation and protective immunity of the membrane antigen from hepato-portal juveniles of Schistosoma japonicum].

OBJECTIVE: To explore the immunological characteristics of the membrane antigen from hepato-portal juveniles of Schistosoma japonicum and its protective immunity against S. japonicum (Sj) in mice. METHODS: Sodium dodecyl sulfate-polyacrylamide gel electrophoresis(SDS-PAGE) and enzyme-linked immune electro-transfer blot(EITB) methods were used to recognize the membrane antigens from hepato-portal schistosomula (SjHmAg) by infected rabbit sera (IRS) and normal rabbit sera (NRS). Kunming mice were immunized subcutaneously three times(0, 2, 4 weeks) with SjHmAg. Each mouse was challenged with 40 +/- 1 cercariae. Six weeks later the mice were killed, worms and liver eggs were counted. RESULTS: 7 major protein bands appeared on SDS-PAGE. IRS mainly reacted specifically with SjHmAg of 23, 33 and 63 kDa. Compared with the control groups, the reduction rate of worms and eggs per gram liver in the experimental group were 16.2% and 54.4%, respectively. CONCLUSION: Different protein components from SjHmAg are obtained using SDS-PAGE, and the antigen can induce a protective immunity against Sj in mice.

[Study on diagnosis of schistosomiasis by ELISA using periodate-treated soluble egg antigen].

OBJECTIVE: To improve SEA-ELISA, an immunodiagnostic assay for schistosomiasis. METHODS: Soluble egg antigen (SEA) of Schistosoma japonicum was treated with sodium periodate (SP) in order to oxidate its glycosylated epitopes. ELISA using the treated SEA was then performed to detect specific antibodies to SEA in the sera of schistosomiasis patients. RESULTS: Serum samples were tested by ELISA using SEA treated with sodium periodate (SP-SEA-ELISA), including 64 sera from cases with chronic schistosomiasis japonica, 119 sera from normal individuals in non-endemic area, 34 sera from patients with clonorchiasis, 33 sera of paragonimiasis cases and 36 sera from patients with cysticercosis. The results showed that its sensitivity (98.4%) was similar to that of the routine SEA-ELISA (100.0%) (P > 0.05) and the specificity is higher than that of the SEA-ELISA (P < 0.05). SP-SEA-ELISA showed a higher negative rate (89.0%) for sera of schistosomiasis patients 12 months post-treatment than that of the SEA-ELISA (42.1%). CONCLUSION: Use of SP-SEA can increase the specificity of ELISA, reduce cross-reactivity with serum samples from cases infected with other parasites and improve its value in evaluating therapeutic efficacy.

Molecular cloning, expression and vaccination of a novel gene Sj-MA of Schistosoma japonicum.

In order to obtain new gene and to develop the new vaccine candidate of immunoprotection against Schistosoma japonicum (Sj), Sj adult worm cDNA library was screened with anti-sera to soluble male adult worm antigen and resulted in the discovery of a novel gene designated as Sj-MA. Sequence analysis showed that Sj-MA as a complete cDNA contains one open reading frame. It was deduced to contain 249 amino acid residues and encode a 28.8 kD soluble protein with plenty of phosphorylation sites, supposing its action in signal transduction. Furthermore, Sj-MA cDNA was cloned into a prokaryotic expression vector pGEX-5X to construct the recombinant plasmid which was transformed and highly expressed in E. coli as a 54.8 kD glutathione-S-transferase (GST) fusion protein. The fusion protein rSj-MA/GST could be recognized with both anti-male adult worm sera and anti-GST sera in Western blotting. Mice vaccinated with the fusion protein revealed significant worm reduction rate of 34.29% (P<0.001), compared with the control groups. Taken together, the novel gene Sj-MA can be expressed in E. coli as a fusion protein that can elicit immunity against Schistosoma japonicum, suggesting its potential as a new vaccine candidate.

[Protective effect against Schistosoma japonicum of recombinant fusion protein SjGST-32 in mice].

OBJECTIVE: To explore the optimal immune doses of recombinant antigen rSjGST-32, with QuilA as adjuvant. METHODS: BALB/c mice were immunized with doses of 50, 100 and 200 micrograms rSjGST-32/mouse plus 50 micrograms QuilA adjuvant. QuilA and PBS were used as controls. Levels of specific antibodies were detected by ELISA. The mice were challenged 4 weeks after last vaccination. Worms and eggs collected from the livers of mice were counted 45 days after challenge. RESULTS: As compared with the control groups, the worm reduction rate in the 50, 100 and 200 micrograms experiment groups was 38.1%, 47.8% and 48.8%, respectively. The reduction rate of liver eggs per gram (LEPG) was 39.1%, 53.5% and 53.6%, respectively, and the reduction rate of the liver eggs per female (LEPF) was 22.5%, 22.8% and 21.8%, respectively. The specific antibody titers in sera reached 1:51,200, 1:102,400 and 1:102,400, respectively before challenge. CONCLUSION: The results show that for vaccinating BALB/c mice, the optimal antigen dose is 100 micrograms of recombinant rSjGST-32 plus QuilA adjuvant.

[Intranasal or intragastric vaccination of mice with recombinant Schistosoma japonicum ferritin induces immune protection against challenge infection].

OBJECTIVE: To test the immune protection against challenge infection in mice vaccinated intranasally or intragastrically with recombinant Schistosoma japonicum (S.j) Ferritin (rSjFer). METHODS: Mice were divided into 8 groups each with 10 mice. They were immunized intragastically with rSjFer, CTB, rSjFer + CTB and intranasally with rSjFer, CTB and rSjFer + CTB respectively. PBS was used intragastically or intranasally as control groups. The mice were challenged with 40 +/- 1 S. j cercariae per mouse 2 wk after the third vaccinization. Forty-five days later, mice were killed and perfused, and the adult worms and eggs were counted. Serum and fecal samples were obtained before the first immunization and the challenge infection. IgA and IgG in sera and sIgA in feces were detected by ELISA. RESULTS: The worm reduction rate was 3.98%, 3.77%, 25.57% in the intragastric vaccination groups and 7.59%, 4.50%, 33.35% in the intranasal vaccination groups respectively. The egg reduction rate was 3.76%, 2.46%, 34.75% and 4.40%, 0.06%, 60.10% respectively. CONCLUSION: This study showed that a significant immune protection against Schistosoma japonicum infection was induced by mucosal (intranasal and intragastic) vaccination with rSjFer.

Partial protection induced by phage library-selected peptides mimicking epitopes of Schistosoma japonicum.

OBJECTIVE: To obtain peptide mimicking epitopes of Schistosoma japonicum (S. japonicum) through screening of a phage peptide library and to test their potential for induction of protection. METHODS: S. japonicum infected sera from Microtus fortis (IMFS) and normal sera from Microtus fortis (NMFS) were used respectively to screen a 12-mers random peptide library by testing the reactivity of anti-S. japonicum serum with the phagotopes. After three rounds of biopanning, the pooled phages were used to immunize mice, after which challenge infection was performed. RESULTS: Of 12 randomly picked clones, 10 clones selected using IMFS and 7 clones selected using NMFS were shown to be antigenic. Significant reduction in adult worms (22.6%) and a high reduction (68.9%) in liver eggs were achieved following immunization with phages screened with IMFS. However, no protection was elicited by those selected with NMFS. CONCLUSION: The results show that the phagotopes are both antigenic and immunogenic, suggesting a potential use of phage displayed peptide as novel vaccines against S. japonicum.

[Shistosoma japonicum: dynamics of cytokine production induced by immunization of mice with recombinant GST-Sj32 protein before and after challenge infection].

AIM: To explore the mechanism of protective immunity of rGST-Sj32. METHODS: BALB/c mice were immunized subcutaneously with rGST-Sj32 emulsified with Freund's complete adjuvant. Five mice from each group were killed prior to immunization, prior to challenge and 10 days, 30 days and 45 days post-challenge, respectively. Spleen cells from these mice were cultured and levels of cytokine secreted by splenocytes were detected by ELISA. RESULTS: Compared with the control group, the quantity of IL-4, IL-5 and IFN-gamma released from rGST-Sj32-primed splenocytes were increased to varying degrees, being (10.21+/-3.65) ng/L, (19.89+/-9.57) ng/L and (5.09+/-2.51) microg/L, respectively. The level of IFN-gamma post challenge did not increase in control group and immunization group. The levels of IL- 4 and IL-5 in the control group were elevated gradually following challenge, and were much higher than those in immunization group. CONCLUSION: The immunization with rGST-Sj32 elicited mainly a Th1 type immune response in BALB/c mice, whereas the immune response induced by challenge was a Th2 type.