Feature Mixture on Pre-Trained Model for Few-Shot Learning.

Few-shot learning (FSL) aims at recognizing a novel object under limited training samples. A robust feature extractor (backbone) can significantly improve the recognition performance of the FSL model. However, training an effective backbone is a challenging issue since 1) designing and validating structures of backbones are time-consuming and expensive processes, and 2) a backbone trained on the known (base) categories is more inclined to focus on the textures of the objects it learns, which is hard to describe the novel samples. To solve these problems, we propose a feature mixture operation on the pre-trained (fixed) features: 1) We replace a part of the values of the feature map from a novel category with the content of other feature maps to increase the generalizability and diversity of training samples, which avoids retraining a complex backbone with high computational costs. 2) We use the similarities between the features to constrain the mixture operation, which helps the classifier focus on the representations of the novel object where these representations are hidden in the features from the pre-trained backbone with biased training. Experimental studies on five benchmark datasets in both inductive and transductive settings demonstrate the effectiveness of our feature mixture (FM). Specifically, compared with the baseline on the Mini-ImageNet dataset, it achieves 3.8% and 4.2% accuracy improvements for 1 and 5 training samples, respectively. Additionally, the proposed mixture operation can be used to improve other existing FSL methods based on backbone training.

Reconfigurable 4D printing via mechanically robust covalent adaptable network shape memory polymer.

4D printing enables 3D printed structures to change shape over "time" in response to environmental stimulus. Because of relatively high modulus, shape memory polymers (SMPs) have been widely used for 4D printing. However, most SMPs for 4D printing are thermosets, which only have one permanent shape. Despite the efforts that implement covalent adaptable networks (CANs) into SMPs to achieve shape reconfigurability, weak thermomechanical properties of the current CAN-SMPs exclude them from practical applications. Here, we report reconfigurable 4D printing via mechanically robust CAN-SMPs (MRC-SMPs), which have high deformability at both programming and reconfiguration temperatures (>1400%), high Tg (75°C), and high room temperature modulus (1.06 GPa). The high printability for DLP high-resolution 3D printing allows MRC-SMPs to create highly complex SMP 3D structures that can be reconfigured multiple times under large deformation. The demonstrations show that the reconfigurable 4D printing allows one printed SMP structure to fulfill multiple tasks.

Direct 4D printing of ceramics driven by hydrogel dehydration.

4D printing technology combines 3D printing and stimulus-responsive materials, enabling construction of complex 3D objects efficiently. However, unlike smart soft materials, 4D printing of ceramics is a great challenge due to the extremely weak deformability of ceramics. Here, we report a feasible and efficient manufacturing and design approach to realize direct 4D printing of ceramics. Photocurable ceramic elastomer slurry and hydrogel precursor are developed for the fabrication of hydrogel-ceramic laminates via multimaterial digital light processing 3D printing. Flat patterned laminates evolve into complex 3D structures driven by hydrogel dehydration, and then turn into pure ceramics after sintering. Considering the dehydration-induced deformation and sintering-induced shape retraction, we develop a theoretical model to calculate the curvatures of bent laminate and sintered ceramic part. Then, we build a design flow for direct 4D printing of various complex ceramic objects. This approach opens a new avenue for the development of ceramic 4D printing technology.

Learning to Double-Check Model Prediction From a Causal Perspective.

The present machine learning schema typically uses a one-pass model inference (e.g., forward propagation) to make predictions in the testing phase. It is inherently different from human students who double-check the answer during examinations especially when the confidence is low. To bridge this gap, we propose a learning to double-check (L2D) framework, which formulates double check as a learnable procedure with two core operations: recognizing unreliable predictions and revising predictions. To judge the correctness of a prediction, we resort to counterfactual faithfulness in causal theory and design a contrastive faithfulness measure. In particular, L2D generates counterfactual features by imagining: "what would the sample features be if its label was the predicted class" and judges the prediction by the faithfulness of the counterfactual features. Furthermore, we design a simple and effective revision module to revise the original model prediction according to the faithfulness. We apply the L2D framework to three classification models and conduct experiments on two public datasets for image classification, validating the effectiveness of L2D in prediction correctness judgment and revision.

Test-Time Training for Deep MS/MS Spectrum Prediction Improves Peptide Identification.

In bottom-up proteomics, peptide-spectrum matching is critical for peptide and protein identification. Recently, deep learning models have been used to predict tandem mass spectra of peptides, enabling the calculation of similarity scores between the predicted and experimental spectra for peptide-spectrum matching. These models follow the supervised learning paradigm, which trains a general model using paired peptides and spectra from standard data sets and directly employs the model on experimental data. However, this approach can lead to inaccurate predictions due to differences between the training data and the experimental data, such as sample types, enzyme specificity, and instrument calibration. To tackle this problem, we developed a test-time training paradigm that adapts the pretrained model to generate experimental data-specific models, namely, PepT3. PepT3 yields a 10-40% increase in peptide identification depending on the variability in training and experimental data. Intriguingly, when applied to a patient-derived immunopeptidomic sample, PepT3 increases the identification of tumor-specific immunopeptide candidates by 60%. Two-thirds of the newly identified candidates are predicted to bind to the patient's human leukocyte antigen isoforms. To facilitate access of the model and all the results, we have archived all the intermediate files in Zenodo.org with identifier 8231084.

Shape-Tunable 4D Printing of LCEs via Cooling Rate Modulation: Stimulus-Free Locking of Actuated State at Room Temperature.

Liquid crystal elastomers (LCEs) have garnered considerable attention in the field of four-dimensional (4D) printing due to their large, reversible, and anisotropic shape-morphing capabilities. By utilizing direct ink writing, intricate LCE structures with programmable shape morphing can be achieved. However, the maintenance of the actuated state for LCEs requires continuous and substantial external stimuli, presenting challenges for practical applications, particularly under ambient conditions. This study reports a straightforward and effective physical approach to lock the actuated state of LCEs through rapid cooling while preserving their reversible performance. Rapid cooling significantly reduces the mobility of the lightly cross-linked network in LCEs, resulting in a notably slow recovery of mesogen alignment. As a result, the locked LCE structures retain their actuated state even at room temperature. Moreover, we demonstrate the ability to achieve tunable shapes between the original and actuated states by modulating the cooling rate, i.e., varying the temperature and type of cooling medium. The proposed method opens up new possibilities to achieve stable and tunable shape locking of soft devices for engineering applications.

Polyelectrolyte elastomer-based ionotronic sensors with multi-mode sensing capabilities via multi-material 3D printing.

Stretchable ionotronics have drawn increasing attention during the past decade, enabling myriad applications in engineering and biomedicine. However, existing ionotronic sensors suffer from limited sensing capabilities due to simple device structures and poor stability due to the leakage of ingredients. In this study, we rationally design and fabricate a plethora of architected leakage-free ionotronic sensors with multi-mode sensing capabilities, using DLP-based 3D printing and a polyelectrolyte elastomer. We synthesize a photo-polymerizable ionic monomer for the polyelectrolyte elastomer, which is stretchable, transparent, ionically conductive, thermally stable, and leakage-resistant. The printed sensors possess robust interfaces and extraordinary long-term stability. The multi-material 3D printing allows high flexibility in structural design, enabling the sensing of tension, compression, shear, and torsion, with on-demand tailorable sensitivities through elaborate programming of device architectures. Furthermore, we fabricate integrated ionotronic sensors that can perceive different mechanical stimuli simultaneously without mutual signal interferences. We demonstrate a sensing kit consisting of four shear sensors and one compressive sensor, and connect it to a remote-control system that is programmed to wirelessly control the flight of a drone. Multi-material 3D printing of leakage-free polyelectrolyte elastomers paves new avenues for manufacturing stretchable ionotronics by resolving the deficiencies of stability and functionalities simultaneously.

A volatile microemulsion method of preparing water-soluble photo-absorbers for 3D printing of high-resolution, high-water-content hydrogel structures.

Digital light processing (DLP)-based three-dimensional (3D) printing is an ideal tool to manufacture hydrogel structures in complex 3D forms. Using DLP to print hydrogel structures with high resolution requires the addition of water-soluble photo-absorbers to absorb excess light and confine photopolymerization to the desired area. However, the current photo-absorbers for hydrogel printing are neither efficient to absorb the excess light nor water-soluble. Herein, we report a volatile microemulsion template method that converts a wide range of commercial non-water-soluble photo-absorbers including Sudan orange G, quercetin, and many others to water-soluble nanoparticles with solubility above 1.0 g mL-1. After using these water-soluble photo-absorber nanoparticles, the highest lateral and vertical resolutions of printing high-water-content (70-80 wt%) hydrogels can be improved to 5 µm and 20 µm, respectively. Moreover, the quercetin nanoparticle can be easily washed out so that we achieve colorless and transparent printed hydrogel structures with excellent mechanical deformability and biocompatibility as well as thermally controllable variations on transparency and actuation. The proposed methods pave a new efficient way to develop water-soluble photo-absorbers, which helps to greatly improve the printing resolution of the high-water-content hydrogel structure and would be beneficial to extend the application scope of hydrogels.

GCRec: Graph-Augmented Capsule Network for Next-Item Recommendation.

Next-item recommendation has been a hot research, which aims at predicting the next action by modeling users' behavior sequences. While previous efforts toward this task have been made in capturing complex item transition patterns, we argue that they still suffer from three limitations: 1) they have difficulty in explicitly capturing the impact of inherent order of item transition patterns; 2) only a simple and crude embedding is insufficient to yield satisfactory long-term users' representations from limited training sequences; and 3) they are incapable of dynamically integrating long-term and short-term user interest modeling. In this work, we propose a novel solution named graph-augmented capsule network (GCRec), which exploits sequential user behaviors in a more fine-grained manner. Specifically, we employ a linear graph convolution module to learn informative long-term representations of users. Furthermore, we devise a user-specific capsule module and a position-aware gating module, which are sensitive to the relative sequential order of the recently interacted items, to capture sequential patterns at union-level and point-level. To aggregate the long-term and short-term user interests as a representative vector, we design a dual-gating mechanism, which could decide the contribution ratio of each module given different contextual information. Through extensive experiments on four benchmarks, we validate the rationality and effectiveness of GCRec on the next-item recommendation task.

Reinforced Causal Explainer for Graph Neural Networks.

Explainability is crucial for probing graph neural networks (GNNs), answering questions like "Why the GNN model makes a certain prediction?". Feature attribution is a prevalent technique of highlighting the explanatory subgraph in the input graph, which plausibly leads the GNN model to make its prediction. Various attribution methods have been proposed to exploit gradient-like or attention scores as the attributions of edges, then select the salient edges with top attribution scores as the explanation. However, most of these works make an untenable assumption - the selected edges are linearly independent - thus leaving the dependencies among edges largely unexplored, especially their coalition effect. We demonstrate unambiguous drawbacks of this assumption - making the explanatory subgraph unfaithful and verbose. To address this challenge, we propose a reinforcement learning agent, Reinforced Causal Explainer (RC-Explainer). It frames the explanation task as a sequential decision process - an explanatory subgraph is successively constructed by adding a salient edge to connect the previously selected subgraph. Technically, its policy network predicts the action of edge addition, and gets a reward that quantifies the action's causal effect on the prediction. Such reward accounts for the dependency of the newly-added edge and the previously-added edges, thus reflecting whether they collaborate together and form a coalition to pursue better explanations. It is trained via policy gradient to optimize the reward stream of edge sequences. As such, RC-Explainer is able to generate faithful and concise explanations, and has a better generalization power to unseen graphs. When explaining different GNNs on three graph classification datasets, RC-Explainer achieves better or comparable performance to state-of-the-art approaches w.r.t. two quantitative metrics: predictive accuracy, contrastivity, and safely passes sanity checks and visual inspections. Codes and datasets are available at https://github.com/xiangwang1223/reinforced_causal_explainer.

Multimaterial Three-Dimensional Printing of Ultraviolet-Curable Ionic Conductive Elastomers with Diverse Polymers for Multifunctional Flexible Electronics.

Ionic conductive elastomers (ICEs) are emerging stretchable and ionic conductive materials that are solvent-free and thus demonstrate excellent thermal stability. Three-dimensional (3D) printing that creates complex 3D structures in free forms is considered as an ideal approach to manufacture sophisticated ICE-based devices. However, the current technologies constrain 3D printed ICE structures in a single material, which greatly limits functionality and performance of ICE-based devices and machines. Here, we report a digital light processing (DLP)-based multimaterial 3D printing capability to seemly integrate ultraviolet-curable ICE (UV-ICE) with nonconductive materials to create ionic flexible electronic devices in 3D forms with enhanced performance. This unique capability allows us to readily manufacture various 3D flexible electronic devices. To demonstrate this, we printed UV-ICE circuits into polymer substrates with different mechanical properties to create resistive strain and force sensors; we printed flexible capacitive sensors with high sensitivity (2 kPa-1) and a wide range of measured pressures (from 5 Pa to 550 kPa) by creating a complex microstructure in the dielectric layer; we even realized ionic conductor-activated four-dimensional (4D) printing by printing a UV-ICE circuit into a shape memory polymer substrate. The proposed approach paves a new efficient way to realize multifunctional flexible devices and machines by bonding ICEs with other polymers in 3D forms.

Centrifugal multimaterial 3D printing of multifunctional heterogeneous objects.

There are growing demands for multimaterial three-dimensional (3D) printing to manufacture 3D object where voxels with different properties and functions are precisely arranged. Digital light processing (DLP) is a high-resolution fast-speed 3D printing technology suitable for various materials. However, multimaterial 3D printing is challenging for DLP as the current multimaterial switching methods require direct contact onto the printed part to remove residual resin. Here we report a DLP-based centrifugal multimaterial (CM) 3D printing method to generate large-volume heterogeneous 3D objects where composition, property and function are programmable at voxel scale. Centrifugal force enables non-contact, high-efficiency multimaterial switching, so that the CM 3D printer can print heterogenous 3D structures in large area (up to 180 mm × 130 mm) made of materials ranging from hydrogels to functional polymers, and even ceramics. Our CM 3D printing method exhibits excellent capability of fabricating digital materials, soft robots, and ceramic devices.

Spatio-Temporal Collaborative Module for Efficient Action Recognition.

Efficient action recognition aims to classify a video clip into a specific action category with a low computational cost. It is challenging since the integrated spatial-temporal calculation (e. g., 3D convolution) introduces intensive operations and increases complexity. This paper explores the feasibility of the integration of channel splitting and filter decoupling for efficient architecture design and feature refinement by proposing a novel spatio-temporal collaborative (STC) module. STC splits the video feature channels into two groups and separately learns spatio-temporal representations in parallel with decoupled convolutional operators. Particularly, STC consists of two computation-efficient blocks, i.e., [Formula: see text] and [Formula: see text], where they extract either spatial ( S· ) or temporal ( T· ) features and further refine their features with either temporal ( ·T ) or spatial ( ·S ) contexts globally. The spatial/temporal context refers to information dynamics aggregated from temporal/spatial axis. To thoroughly examine our method's performance in video action recognition tasks, we conduct extensive experiments using five video benchmark datasets requiring temporal reasoning. Experimental results show that the proposed STC networks achieve a competitive trade-off between model efficiency and effectiveness.

Causal Incremental Graph Convolution for Recommender System Retraining.

The real-world recommender system needs to be regularly retrained to keep with the new data. In this work, we consider how to efficiently retrain graph convolution network (GCN)-based recommender models that are state-of-the-art techniques for the collaborative recommendation. To pursue high efficiency, we set the target as using only new data for model updating, meanwhile not sacrificing the recommendation accuracy compared with full model retraining. This is nontrivial to achieve since the interaction data participates in both the graph structure for model construction and the loss function for model learning, whereas the old graph structure is not allowed to use in model updating. Toward the goal, we propose a causal incremental graph convolution (IGC) approach, which consists of two new operators named IGC and colliding effect distillation (CED) to estimate the output of full graph convolution. In particular, we devise simple and effective modules for IGC to ingeniously combine the old representations and the incremental graph and effectively fuse the long- and short-term preference signals. CED aims to avoid the out-of-date issue of inactive nodes that are not in the incremental graph, which connects the new data with inactive nodes through causal inference. In particular, CED estimates the causal effect of new data on the representation of inactive nodes through the control of their collider. Extensive experiments on three real-world datasets demonstrate both accuracy gains and significant speed-ups over the existing retraining mechanism.

Learning to Compose and Reason with Language Tree Structures for Visual Grounding.

Grounding natural language in images, such as localizing "the black dog on the left of the tree", is one of the core problems in artificial intelligence, as it needs to comprehend the fine-grained language compositions. However, existing solutions merely rely on the association between the holistic language features and visual features, while neglect the nature of composite reasoning implied in the language. In this paper, we propose a natural language grounding model that can automatically compose a binary tree structure for parsing the language and then perform visual reasoning along the tree in a bottom-up fashion. We call our model RvG-Tree: Recursive Grounding Tree, which is inspired by the intuition that any language expression can be recursively decomposed into two constituent parts, and the grounding confidence score can be recursively accumulated by calculating their grounding scores returned by the two sub-trees.RvG-Tree can be trained end-to-end by using the Straight-Through Gumbel-Softmax estimator that allows the gradients from the continuous score functions passing through the discrete tree construction. Experiments on several benchmarks show that our model achieves the state-of-the-art performance with more explainable reasoning.

A KG-Enhanced Multi-Graph Neural Network for Attentive Herb Recommendation.

Traditional Chinese Medicine (TCM) has the longest clinical history in Asia and contributes a lot to health maintenance worldwide. An essential step during the TCM diagnostic process is syndrome induction, which comprehensively analyzes the symptoms and generates an overall summary of the symptoms. Given a set of symptoms, the existing herb recommenders aim to generate the corresponding herbs as a treatment by inducing the implicit syndrome representations based on TCM prescriptions. As different symptoms have various importance during the comprehensive consideration, we argue that treating the co-occurred symptoms equally to do syndrome induction in the previous studies will lead to the coarse-grained syndrome representation. In this paper, we bring the attention mechanism to model the syndrome induction process. Given a set of symptoms, we leverage an attention network to discriminate the symptom importance and adaptively fuse the symptom embeddings. Besides, we introduce a TCM knowledge graph to enrich the input corpus and improve the quality of representation learning. Further, we build a KG-enhanced Multi-Graph Neural Network architecture, which performs the attentive propagation to combine node feature and graph structural information. Extensive experimental results on two TCM data sets show that our proposed model has the outstanding performance over the state-of-the-arts.

General One-Pot Method for Preparing Highly Water-Soluble and Biocompatible Photoinitiators for Digital Light Processing-Based 3D Printing of Hydrogels.

We report a facile but general method to prepare highly water-soluble and biocompatible photoinitiators for digital light processing (DLP)-based 3D printing of high-resolution hydrogel structures. Through a simple and straightforward one-pot procedure, we can synthesize a metal-phenyl(2,4,6-trimethylbenzoyl)phosphinates (M-TMPP)-based photoinitiator with excellent water solubility (up to ∼50 g/L), which is much higher than that of previously reported water-soluble photoinitiators. The M-TMPP aqueous solutions show excellent biocompatibility, which meets the prerequisite for biomedical applications. Moreover, we used M-TMPP to prepare visible light (405 nm)-curable hydrogel precursor solutions for 3D printing hydrogel structures with a high water content (80 wt %), high resolution (∼7 µm), high deformability (more than 80% compression), and complex geometry. The printed hydrogel structures demonstrate great potential in flexible electronic sensors due to the fast mechanical response and high stability under cyclic loadings.

Mechanically Robust and UV-Curable Shape-Memory Polymers for Digital Light Processing Based 4D Printing.

4D printing is an emerging fabrication technology that enables 3D printed structures to change configuration over "time" in response to an environmental stimulus. Compared with other soft active materials used for 4D printing, shape-memory polymers (SMPs) have higher stiffness, and are compatible with various 3D printing technologies. Among them, ultraviolet (UV)-curable SMPs are compatible with Digital Light Processing (DLP)-based 3D printing to fabricate SMP-based structures with complex geometry and high-resolution. However, UV-curable SMPs have limitations in terms of mechanical performance, which significantly constrains their application ranges. Here, a mechanically robust and UV-curable SMP system is reported, which is highly deformable, fatigue resistant, and compatible with DLP-based 3D printing, to fabricate high-resolution (up to 2 µm), highly complex 3D structures that exhibit large shape change (up to 1240%) upon heating. More importantly, the developed SMP system exhibits excellent fatigue resistance and can be repeatedly loaded more than 10 000 times. The development of the mechanically robust and UV-curable SMPs significantly improves the mechanical performance of the SMP-based 4D printing structures, which allows them to be applied to engineering applications such as aerospace, smart furniture, and soft robots.

3D printing of highly stretchable hydrogel with diverse UV curable polymers.

Hydrogel-polymer hybrids have been widely used for various applications such as biomedical devices and flexible electronics. However, the current technologies constrain the geometries of hydrogel-polymer hybrid to laminates consisting of hydrogel with silicone rubbers. This greatly limits functionality and performance of hydrogel-polymer-based devices and machines. Here, we report a simple yet versatile multimaterial 3D printing approach to fabricate complex hybrid 3D structures consisting of highly stretchable and high-water content acrylamide-PEGDA (AP) hydrogels covalently bonded with diverse UV curable polymers. The hybrid structures are printed on a self-built DLP-based multimaterial 3D printer. We realize covalent bonding between AP hydrogel and other polymers through incomplete polymerization of AP hydrogel initiated by the water-soluble photoinitiator TPO nanoparticles. We demonstrate a few applications taking advantage of this approach. The proposed approach paves a new way to realize multifunctional soft devices and machines by bonding hydrogel with other polymers in 3D forms.

Generating Face Images With Attributes for Free.

With superhuman-level performance of face recognition, we are more concerned about the recognition of fine-grained attributes, such as emotion, age, and gender. However, given that the label space is extremely large and follows a long-tail distribution, it is quite expensive to collect sufficient samples for fine-grained attributes. This results in imbalanced training samples and inferior attribute recognition models. To this end, we propose the use of arbitrary attribute combinations, without human effort, to synthesize face images. In particular, to bridge the semantic gap between high-level attribute label space and low-level face image, we propose a novel neural-network-based approach that maps the target attribute labels to an embedding vector, which can be fed into a pretrained image decoder to synthesize a new face image. Furthermore, to regularize the attribute for image synthesis, we propose to use a perceptual loss to make the new image explicitly faithful to target attributes. Experimental results show that our approach can generate photorealistic face images from attribute labels, and more importantly, by serving as augmented training samples, these images can significantly boost the performance of attribute recognition model. The code is open-sourced at this link.