A Siamese Vision Transformer for Bearings Fault Diagnosis.

Fault diagnosis methods based on deep learning have progressed greatly in recent years. However, the limited training data and complex work conditions still restrict the application of these intelligent methods. This paper proposes an intelligent bearing fault diagnosis method, i.e., Siamese Vision Transformer, suiting limited training data and complex work conditions. The Siamese Vision Transformer, combining Siamese network and Vision Transformer, is designed to efficiently extract the feature vectors of input samples in high-level space and complete the classification of the fault. In addition, a new loss function combining the Kullback-Liebler divergence both directions is proposed to improve the performance of the proposed model. Furthermore, a new training strategy termed random mask is designed to enhance input data diversity. A comparative test is conducted on the Case Western Reserve University bearing dataset and Paderborn dataset and our method achieves reasonably high accuracy with limited data and satisfactory generation capability for cross-domain tasks.

A Novel Bearing Fault Diagnosis Method Based on Few-Shot Transfer Learning across Different Datasets.

At present, the success of most intelligent fault diagnosis methods is heavily dependent on large datasets of artificial simulation faults (ASF), which have not been widely used in practice because it is often costly to obtain a large number of samples in reality. Fortunately, various faults can be easily simulated in the laboratory, and these simulated faults contain a lot of fault diagnosis knowledge. In this study, based on a Siamese network framework, we propose a bearing fault diagnosis based on few-shot transfer learning across different datasets (cross-machine), using the knowledge of ASF to diagnose bearings with natural faults (NF). First of all, the model obtains a good feature encoder in the source domain, then defines a fault support set for comparison, and finally adjusts the support set with a very small number of target domain samples to improve the fault diagnosis performance of the model. We carried out experimental verification from many aspects on the ASF and NF datasets provided by Case Western Reserve University (CWRU) and Paderborn University (PU). The results show that the proposed method can fully learn diagnostic knowledge in different ASF datasets and sample numbers, and effectively use this knowledge to accurately identify the health state of the NF bearing, which has strong generalization and robustness. Our method does not need second training, which may be more convenient in some practical applications. Finally, we also discuss the possible limitations of this method.

A Hybrid Matching Network for Fault Diagnosis under Different Working Conditions with Limited Data.

Intelligent fault diagnosis methods based on deep learning have achieved much progress in recent years. However, there are two major factors causing serious degradation of the performance of these algorithms in real industrial applications, i.e., limited labeled training data and complex working conditions. To solve these problems, this study proposed a domain generalization-based hybrid matching network utilizing a matching network to diagnose the faults using features encoded by an autoencoder. The main idea was to regularize the feature extractor of the network with an autoencoder in order to reduce the risk of overfitting with limited training samples. In addition, a training strategy using dropout with random changing rates on inputs was implemented to enhance the model's generalization on unseen domains. The proposed method was validated on two different datasets containing artificial and real faults. The results showed that considerable performance was achieved by the proposed method under cross-domain tasks with limited training samples.

Nonlinear Hyperparameter Optimization of a Neural Network in Image Processing for Micromachines.

Deep neural networks are widely used in the field of image processing for micromachines, such as in 3D shape detection in microelectronic high-speed dispensing and object detection in microrobots. It is already known that hyperparameters and their interactions impact neural network model performance. Taking advantage of the mathematical correlations between hyperparameters and the corresponding deep learning model to adjust hyperparameters intelligently is the key to obtaining an optimal solution from a deep neural network model. Leveraging these correlations is also significant for unlocking the "black box" of deep learning by revealing the mechanism of its mathematical principle. However, there is no complete system for studying the combination of mathematical derivation and experimental verification methods to quantify the impacts of hyperparameters on the performances of deep learning models. Therefore, in this paper, the authors analyzed the mathematical relationships among four hyperparameters: the learning rate, batch size, dropout rate, and convolution kernel size. A generalized multiparameter mathematical correlation model was also established, which showed that the interaction between these hyperparameters played an important role in the neural network's performance. Different experiments were verified by running convolutional neural network algorithms to validate the proposal on the MNIST dataset. Notably, this research can help establish a universal multiparameter mathematical correlation model to guide the deep learning parameter adjustment process.

Intelligent Optimization Algorithm-Based Path Planning for a Mobile Robot.

The purpose of mobile robot path planning is to produce the optimal safe path. However, mobile robots have poor real-time obstacle avoidance in local path planning and longer paths in global path planning. In order to improve the accuracy of real-time obstacle avoidance prediction of local path planning, shorten the path length of global path planning, reduce the path planning time, and then obtain a better safe path, we propose a real-time obstacle avoidance decision model based on machine learning (ML) algorithms, an improved smooth rapidly exploring random tree (S-RRT) algorithm, and an improved hybrid genetic algorithm-ant colony optimization (HGA-ACO). Firstly, in local path planning, the machine learning algorithms are used to train the datasets, the real-time obstacle avoidance decision model is established, and cross validation is performed. Secondly, in global path planning, the greedy algorithm idea and B-spline curve are introduced into the RRT algorithm, redundant nodes are removed, and the reverse iteration is performed to generate a smooth path. Then, in path planning, the fitness function and genetic operation method of genetic algorithm are optimized, the pheromone update strategy and deadlock elimination strategy of ant colony algorithm are optimized, and the genetic-ant colony fusion strategy is used to fuse the two algorithms. Finally, the optimized path planning algorithm is used for simulation experiment. Comparative simulation experiments show that the random forest has the highest real-time obstacle avoidance prediction accuracy in local path planning, and the S-RRT algorithm can effectively shorten the total path length generated by the RRT algorithm in global path planning. The HGA-ACO algorithm can reduce the iteration number reasonably, reduce the search time effectively, and obtain the optimal solution in path planning.

Trajectory Planning of Robot Manipulator Based on RBF Neural Network.

Robot manipulator trajectory planning is one of the core robot technologies, and the design of controllers can improve the trajectory accuracy of manipulators. However, most of the controllers designed at this stage have not been able to effectively solve the nonlinearity and uncertainty problems of the high degree of freedom manipulators. In order to overcome these problems and improve the trajectory performance of the high degree of freedom manipulators, a manipulator trajectory planning method based on a radial basis function (RBF) neural network is proposed in this work. Firstly, a 6-DOF robot experimental platform was designed and built. Secondly, the overall manipulator trajectory planning framework was designed, which included manipulator kinematics and dynamics and a quintic polynomial interpolation algorithm. Then, an adaptive robust controller based on an RBF neural network was designed to deal with the nonlinearity and uncertainty problems, and Lyapunov theory was used to ensure the stability of the manipulator control system and the convergence of the tracking error. Finally, to test the method, a simulation and experiment were carried out. The simulation results showed that the proposed method improved the response and tracking performance to a certain extent, reduced the adjustment time and chattering, and ensured the smooth operation of the manipulator in the course of trajectory planning. The experimental results verified the effectiveness and feasibility of the method proposed in this paper.

Deep learning pan-specific model for interpretable MHC-I peptide binding prediction with improved attention mechanism.

Accurate prediction of peptide binding affinity to the major histocompatibility complex (MHC) proteins has the potential to design better therapeutic vaccines. Previous work has shown that pan-specific prediction algorithms can achieve better prediction performance than other approaches. However, most of the top algorithms are neural networks based black box models. Here, we propose DeepAttentionPan, an improved pan-specific model, based on convolutional neural networks and attention mechanisms for more flexible, stable and interpretable MHC-I binding prediction. With the attention mechanism, our ensemble model consisting of 20 trained networks achieves high and more stabilized prediction performance. Extensive tests on IEDB's weekly benchmark dataset show that our method achieves state-of-the-art prediction performance on 21 test allele datasets. Analysis of the peptide positional attention weights learned by our model demonstrates its capability to capture critical binding positions of the peptides, which leads to mechanistic understanding of MHC-peptide binding with high alignment with experimentally verified results. Furthermore, we show that with transfer learning, our pan model can be fine-tuned for alleles with few samples to achieve additional performance improvement. DeepAttentionPan is freely available as an open-source software at https://github.com/jjin49/DeepAttentionPan.

DeepSeqPan, a novel deep convolutional neural network model for pan-specific class I HLA-peptide binding affinity prediction.

Interactions between human leukocyte antigens (HLAs) and peptides play a critical role in the human immune system. Accurate computational prediction of HLA-binding peptides can be used for peptide drug discovery. Currently, the best prediction algorithms are neural network-based pan-specific models, which take advantage of the large amount of data across HLA alleles. However, current pan-specific models are all based on the pseudo sequence encoding for modeling the binding context, which is based on 34 positions identified from the HLA protein-peptide bound structures in early works. In this work, we proposed a novel deep convolutional neural network model (DCNN) for HLA-peptide binding prediction, in which the encoding of the HLA sequence and the binding context are both learned by the network itself without requiring the HLA-peptide bound structure information. Our DCNN model is also characterized by its binding context extraction layer and dual outputs with both binding affinity output and binding probability outputs. Evaluation on public benchmark datasets shows that our DeepSeqPan model without HLA structural information in training achieves state-of-the-art performance on a large number of HLA alleles with good generalization capability. Since our model only needs raw sequences from the HLA-peptide binding pairs, it can be applied to binding predictions of HLAs without structure information and can also be applied to other protein binding problems such as protein-DNA and protein-RNA bindings. The implementation code and trained models are freely available at https://github.com/pcpLiu/DeepSeqPan .