Self-Distillation for Randomized Neural Networks.

Knowledge distillation (KD) is a conventional method in the field of deep learning that enables the transfer of dark knowledge from a teacher model to a student model, consequently improving the performance of the student model. In randomized neural networks, due to the simple topology of network architecture and the insignificant relationship between model performance and model size, KD is not able to improve model performance. In this work, we propose a self-distillation pipeline for randomized neural networks: the predictions of the network itself are regarded as the additional target, which are mixed with the weighted original target as a distillation target containing dark knowledge to supervise the training of the model. All the predictions during multi-generation self-distillation process can be integrated by a multi-teacher method. By induction, we have additionally arrived at the methods for infinite self-distillation (ISD) of randomized neural networks. We then provide relevant theoretical analysis about the self-distillation method for randomized neural networks. Furthermore, we demonstrated the effectiveness of the proposed method in practical applications on several benchmark datasets.

Online dynamic ensemble deep random vector functional link neural network for forecasting.

This paper proposes a three-stage online deep learning model for time series based on the ensemble deep random vector functional link (edRVFL). The edRVFL stacks multiple randomized layers to enhance the single-layer RVFL's representation ability. Each hidden layer's representation is utilized for training an output layer, and the ensemble of all output layers forms the edRVFL's output. However, the original edRVFL is not designed for online learning, and the randomized nature of the features is harmful to extracting meaningful temporal features. In order to address the limitations and extend the edRVFL to an online learning mode, this paper proposes a dynamic edRVFL consisting of three online components, the online decomposition, the online training, and the online dynamic ensemble. First, an online decomposition is utilized as a feature engineering block for the edRVFL. Then, an online learning algorithm is designed to learn the edRVFL. Finally, an online dynamic ensemble method, which can measure the change in the distribution, is proposed for aggregating all layers' outputs. This paper evaluates and compares the proposed model with state-of-the-art methods on sixteen time series.

Learning-Based Motion-Intention Prediction for End-Point Control of Upper-Limb-Assistive Robots.

The lack of intuitive and active human-robot interaction makes it difficult to use upper-limb-assistive devices. In this paper, we propose a novel learning-based controller that intuitively uses onset motion to predict the desired end-point position for an assistive robot. A multi-modal sensing system comprising inertial measurement units (IMUs), electromyographic (EMG) sensors, and mechanomyography (MMG) sensors was implemented. This system was used to acquire kinematic and physiological signals during reaching and placing tasks performed by five healthy subjects. The onset motion data of each motion trial were extracted to input into traditional regression models and deep learning models for training and testing. The models can predict the position of the hand in planar space, which is the reference position for low-level position controllers. The results show that using IMU sensor with the proposed prediction model is sufficient for motion intention detection, which can provide almost the same prediction performance compared with adding EMG or MMG. Additionally, recurrent neural network (RNN)-based models can predict target positions over a short onset time window for reaching motions and are suitable for predicting targets over a longer horizon for placing tasks. This study's detailed analysis can improve the usability of the assistive/rehabilitation robots.

Inpatient Discharges Forecasting for Singapore Hospitals by Machine Learning.

Hospitals can predetermine the admission rate and facilitate resource allocation based on valid emergency requests and bed capacity estimation. The excess unoccupied beds can be determined with the help of forecasting the number of discharged patients. Extracting predictive features and mining the temporal patterns from historical observations are crucial for accurate and reliable forecasts. Machine learning algorithms have demonstrated the ability to learn temporal knowledge and make predictions for unseen inputs. This paper utilizes several machine learning algorithms to forecast the inpatient discharges of Singapore hospitals and compare them with statistical methods. A novel ensemble deep learning algorithm based on random vector functional links is established to predict inpatient discharges. The ensemble deep learning framework is optimized in a greedy layer-wise fashion. Several forecasting metrics and statistical tests are utilized to demonstrate the proposed method's superiority. The proposed algorithm statistically outperforms the benchmark with a ranking of 1.875. Finally, practical implications and future directions are discussed.