Deterministic Gradient-Descent Learning of Linear Regressions: Adaptive Algorithms, Convergence Analysis and Noise Compensation.

Weight learning forms a basis for the machine learning and numerous algorithms have been adopted up to date. Most of the algorithms were either developed in the stochastic framework or aimed at minimization of loss or regret functions. Asymptotic convergence of weight learning, vital for good output prediction, was seldom guaranteed for online applications. Since linear regression is the most fundamental component in machine learning, we focus on this model in this paper. Aiming at online applications, a deterministic analysis method is developed based on LaSalle's invariance principle. Convergence conditions are derived for both the first-order and the second-order learning algorithms, without resorting to any stochastic argument. Moreover, the deterministic approach makes it easy to analyze the noise influence. Specifically, adaptive hyperparameters are derived in this framework and their tuning rules disclosed for the compensation of measurement noise. Comparison with four most popular algorithms validates that this approach has a higher learning capability and is quite promising in enhancing the weight learning performance.

A control-theoretic approach for input saturated linear systems: Integration of phase-shaping and gain-scheduling.

Saturation is mainly characterized by its passivity and magnitude bound. But most of the saturation control methods only make use of either of these features. To enhance the performance of saturated systems, this paper develops a novel method capable of fully using both of these two features. This method is a two-stage design scheme which integrates the phase-shaping technique with the gain-scheduled control. The phase-shaping fully uses the passivity of saturation while the gain-scheduling actively utilizes the magnitude bound of saturation. In this way, the design conservatism associated with existing methods is reduced substantially. Specifically, a matrix-type phase-shaping method is developed through the placement of systems' frequency loci, and a meta-heuristic method is devised for the design of the phase-shaping function. Furthermore, the gain-scheduled control is transformed into the robust performance problem of a passive uncertain system, and designed by the passivity-based robust control method of the authors. Application to two practical control systems validates the effectiveness of the proposed method. The superiority is demonstrated via comparisons with typical saturation control methods.

A time series model based on hybrid-kernel least-squares support vector machine for short-term wind power forecasting.

In this paper, a time series model based on hybrid-kernel least-squares support vector machine (HKLSSVM) with three processes of decomposition, classification, and reconstruction is proposed to predict short-term wind power. Firstly, on the basis of the maximal wavelet decomposition (MWD) and fuzzy C-means algorithm, a decomposition method decomposes wind power time series and classifies the decomposition time series components into three classes according to amplitude-frequency characteristics. Then, time series models on the basis of least-squares support vector machine (LSSVM) with three different kernels are established for these three classes. Non-dominated sorting genetic algorithm II optimizes the parameters of each forecasting model. Finally, outputs of forecasting models are reconstructed to obtain the forecasting power. The proposed model is compared with the empirical-mode-decomposition least-squares support vector machine (EMD-LSSVM) model and wavelet-decomposition least-squares support vector machine (WDLSSVM) model. The results of the comparison show that proposed model performs better than these benchmark models.

A detailed 3D ankle-foot model for simulate dynamics of lower limb orthosis.

The objective of this study is to develop a 3D ankle-foot model containing toe expression for designing an AFO (ankle-foot orthosis) with a training function. Two experiments were conducted to (1) show the influence of toes by comparing walking with and without an AFO, and (2) clarify the functions of toes during walking by correlating the activity of the major muscles controlling the ankle and the toes to the sole pressure data during walking. By analyzing the results of these two experiments, the necessary components and conditions of a detailed 3D foot-ankle model for developing an AFO with a training effect were clarified. A model was built and examined with empirical facts, and data were collected from the AFO simulation.