Accelerating gradient descent and Adam via fractional gradients.

We propose a class of novel fractional-order optimization algorithms. We define a fractional-order gradient via the Caputo fractional derivatives that generalizes integer-order gradient. We refer it to as the Caputo fractional-based gradient, and develop an efficient implementation to compute it. A general class of fractional-order optimization methods is then obtained by replacing integer-order gradients with the Caputo fractional-based gradients. To give concrete algorithms, we consider gradient descent (GD) and Adam, and extend them to the Caputo fractional GD (CfGD) and the Caputo fractional Adam (CfAdam). We demonstrate the superiority of CfGD and CfAdam on several large scale optimization problems that arise from scientific machine learning applications, such as ill-conditioned least squares problem on real-world data and the training of neural networks involving non-convex objective functions. Numerical examples show that both CfGD and CfAdam result in acceleration over GD and Adam, respectively. We also derive error bounds of CfGD for quadratic functions, which further indicate that CfGD could mitigate the dependence on the condition number in the rate of convergence and results in significant acceleration over GD.

Periodic Artifact Removal With Applications to Deep Brain Stimulation.

Deep brain stimulation (DBS) therapies have shown clinical success in the treatment of a number of neurological illnesses, including obsessive-compulsive disorder, epilepsy, and Parkinson's disease. An emerging strategy for increasing the efficacy of DBS therapies is to develop closed-loop, adaptive DBS systems that can sense biomarkers associated with particular symptoms and in response, adjust DBS parameters in real-time. The development of such systems requires extensive analysis of the underlying neural signals while DBS is on, so that candidate biomarkers can be identified and the effects of varying the DBS parameters can be better understood. However, DBS creates high amplitude, high frequency stimulation artifacts that prevent the underlying neural signals and thus the biological mechanisms underlying DBS from being analyzed. Additionally, DBS devices often require low sampling rates, which alias the artifact frequency, and rely on wireless data transmission methods that can create signal recordings with missing data of unknown length. Thus, traditional artifact removal methods cannot be applied to this setting. We present a novel periodic artifact removal algorithm for DBS applications that can accurately remove stimulation artifacts in the presence of missing data and in some cases where the stimulation frequency exceeds the Nyquist frequency. The numerical examples suggest that, if implemented on dedicated hardware, this algorithm has the potential to be used in embedded closed-loop DBS therapies to remove DBS stimulation artifacts and hence, to aid in the discovery of candidate biomarkers in real-time. Code for our proposed algorithm is publicly available on Github.

SAR image regularization with fast approximate discrete minimization.

Synthetic aperture radar (SAR) images, like other coherent imaging modalities, suffer from speckle noise. The presence of this noise makes the automatic interpretation of images a challenging task and noise reduction is often a prerequisite for successful use of classical image processing algorithms. Numerous approaches have been proposed to filter speckle noise. Markov random field (MRF) modelization provides a convenient way to express both data fidelity constraints and desirable properties of the filtered image. In this context, total variation minimization has been extensively used to constrain the oscillations in the regularized image while preserving its edges. Speckle noise follows heavy-tailed distributions, and the MRF formulation leads to a minimization problem involving nonconvex log-likelihood terms. Such a minimization can be performed efficiently by computing minimum cuts on weighted graphs. Due to memory constraints, exact minimization, although theoretically possible, is not achievable on large images required by remote sensing applications. The computational burden of the state-of-the-art algorithm for approximate minimization (namely the alpha -expansion) is too heavy specially when considering joint regularization of several images. We show that a satisfying solution can be reached, in few iterations, by performing a graph-cut-based combinatorial exploration of large trial moves. This algorithm is applied to joint regularization of the amplitude and interferometric phase in urban area SAR images.

Shape-based hand recognition.

The problem of person recognition and verification based on their hand images has been addressed. The system is based on the images of the right hands of the subjects, captured by a flatbed scanner in an unconstrained pose at 45 dpi. In a pre-processing stage of the algorithm, the silhouettes of hand images are registered to a fixed pose, which involves both rotation and translation of the hand and, separately, of the individual fingers. Two feature sets have been comparatively assessed, Hausdorff distance of the hand contours and independent component features of the hand silhouette images. Both the classification and the verification performances are found to be very satisfactory as it was shown that, at least for groups of about five hundred subjects, hand-based recognition is a viable secure access control scheme.