Playing the Lottery With Concave Regularizers for Sparse Trainable Neural Networks.

The design of sparse neural networks, i.e., of networks with a reduced number of parameters, has been attracting increasing research attention in the last few years. The use of sparse models may significantly reduce the computational and storage footprint in the inference phase. In this context, the lottery ticket hypothesis (LTH) constitutes a breakthrough result, that addresses not only the performance of the inference phase, but also of the training phase. It states that it is possible to extract effective sparse subnetworks, called winning tickets, that can be trained in isolation. The development of effective methods to play the lottery, i.e., to find winning tickets, is still an open problem. In this article, we propose a novel class of methods to play the lottery. The key point is the use of concave regularization to promote the sparsity of a relaxed binary mask, which represents the network topology. We theoretically analyze the effectiveness of the proposed method in the convex framework. Then, we propose extended numerical tests on various datasets and architectures, that show that the proposed method can improve the performance of state-of-the-art algorithms.

Multiclass Sparse Centroids With Application to Fast Time Series Classification.

In this article, we propose an efficient multiclass classification scheme based on sparse centroids classifiers. The proposed strategy exhibits linear complexity with respect to both the number of classes and the cardinality of the feature space. The classifier we introduce is based on binary space partitioning, performed by a decision tree where the assignation law at each node is defined via a sparse centroid classifier. We apply the presented strategy to the time series classification problem, showing by experimental evidence that it achieves performance comparable to that of state-of-the-art methods, but with a significantly lower classification time. The proposed technique can be an effective option in resource-constrained environments where the classification time and the computational cost are critical or, in scenarios, where real-time classification is necessary.

Sparse ℓ1- and ℓ2-Center Classifiers.

In this article, we discuss two novel sparse versions of the classical nearest-centroid classifier. The proposed sparse classifiers are based on l1 and l2 distance criteria, respectively, and perform simultaneous feature selection and classification, by detecting the features that are most relevant for the classification purpose. We formally prove that the training of the proposed sparse models, with both distance criteria, can be performed exactly (i.e., the globally optimal set of features is selected) at a linear computational cost. Especially, the proposed sparse classifiers are trained in O(mn)+O(mlogk) operations, where n is the number of samples, m is the total number of features, and k ≤ m is the number of features to be retained in the classifier. Furthermore, the complexity of testing and classifying a new sample is simply O(k) for both methods. The proposed models can be employed either as stand-alone sparse classifiers or fast feature-selection techniques for prefiltering the features to be later fed to other types of classifiers (e.g., SVMs). The experimental results show that the proposed methods are competitive in accuracy with state-of-the-art feature selection and classification techniques while having a substantially lower computational cost.

A model predictive control approach to optimally devise a two-dose vaccination rollout: A case study on COVID-19 in Italy.

The COVID-19 pandemic has led to the unprecedented challenge of devising massive vaccination rollouts, toward slowing down and eventually extinguishing the diffusion of the virus. The two-dose vaccination procedure, speed requirements, and the scarcity of doses, suitable spaces, and personnel, make the optimal design of such rollouts a complex problem. Mathematical modeling, which has already proved to be determinant in the early phases of the pandemic, can again be a powerful tool to assist public health authorities in optimally planning the vaccination rollout. Here, we propose a novel epidemic model tailored to COVID-19, which includes the effect of nonpharmaceutical interventions and a concurrent two-dose vaccination campaign. Then, we leverage nonlinear model predictive control to devise optimal scheduling of first and second doses, accounting both for the healthcare needs and for the socio-economic costs associated with the epidemics. We calibrate our model to the 2021 COVID-19 vaccination campaign in Italy. Specifically, once identified the epidemic parameters from officially reported data, we numerically assess the effectiveness of the obtained optimal vaccination rollouts for the two most used vaccines. Determining the optimal vaccination strategy is nontrivial, as it depends on the efficacy and duration of the first-dose partial immunization, whereby the prioritization of first doses and the delay of second doses may be effective for vaccines with sufficiently strong first-dose immunization. Our model and optimization approach provide a flexible tool that can be adopted to help devise the current COVID-19 vaccination campaign, and increase preparedness for future epidemics.

Output Feedback Q-Learning for Linear-Quadratic Discrete-Time Finite-Horizon Control Problems.

An algorithm is proposed to determine output feedback policies that solve finite-horizon linear-quadratic (LQ) optimal control problems without requiring knowledge of the system dynamical matrices. To reach this goal, the Q -factors arising from finite-horizon LQ problems are first characterized in the state feedback case. It is then shown how they can be parameterized as functions of the input-output vectors. A procedure is then proposed for estimating these functions from input/output data and using these estimates for computing the optimal control via the measured inputs and outputs.

A time-varying SIRD model for the COVID-19 contagion in Italy.

The purpose of this work is to give a contribution to the understanding of the COVID-19 contagion in Italy. To this end, we developed a modified Susceptible-Infected-Recovered-Deceased (SIRD) model for the contagion, and we used official data of the pandemic for identifying the parameters of this model. Our approach features two main non-standard aspects. The first one is that model parameters can be time-varying, allowing us to capture possible changes of the epidemic behavior, due for example to containment measures enforced by authorities or modifications of the epidemic characteristics and to the effect of advanced antiviral treatments. The time-varying parameters are written as linear combinations of basis functions and are then inferred from data using sparse identification techniques. The second non-standard aspect resides in the fact that we consider as model parameters also the initial number of susceptible individuals, as well as the proportionality factor relating the detected number of positives with the actual (and unknown) number of infected individuals. Identifying the model parameters amounts to a non-convex identification problem that we solve by means of a nested approach, consisting in a one-dimensional grid search in the outer loop, with a Lasso optimization problem in the inner step.

A new metric for understanding hidden political influences from voting records.

Inspired by the increasing attention of the scientific community towards the understanding of human relationships and actions in social sciences, in this paper we address the problem of inferring from voting data the hidden influence on individuals from competing ideology groups. As a case study, we present an analysis of the closeness of members of the Italian Senate to political parties during the XVII Legislature. The proposed approach is aimed at automatic extraction of the relevant information by disentangling the actual influences from noise, via a two step procedure. First, a sparse principal component projection is performed on the standardized voting data. Then, the projected data is combined with a generative mixture model, and an information theoretic measure, which we refer to as Political Data-aNalytic Affinity (Political DNA), is finally derived. We show that the definition of this new affinity measure, together with suitable visualization tools for displaying the results of analysis, allows a better understanding and interpretability of the relationships among political groups.

A Universal Approximation Result for Difference of Log-Sum-Exp Neural Networks.

We show that a neural network whose output is obtained as the difference of the outputs of two feedforward networks with exponential activation function in the hidden layer and logarithmic activation function in the output node, referred to as log-sum-exp (LSE) network, is a smooth universal approximator of continuous functions over convex, compact sets. By using a logarithmic transform, this class of network maps to a family of subtraction-free ratios of generalized posynomials (GPOS), which we also show to be universal approximators of positive functions over log-convex, compact subsets of the positive orthant. The main advantage of difference-LSE networks with respect to classical feedforward neural networks is that, after a standard training phase, they provide surrogate models for a design that possesses a specific difference-of-convex-functions form, which makes them optimizable via relatively efficient numerical methods. In particular, by adapting an existing difference-of-convex algorithm to these models, we obtain an algorithm for performing an effective optimization-based design. We illustrate the proposed approach by applying it to the data-driven design of a diet for a patient with type-2 diabetes and to a nonconvex optimization problem.

Log-Sum-Exp Neural Networks and Posynomial Models for Convex and Log-Log-Convex Data.

In this paper, we show that a one-layer feedforward neural network with exponential activation functions in the inner layer and logarithmic activation in the output neuron is a universal approximator of convex functions. Such a network represents a family of scaled log-sum exponential functions, here named log-sum-exp ( LSET ). Under a suitable exponential transformation, the class of LSET functions maps to a family of generalized posynomials GPOST , which we similarly show to be universal approximators for log-log-convex functions. A key feature of an LSET network is that, once it is trained on data, the resulting model is convex in the variables, which makes it readily amenable to efficient design based on convex optimization. Similarly, once a GPOST model is trained on data, it yields a posynomial model that can be efficiently optimized with respect to its variables by using geometric programming (GP). The proposed methodology is illustrated by two numerical examples, in which, first, models are constructed from simulation data of the two physical processes (namely, the level of vibration in a vehicle suspension system, and the peak power generated by the combustion of propane), and then optimization-based design is performed on these models.