Quantifying Non-Stationarity with Information Theory.

We introduce an index based on information theory to quantify the stationarity of a stochastic process. The index compares on the one hand the information contained in the increment at the time scale τ of the process at time t with, on the other hand, the extra information in the variable at time t that is not present at time t-τ. By varying the scale τ, the index can explore a full range of scales. We thus obtain a multi-scale quantity that is not restricted to the first two moments of the density distribution, nor to the covariance, but that probes the complete dependences in the process. This index indeed provides a measure of the regularity of the process at a given scale. Not only is this index able to indicate whether a realization of the process is stationary, but its evolution across scales also indicates how rough and non-stationary it is. We show how the index behaves for various synthetic processes proposed to model fluid turbulence, as well as on experimental fluid turbulence measurements.

Kullback-Leibler divergence measure of intermittency: Application to turbulence.

For generic systems exhibiting power law behaviors, and hence multiscale dependencies, we propose a simple tool to analyze multifractality and intermittency, after noticing that these concepts are directly related to the deformation of a probability density function from Gaussian at large scales to non-Gaussian at smaller scales. Our framework is based on information theory and uses Shannon entropy and Kullback-Leibler divergence. We provide an extensive application to three-dimensional fully developed turbulence, seen here as a paradigmatic complex system where intermittency was historically defined and the concepts of scale invariance and multifractality were extensively studied and benchmarked. We compute our quantity on experimental Eulerian velocity measurements, as well as on synthetic processes and phenomenological models of fluid turbulence. Our approach is very general and does not require any underlying model of the system, although it can probe the relevance of such a model.

Mutual information for intrapartum fetal heart rate analysis.

The analysis of the temporal dynamics in intrapartum fetal heart rate (FHR), aiming at early detection of fetal acidosis, constitutes an intricate signal processing task, that continuously receives significant research efforts. Entropy and entropy rates, envisaged as measures of complexity, often computed via popular implementations referred to as Approximate Entropy (ApEn) or Sample Entropy (SampEn), have regularly been reported as significant features for intrapartum FHR analysis. The present contribution aims to show how mutual information enhances characterization of FHR temporal dynamics and improves fetal acidosis detection performance. To that end, mutual information is first connected to ApEn and SampEn both conceptually and with respect to estimation procedure. Second, mutual information, ApEn and SampEn are computed on a large (≃ 1000 subjects) and documented database of FHR data, collected in a French academic hospital. Reported results show that the use of mutual information permits to significantly outperform ApEn and SampEn for acidosis detection, during any stage of labor.