Leveraging VQ-VAE tokenization for autoregressive modeling of medical time series.

In this work, we present CodeAR, a medical time series generative model for electronic health record (EHR) synthesis. CodeAR employs autoregressive modeling on discrete tokens obtained using a vector quantized-variational autoencoder (VQ-VAE), which addresses key challenges of accurate distribution modeling and patient privacy preservation in the medical domain. The proposed model is trained with next-token prediction instead of a regression problem for more accurate distribution modeling, where the autoregressive property of CodeAR is useful to capture the inherent causality in time series data. In addition, the compressive property of the VQ-VAE prevents CodeAR from memorizing the original training data, which ensures patient privacy. Experimental results demonstrate that CodeAR outperforms the baseline autoregressive-based and GAN-based models in terms of maximum mean discrepancy (MMD) and Train on Synthetic, Test on Real tests. Our results highlight the effectiveness of autoregressive modeling on discrete tokens, the utility of CodeAR in causal modeling, and its robustness against data memorization.

Examining sea levels forecasting using autoregressive and prophet models.

Global climate change in recent years has resulted in significant changes in sea levels at both global and local scales. Various oceanic and climatic factors play direct and indirect roles in influencing sea level changes, such as temperature, ocean heat, and Greenhouse gases (GHG) emissions. This study examined time series analysis models, specifically Autoregressive Moving Average (ARIMA) and Facebook's prophet, in forecasting the Global Mean Sea Level (GMSL). Additionally, Vector Autoregressive (VAR) model was utilized to investigate the influence of selected oceanic and climatic factors contributing to sea level rise, including ocean heat, air temperature, and GHG emissions. Moreover, the models were applied to regional sea level data from the Arabian Gulf, which experienced higher fluctuations compared to GMSL. Results showed the capability of autoregressive models in long-term forecasting, while the Prophet model excelled in capturing trends and patterns in the time series over extended periods of time.

A Semiparametric Bayesian Approach to Heterogeneous Spatial Autoregressive Models.

Many semiparametric spatial autoregressive (SSAR) models have been used to analyze spatial data in a variety of applications; however, it is a common phenomenon that heteroscedasticity often occurs in spatial data analysis. Therefore, when considering SSAR models in this paper, it is allowed that the variance parameters of the models can depend on the explanatory variable, and these are called heterogeneous semiparametric spatial autoregressive models. In order to estimate the model parameters, a Bayesian estimation method is proposed for heterogeneous SSAR models based on B-spline approximations of the nonparametric function. Then, we develop an efficient Markov chain Monte Carlo sampling algorithm on the basis of the Gibbs sampler and Metropolis-Hastings algorithm that can be used to generate posterior samples from posterior distributions and perform posterior inference. Finally, some simulation studies and real data analysis of Boston housing data have demonstrated the excellent performance of the proposed Bayesian method.

A Multilevel Spatial Model to Investigate Voting Behaviour in the 2019 UK General Election.

This paper presents a modelling framework which can detect the simultaneous presence of two different types of spatial process. The first is the variation from a global mean resulting from a geographical unit's 'vertical' position within a nested hierarchical structure such as the county and region where it is situated. The second is the variation at the smaller scale of individual units due to the 'horizontal' influence of nearby locations. The former is captured using a multi-level modelling structure while the latter is accounted for by an autoregressive component at the lowest level of the hierarchy. Such a model not only estimates spatially-varying parameters according to geographical scale, but also the relative contribution of each process to the overall spatial variation. As a demonstration, the study considers the association of a selection of socio-economic attributes with voting behaviour in the 2019 UK general election. It finds evidence of the presence of both types of spatial effects, and describes how they suggest different associations between census profile and voting behaviour in different parts of England and Wales.

A new sample-size planning approach for person-specific VAR(1) studies: Predictive accuracy analysis.

Researchers increasingly study short-term dynamic processes that evolve within single individuals using N = 1 studies. The processes of interest are typically captured by fitting a VAR(1) model to the resulting data. A crucial question is how to perform sample-size planning and thus decide on the number of measurement occasions that are needed. The most popular approach is to perform a power analysis, which focuses on detecting the effects of interest. We argue that performing sample-size planning based on out-of-sample predictive accuracy yields additional important information regarding potential overfitting of the model. Predictive accuracy quantifies how well the estimated VAR(1) model will allow predicting unseen data from the same individual. We propose a new simulation-based sample-size planning method called predictive accuracy analysis (PAA), and an associated Shiny app. This approach makes use of a novel predictive accuracy metric that accounts for the multivariate nature of the prediction problem. We showcase how the values of the different VAR(1) model parameters impact power and predictive accuracy-based sample-size recommendations using simulated data sets and real data applications. The range of recommended sample sizes is smaller for predictive accuracy analysis than for power analysis.

Impact of urban form and street infrastructure on pedestrian-motorist collisions.

This study examines the impact of urban form and street infrastructure on pedestrian safety in Atlanta, Georgia, and Boston, Massachusetts. With a significant rise in pedestrian fatalities over the past decade, understanding how cities' built environments influence safety is critical. We conducted geospatial analyses and statistical tests, revealing unique patterns in each city. Atlanta's sprawling, motorist-oriented layout is associated with increased pedestrian accidents, particularly at crosswalks, due to limited land use diversity, arterial roads, and streets with high speed limits and multiple lanes. In contrast, Boston's compact, pedestrian-oriented design leads to improved safety, featuring safer pedestrian crossings, greater land use diversity, reduced arterial roads and lower speed limits on single-lane streets. This study also highlights the importance of diverse urban forms and pedestrian infrastructure in shaping pedestrian safety. While population density and land use diversity impact accident rates, the presence of crosswalks and street configurations play crucial roles. Our findings underscore the urgency for urban planners to prioritize pedestrian safety through targeted interventions, such as enhancing crosswalks, reducing speed limits and promoting mixed land use. Future research should explore additional variables, alternative modelling techniques and non-linear approaches to gain a more comprehensive understanding of these complex relationships.

Intraindividual time-varying dynamic network of affects: linear autoregressive mixed-effects models for ecological momentary assessment.

An interesting recent development in emotion research and clinical psychology is the discovery that affective states can be modeled as a network of temporally interacting moods or emotions. Additionally, external factors like stressors or treatments can influence the mood network by amplifying or dampening the activation of specific moods. Researchers have turned to multilevel autoregressive models to fit these affective networks using intensive longitudinal data gathered through ecological momentary assessment. Nonetheless, a more comprehensive examination of the performance of such models is warranted. In our study, we focus on simple directed intraindividual networks consisting of two interconnected mood nodes that mutually enhance or dampen each other. We also introduce a node representing external factors that affect both mood nodes unidirectionally. Importantly, we disregard the potential effects of a current mood/emotion on the perception of external factors. We then formalize the mathematical representation of such networks by exogenous linear autoregressive mixed-effects models. In this representation, the autoregressive coefficients signify the interactions between moods, while external factors are incorporated as exogenous covariates. We let the autoregressive and exogenous coefficients in the model have fixed and random components. Depending on the analysis, this leads to networks with variable structures over reasonable time units, such as days or weeks, which are captured by the variability of random effects. Furthermore, the fixed-effects parameters encapsulate a subject-specific network structure. Leveraging the well-established theoretical and computational foundation of linear mixed-effects models, we transform the autoregressive formulation to a classical one and utilize the existing methods and tools. To validate our approach, we perform simulations assuming our model as the true data-generating process. By manipulating a predefined set of parameters, we investigate the reliability and feasibility of our approach across varying numbers of observations, levels of noise intensity, compliance rates, and scalability to higher dimensions. Our findings underscore the challenges associated with estimating individualized parameters in the context of common longitudinal designs, where the required number of observations may often be unattainable. Moreover, our study highlights the sensitivity of autoregressive mixed-effect models to noise levels and the difficulty of scaling due to the substantial number of parameters.

A Time-Varying Mixture Integer-Valued Threshold Autoregressive Process Driven by Explanatory Variables.

In this paper, a time-varying first-order mixture integer-valued threshold autoregressive process driven by explanatory variables is introduced. The basic probabilistic and statistical properties of this model are studied in depth. We proceed to derive estimators using the conditional least squares (CLS) and conditional maximum likelihood (CML) methods, while also establishing the asymptotic properties of the CLS estimator. Furthermore, we employed the CLS and CML score functions to infer the threshold parameter. Additionally, three test statistics to detect the existence of the piecewise structure and explanatory variables were utilized. To support our findings, we conducted simulation studies and applied our model to two applications concerning the daily stock trading volumes of VOW.

Vibration-Based SHM in the Synthetic Mooring Lines of the Semisubmersible OO-Star Wind Floater under Varying Environmental and Operational Conditions.

As the industry transitions toward Floating Offshore Wind Turbines (FOWT) in greater depths, conventional chain mooring lines become impractical, prompting the adoption of synthetic fiber ropes. Despite their advantages, these mooring lines present challenges in inspection due to their exterior jacket, which prevents visual assessment. The current study focuses on vibration-based Structural Health Monitoring (SHM) in FOWT synthetic mooring lines under uncertainty arising from varying Environmental and Operational Conditions (EOCs). Six damage detection methods are assessed, utilizing either multiple models or a single functional model. The methods are based on Vector Autoregressive (VAR) or Transmittance Function Autoregressive with exogenous input (TF-ARX) models. All methods are evaluated through a Monte Carlo study involving 1100 simulations, utilizing acceleration signals generated from a finite element model of the OO-Star Wind Floater Semi 10 MW wind turbine. With signals from only two measuring positions, the methods demonstrate excellent results, detecting the stiffness reduction of a mooring line at levels 10% through 50%. The methods are also tested for healthy cases, with those utilizing TF-ARX models achieving zero false alarms, even for EOCs not encountered in the training data.

Time-varying information measures: an adaptive estimation of information storage with application to brain-heart interactions.

Network Physiology is a rapidly growing field of study that aims to understand how physiological systems interact to maintain health. Within the information theory framework the information storage (IS) allows to measure the regularity and predictability of a dynamic process under stationarity assumption. However, this assumption does not allow to track over time the transient pathways occurring in the dynamical activity of a physiological system. To address this limitation, we propose a time-varying approach based on the recursive least squares algorithm (RLS) for estimating IS at each time instant, in non-stationary conditions. We tested this approach in simulated time-varying dynamics and in the analysis of electroencephalographic (EEG) signals recorded from healthy volunteers and timed with the heartbeat to investigate brain-heart interactions. In simulations, we show that the proposed approach allows to track both abrupt and slow changes in the information stored in a physiological system. These changes are reflected in its evolution and variability over time. The analysis of brain-heart interactions reveals marked differences across the cardiac cycle phases of the variability of the time-varying IS. On the other hand, the average IS values exhibit a weak modulation over parieto-occiptal areas of the scalp. Our study highlights the importance of developing more advanced methods for measuring IS that account for non-stationarity in physiological systems. The proposed time-varying approach based on RLS represents a useful tool for identifying spatio-temporal dynamics within the neurocardiac system and can contribute to the understanding of brain-heart interactions.

Diffusion Probabilistic Modeling for Video Generation.

Denoising diffusion probabilistic models are a promising new class of generative models that mark a milestone in high-quality image generation. This paper showcases their ability to sequentially generate video, surpassing prior methods in perceptual and probabilistic forecasting metrics. We propose an autoregressive, end-to-end optimized video diffusion model inspired by recent advances in neural video compression. The model successively generates future frames by correcting a deterministic next-frame prediction using a stochastic residual generated by an inverse diffusion process. We compare this approach against six baselines on four datasets involving natural and simulation-based videos. We find significant improvements in terms of perceptual quality and probabilistic frame forecasting ability for all datasets.

Robust clustering of COVID-19 cases across U.S. counties using mixtures of asymmetric time series models with time varying and freely indexed covariates.

In this paper, we develop a mixture of autoregressive (MoAR) process model with time varying and freely indexed covariates under the flexible class of two-piece distributions using the scale mixtures of normal (TP-SMN) family. This novel family of time series (TP-SMN-MoAR) models was used to examine flexible and robust clustering of reported cases of Covid-19 across 313 counties in the U.S. The TP-SMN distributions allow for symmetrical/ asymmetrical distributions as well as heavy-tailed distributions providing for flexibility to handle outliers and complex data. Developing a suitable hierarchical representation of the TP-SMN family enabled the construction of a pseudo-likelihood function to derive the maximum pseudo-likelihood estimates via an EM-type algorithm.

Determinants of Pneumonia mortality in Bogota, Colombia: A spatial econometrics approach.

Bogota, the capital and largest city of Colombia, constantly fights against easily transmitted and endemic-epidemic diseases that lead to enormous public health problems. Pneumonia is currently the leading cause of mortality attributable respiratory infections in the city. Its recurrence and impact have been partially explained by biological, medical, and behavioural factors. Against this background, this study investigates Pneumonia mortality rates in Bogota from 2004 and 2014. We identified a set of environmental, socioeconomic, behavioural, and medical care factors whose interaction in space could explain the occurrence and impact of the disease in the Iberoamerican city. We adopted a spatial autoregressive models framework to study the spatial dependence and heterogeneity of Pneumonia mortality rates associated with well-known risk factors. The results highlight the different types of spatial processes governing Pneumonia mortality. Furthermore, they demonstrate and quantify the driving factors that stimulate the spatial spread and clustering of mortality rates. Our study stresses the importance of spatial modelling of context-dependent diseases such as Pneumonia. Likewise, we emphasize the need to develop comprehensive public health policies that consider the space and contextual factors.

AI enhanced collaborative human-machine interactions for home-based telerehabilitation.

The use of robots in a telerehabilitation paradigm could facilitate the delivery of rehabilitation on demand while reducing transportation time and cost. As a result, it helps to motivate patients to exercise frequently in a more comfortable home environment. However, for such a paradigm to work, it is essential that the robustness of the system is not compromised due to network latency, jitter, and delay of the internet. This paper proposes a solution to data loss compensation to maintain the quality of the interaction between the user and the system. Data collected from a well-defined collaborative task using a virtual reality (VR) environment was used to train a robotic system to adapt to the users' behaviour. The proposed approach uses nonlinear autoregressive models with exogenous input (NARX) and long-short term memory (LSTM) neural networks to smooth out the interaction between the user and the predicted movements generated from the system. LSTM neural networks are shown to learn to act like an actual human. The results from this paper have shown that, with an appropriate training method, the artificial predictor can perform very well by allowing the predictor to complete the task within 25 s versus 23 s when executed by the human.

Generating High-Resolution 3D Faces and Bodies Using VQ-VAE-2 with PixelSNAIL Networks on 2D Representations.

Modeling and representing 3D shapes of the human body and face is a prominent field due to its applications in the healthcare, clothes, and movie industry. In our work, we tackled the problem of 3D face and body synthesis by reducing 3D meshes to 2D image representations. We show that the face can naturally be modeled on a 2D grid. At the same time, for more challenging 3D body geometries, we proposed a novel non-bijective 3D-2D conversion method representing the 3D body mesh as a plurality of rendered projections on the 2D grid. Then, we trained a state-of-the-art vector-quantized variational autoencoder (VQ-VAE-2) to learn a latent representation of 2D images and fit a PixelSNAIL autoregressive model to sample novel synthetic meshes. We evaluated our method versus a classical one based on principal component analysis (PCA) by sampling from the empirical cumulative distribution of the PCA scores. We used the empirical distributions of two commonly used metrics, specificity and diversity, to quantitatively demonstrate that the synthetic faces generated with our method are statistically closer to real faces when compared with the PCA ones. Our experiment on the 3D body geometry requires further research to match the test set statistics but shows promising results.

Environmental and topographic drivers of amphibian phylogenetic diversity and endemism in the Iberian Peninsula.

Understanding the ecological and evolutionary processes driving biodiversity patterns and allowing their persistence is of utmost importance. Many hypotheses have been proposed to explain spatial diversity patterns, including water-energy availability, habitat heterogeneity, and historical climatic refugia. The main goal of this study is to identify if general spatial drivers of species diversity patterns of phylogenetic diversity (PD) and phylogenetic endemism (PE) at the global scale are also predictive of PD and PE at regional scales, using Iberian amphibians as a case study. Our main hypothesis assumes that topography along with contemporary and historical climate are drivers of phylogenetic diversity and endemism, but that the strength of these predictors may be weaker at the regional scale than it tends to be at the global scale. We mapped spatial patterns of Iberian amphibians' phylogenetic diversity and endemism, using previously published phylogenetic and distribution data. Furthermore, we compiled spatial data on topographic and climatic variables related to the water-energy availability, topography, and historical climatic instability hypotheses. To test our hypotheses, we used Spatial Autoregressive Models and selected the best model to explain diversity patterns based on Akaike Information Criterion. Our results show that, out of the variables tested in our study, water-energy availability and historical climate instability are the most important drivers of amphibian diversity in Iberia. However, as predicted, the strength of these predictors in our case study is weaker than it tends to be at global scales. Thus, additional drivers should also be investigated and we suggest caution when interpreting these predictors as surrogates for different components of diversity.

Autoregressive models in environmental forecasting time series: a theoretical and application review.

Though globalization, industrialization, and urbanization have escalated the economic growth of nations, these activities have played foul on the environment. Better understanding of ill effects of these activities on environment and human health and taking appropriate control measures in advance are the need of the hour. Time series analysis can be a great tool in this direction. ARIMA model is the most popular accepted time series model. It has numerous applications in various domains due its high mathematical precision, flexible nature, and greater reliable results. ARIMA and environment are highly correlated. Though there are many research papers on application of ARIMA in various fields including environment, there is no substantial work that reviews the building stages of ARIMA. In this regard, the present work attempts to present three different stages through which ARIMA was evolved. More than 100 papers are reviewed in this study to discuss the application part based on pure ARIMA and its hybrid modeling with special focus in the field of environment/health/air quality. Forecasting in this field can be a great contributor to governments and public at large in taking all the required precautionary steps in advance. After such a massive review of ARIMA and hybrid modeling involving ARIMA in the fields including or excluding environment/health/atmosphere, it can be concluded that the combined models are more robust and have higher ability to capture all the patterns of the series uniformly. Thus, combining several models or using hybrid model has emerged as a routinized custom.

A Bayesian multivariate mixture model for high throughput spatial transcriptomics.

High throughput spatial transcriptomics (HST) is a rapidly emerging class of experimental technologies that allow for profiling gene expression in tissue samples at or near single-cell resolution while retaining the spatial location of each sequencing unit within the tissue sample. Through analyzing HST data, we seek to identify sub-populations of cells within a tissue sample that may inform biological phenomena. Existing computational methods either ignore the spatial heterogeneity in gene expression profiles, fail to account for important statistical features such as skewness, or are heuristic-based network clustering methods that lack the inferential benefits of statistical modeling. To address this gap, we develop SPRUCE: a Bayesian spatial multivariate finite mixture model based on multivariate skew-normal distributions, which is capable of identifying distinct cellular sub-populations in HST data. We further implement a novel combination of Pólya-Gamma data augmentation and spatial random effects to infer spatially correlated mixture component membership probabilities without relying on approximate inference techniques. Via a simulation study, we demonstrate the detrimental inferential effects of ignoring skewness or spatial correlation in HST data. Using publicly available human brain HST data, SPRUCE outperforms existing methods in recovering expertly annotated brain layers. Finally, our application of SPRUCE to human breast cancer HST data indicates that SPRUCE can distinguish distinct cell populations within the tumor microenvironment. An R package spruce for fitting the proposed models is available through The Comprehensive R Archive Network.

A novel method for efficient estimation of brain effective connectivity in EEG.

BACKGROUND AND OBJECTIVE: Brain connectivity plays a pivotal role in understanding the brain's information processing functions by providing various details including magnitude, direction, and temporal dynamics of inter-neuron connections. While the connectivity may be classified as structural, functional and causal, a complete in-vivo directional analysis is guaranteed by the latter and is referred to as Effective Connectivity (EC). Two most widely used EC techniques are Directed Transfer Function (DTF) and Partial Directed Coherence (PDC) which are based on multivariate autoregressive models. The drawbacks of these techniques include poor frequency resolution and the requirement for experimental approach to determine signal normalization and thresholding techniques in identifying significant connectivities between multivariate sources. METHODS: In this study, the drawbacks of DTF and PDC are addressed by proposing a novel technique, termed as Efficient Effective Connectivity (EEC), for the estimation of EC between multivariate sources using AR spectral estimation and Granger causality principle. In EEC, a linear predictive filter with AR coefficients obtained via multivariate EEG is used for signal prediction. This leads to the estimation of full-length signals which are then transformed into frequency domain by using Burg spectral estimation method. Furthermore, the newly proposed normalization method addressed the effect on each source in EEC using the sum of maximum connectivity values over the entire frequency range. Lastly, the proposed dynamic thresholding works by subtracting the first moment of causal effects of all the sources on one source from individual connections present for that source. RESULTS: The proposed method is evaluated using synthetic and real resting-state EEG of 46 healthy controls. A 3D-Convolutional Neural Network is trained and tested using the PDC and EEC samples. The result indicates that compared to PDC, EEC improves the EEG eye-state classification accuracy, sensitivity and specificity by 5.57%, 3.15% and 8.74%, respectively. CONCLUSION: Correct identification of all connections in synthetic data and improved resting-state classification performance using EEC proved that EEC gives better estimation of directed causality and indicates that it can be used for reliable understanding of brain mechanisms. Conclusively, the proposed technique may open up new research dimensions for clinical diagnosis of mental disorders.

Revisiting Gaussian Markov random fields and Bayesian disease mapping.

We revisit several conditionally formulated Gaussian Markov random fields, known as the intrinsic conditional autoregressive model, the proper conditional autoregressive model, and the Leroux et al. conditional autoregressive model, as well as convolution models such as the well known Besag, York and Mollie model, its (adaptive) re-parameterization, and its scaled alternatives, for their roles of modelling underlying spatial risks in Bayesian disease mapping. Analytic and simulation studies, with graphic visualizations, and disease mapping case studies, present insights and critique on these models for their nature and capacities in characterizing spatial dependencies, local influences, and spatial covariance and correlation functions, and in facilitating stabilized and efficient posterior risk prediction and inference. It is illustrated that these models are Gaussian (Markov) random fields of different spatial dependence, local influence, and (covariance) correlation functions and can play different and complementary roles in Bayesian disease mapping applications.