EgPDE-Net: Building Continuous Neural Networks for Time Series Prediction With Exogenous Variables.

While exogenous variables have a major impact on performance improvement in time series analysis, interseries correlation and time dependence among them are rarely considered in the present continuous methods. The dynamical systems of multivariate time series could be modeled with complex unknown partial differential equations (PDEs) which play a prominent role in many disciplines of science and engineering. In this article, we propose a continuous-time model for arbitrary-step prediction to learn an unknown PDE system in multivariate time series whose governing equations are parameterized by self-attention and gated recurrent neural networks. The proposed model, exogenous-guided PDE network (EgPDE-Net), takes account of the relationships among the exogenous variables and their effects on the target series. Importantly, the model can be reduced into a regularized ordinary differential equation (ODE) problem with specially designed regularization guidance, which makes the PDE problem tractable to obtain numerical solutions and feasible to predict multiple future values of the target series at arbitrary time points. Extensive experiments demonstrate that our proposed model could achieve competitive accuracy over strong baselines: on average, it outperforms the best baseline by reducing 9.85% on RMSE and 13.98% on MAE for arbitrary-step prediction.

Generalized image outpainting with U-transformer.

In this paper, we develop a novel transformer-based generative adversarial neural network called U-Transformer for generalized image outpainting problems. Different from most present image outpainting methods conducting horizontal extrapolation, our generalized image outpainting could extrapolate visual context all-side around a given image with plausible structure and details even for complicated scenery, building, and art images. Specifically, we design a generator as an encoder-to-decoder structure embedded with the popular Swin Transformer blocks. As such, our novel neural network can better cope with image long-range dependencies which are crucially important for generalized image outpainting. We propose additionally a U-shaped structure and multi-view Temporal Spatial Predictor (TSP) module to reinforce image self-reconstruction as well as unknown-part prediction smoothly and realistically. By adjusting the predicting step in the TSP module in the testing stage, we can generate arbitrary outpainting size given the input sub-image. We experimentally demonstrate that our proposed method could produce visually appealing results for generalized image outpainting against the state-of-the-art image outpainting approaches.

Densest plane group packings of regular polygons.

Packings of regular convex polygons (n-gons) that are sufficiently dense have been studied extensively in the context of modeling physical and biological systems as well as discrete and computational geometry. Former results were mainly regarding densest lattice or double-lattice configurations. Here we consider all two-dimensional crystallographic symmetry groups (plane groups) by restricting the configuration space of the general packing problem of congruent copies of a compact subset of the two-dimensional Euclidean space to particular isomorphism classes of the discrete group of isometries. We formulate the plane group packing problem as a nonlinear constrained optimization problem. By means of the Entropic Trust Region Packing Algorithm that approximately solves this problem, we examine some known and unknown densest packings of various n-gons in all 17 plane groups and state conjectures about common symmetries of the densest plane group packings for every n-gon.

Discriminative Triad Matching and Reconstruction for Weakly Referring Expression Grounding.

In this paper, we are tackling the weakly-supervised referring expression grounding task, for the localization of a referent object in an image according to a query sentence, where the mapping between image regions and queries are not available during the training stage. In traditional methods, an object region that best matches the referring expression is picked out, and then the query sentence is reconstructed from the selected region, where the reconstruction difference serves as the loss for back-propagation. The existing methods, however, conduct both the matching and the reconstruction approximately as they ignore the fact that the matching correctness is unknown. To overcome this limitation, a discriminative triad is designed here as the basis to the solution, through which a query can be converted into one or multiple discriminative triads in a very scalable way. Based on the discriminative triad, we further propose the triad-level matching and reconstruction modules which are lightweight yet effective for the weakly-supervised training, making it three times lighter and faster than the previous state-of-the-art methods. One important merit of our work is its superior performance despite the simple and neat design. Specifically, the proposed method achieves a new state-of-the-art accuracy when evaluated on RefCOCO (39.21 percent), RefCOCO+ (39.18 percent) and RefCOCOg (43.24 percent) datasets, that is 4.17, 4.08 and 7.8 percent higher than the previous one, respectively. The code is available at https://github.com/insomnia94/DTWREG.

An Interpretable Deep Architecture for Similarity Learning Built Upon Hierarchical Concepts.

In general, development of adequately complex mathematical models, such as deep neural networks, can be an effective way to improve the accuracy of learning models. However, this is achieved at the cost of reduced post-hoc model interpretability, because what is learned by the model can become less intelligible and tractable to humans as the model complexity increases. In this paper, we target a similarity learning task in the context of image retrieval, with a focus on the model interpretability issue. An effective similarity neural network (SNN) is proposed to offer not only to seek robust retrieval performance but also to achieve satisfactory post-hoc interpretability. The network is designed by linking the neuron architecture with the organization of a concept tree and by formulating neuron operations to pass similarity information between concepts. Various ways of understanding and visualizing what is learned by the SNN neurons are proposed. We also exhaustively evaluate the proposed approach using a number of relevant datasets against a number of state-of-the-art approaches to demonstrate the effectiveness of the proposed network. Our results show that the proposed approach can offer superior performance when compared against state-of-the-art approaches. Neuron visualization results are demonstrated to support the understanding of the trained neurons.

Unsupervised t-Distributed Video Hashing and Its Deep Hashing Extension.

In this paper, a novel unsupervised hashing algorithm, referred to as t-USMVH, and its extension to unsupervised deep hashing, referred to as t-UDH, are proposed to support large-scale video-to-video retrieval. To improve robustness of the unsupervised learning, the t-USMVH combines multiple types of feature representations and effectively fuses them by examining a continuous relevance score based on a Gaussian estimation over pairwise distances, and also a discrete neighbor score based on the cardinality of reciprocal neighbors. To reduce sensitivity to scale changes for mapping objects that are far apart from each other, Student t-distribution is used to estimate the similarity between the relaxed hash code vectors for keyframes. This results in more accurate preservation of the desired unsupervised similarity structure in the hash code space. By adapting the corresponding optimization objective and constructing the hash mapping function via a deep neural network, we develop a robust unsupervised training strategy for a deep hashing network. The efficiency and effectiveness of the proposed methods are evaluated on two public video collections via comparisons against multiple classical and the state-of-the-art methods.

A semi-supervised approach using label propagation to support citation screening.

Citation screening, an integral process within systematic reviews that identifies citations relevant to the underlying research question, is a time-consuming and resource-intensive task. During the screening task, analysts manually assign a label to each citation, to designate whether a citation is eligible for inclusion in the review. Recently, several studies have explored the use of active learning in text classification to reduce the human workload involved in the screening task. However, existing approaches require a significant amount of manually labelled citations for the text classification to achieve a robust performance. In this paper, we propose a semi-supervised method that identifies relevant citations as early as possible in the screening process by exploiting the pairwise similarities between labelled and unlabelled citations to improve the classification performance without additional manual labelling effort. Our approach is based on the hypothesis that similar citations share the same label (e.g., if one citation should be included, then other similar citations should be included also). To calculate the similarity between labelled and unlabelled citations we investigate two different feature spaces, namely a bag-of-words and a spectral embedding based on the bag-of-words. The semi-supervised method propagates the classification codes of manually labelled citations to neighbouring unlabelled citations in the feature space. The automatically labelled citations are combined with the manually labelled citations to form an augmented training set. For evaluation purposes, we apply our method to reviews from clinical and public health. The results show that our semi-supervised method with label propagation achieves statistically significant improvements over two state-of-the-art active learning approaches across both clinical and public health reviews.

Raman spectroscopy of uranium compounds and the use of multivariate analysis for visualization and classification.

Raman spectroscopy was used on 95 samples comprising mainly of uranium ore concentrates as well as some UF4 and UO2 samples, in order to classify uranium compounds for nuclear forensic purposes, for the first time. This technique was selected as it is non-destructive and rapid. The spectra obtained from 9 different classes of chemical compounds were subjected to multivariate data analysis such as principal component analysis (PCA), partial least square-discriminant analysis (PLS-DA) and Fisher Discriminant Analysis (FDA). These classes were ammonium diuranate (ADU), sodium diuranate (SDU), ammonium uranyl carbonate (AUC), uranyl hydroxide (UH), UO2, UO3, UO4, U3O8 and UF4. Unsupervised PCA of full spectra shows fairly good distinction among the classes with some overlaps observed with ADU and UH. These overlaps are also reflected in the poorer specificities determined by PLS-DA. Higher values of sensitivities and specificities of remaining compounds were obtained. Supervised FDA based on reduced dataset of only 40 variables shows similar results to that of PCA but with closer clustering of ADU, UH, SDU, AUC. As a rapid and non-destructive technique, Raman spectroscopy is useful and complements existing techniques in multi-faceted nuclear forensics.

Classification of spent reactor fuel for nuclear forensics.

In this paper we demonstrate the use of pattern recognition and machine learning techniques to determine the reactor type from which spent reactor fuel has originated. This has been done using the isotopic and elemental measurements of the sample and proves to be very useful in the field of nuclear forensics. Nuclear materials contain many variables (impurities and isotopes) that are very difficult to consider individually. A method that considers all material parameters simultaneously is advantageous. Currently the field of nuclear forensics focuses on the analysis of key material properties to determine details about the materials processing history, for example, utilizing known half-lives of isotopes can determine when the material was last processed (Stanley, F. E. J. Anal. At. Spectrom. 2012, 27, 1821; Varga, Z.; Wallenius, M.; Mayer, K.; Keegan, E.; Millet, S. Anal. Chem. 2009, 81, 8327-8334). However, it has been demonstrated that multivariate statistical analysis of isotopic concentrations can complement these method and are able to make use of a greater level of information through dimensionality reduction techniques (Robel, M.; Kristo, M. J. J. Environ. Radioact. 2008, 99, 1789-1797; Robel, M.; Kristo, M. J.; Heller, M. A. Nuclear Forensic Inferences Using Iterative Multidimensional Statistics. In Proceedings of the Institute of Nuclear Materials Management 50th Annual Meeting, Tucson, AZ, July 2009; 12 pages; Nicolaou, G. J. Environ. Radioact. 2006, 86, 313-318; Pajo, L.; Mayer, K.; Koch, L. Fresenius' J. Anal. Chem. 2001, 371, 348-352). There has been some success in using such multidimensional statistical methods to determine details about the history of spent reactor fuel (Robel, M.; Kristo, M. J. J. Environ. Radioact. 2008, 99, 1789-1797). Here, we aim to expand on these findings by pursuing more robust dimensionality reduction techniques based on manifold embedding which are able to better capture the intrinsic data set information. Furthermore, we demonstrate the use of a number of classification algorithms to reliably determine the reactor type in which a spent fuel material has been irradiated. A number of these classification techniques are novel applications in nuclear forensics and expand on the existing knowledge in this field by creating a reliable and robust classification model. The results from this analysis show that our techniques have been very successful and further ascertain the excellent potential of these techniques in the field of nuclear forensics at least with regard to spent reactor fuel.

Automated detection of instantaneous gait events using time frequency analysis and manifold embedding.

Accelerometry is a widely used sensing modality in human biomechanics due to its portability, non-invasiveness, and accuracy. However, difficulties lie in signal variability and interpretation in relation to biomechanical events. In walking, heel strike and toe off are primary gait events where robust and accurate detection is essential for gait-related applications. This paper describes a novel and generic event detection algorithm applicable to signals from tri-axial accelerometers placed on the foot, ankle, shank or waist. Data from healthy subjects undergoing multiple walking trials on flat and inclined, as well as smooth and tactile paving surfaces is acquired for experimentation. The benchmark timings at which heel strike and toe off occur, are determined using kinematic data recorded from a motion capture system. The algorithm extracts features from each of the acceleration signals using a continuous wavelet transform over a wide range of scales. A locality preserving embedding method is then applied to reduce the high dimensionality caused by the multiple scales while preserving salient features for classification. A simple Gaussian mixture model is then trained to classify each of the time samples into heel strike, toe off or no event categories. Results show good detection and temporal accuracies for different sensor locations and different walking terrains.

Gait recognition: highly unique dynamic plantar pressure patterns among 104 individuals.

Everyone's walking style is unique, and it has been shown that both humans and computers are very good at recognizing known gait patterns. It is therefore unsurprising that dynamic foot pressure patterns, which indirectly reflect the accelerations of all body parts, are also unique, and that previous studies have achieved moderate-to-high classification rates (CRs) using foot pressure variables. However, these studies are limited by small sample sizes (n < 30), moderate CRs (CR ≃ 90%), or both. Here we show, using relatively simple image processing and feature extraction, that dynamic foot pressures can be used to identify n = 104 subjects with a CR of 99.6 per cent. Our key innovation was improved and automated spatial alignment which, by itself, improved CR to over 98 per cent, a finding that pointedly emphasizes inter-subject pressure pattern uniqueness. We also found that automated dimensionality reduction invariably improved CRs. As dynamic pressure data are immediately usable, with little or no pre-processing required, and as they may be collected discreetly during uninterrupted gait using in-floor systems, foot pressure-based identification appears to have wide potential for both the security and health industries.

Adaptive data embedding framework for multiclass classification.

The objective of this paper is the design of an engine for the automatic generation of supervised manifold embedding models. It proposes a modular and adaptive data embedding framework for classification, referred to as DEFC, which realizes in different stages including initial data preprocessing, relation feature generation and embedding computation. For the computation of embeddings, the concepts of friend closeness and enemy dispersion are introduced, to better control at local level the relative positions of the intraclass and interclass data samples. These are shown to be general cases of the global information setup utilized in the Fisher criterion, and are employed for the construction of different optimization templates to drive the DEFC model generation. For model identification, we use a simple but effective bilevel evolutionary optimization, which searches for the optimal model and its best model parameters. The effectiveness of DEFC is demonstrated with experiments using noisy synthetic datasets possessing nonlinear distributions and real-world datasets from different application fields.

An anatomically unbiased foot template for inter-subject plantar pressure evaluation.

Pedobarographic images reflect the dynamic interaction between the plantar foot and supporting surfaces during gait and postural activities. Since intra-foot and inter-subject contact geometry are grossly similar, images may be spatially registered and directly compared. Previously arbitrary subjects have been selected as registration templates, but this can conceivably introduce anatomical bias. The purposes of this study were: (i) to compute an unbiased pedobarographic template from a large sample of healthy young adult subjects, and (ii) to demonstrate how the resulting template may be used for practical clinical and scientific analyses. Images were obtained from N=104 subjects and were registered (10,712 pairs) using (i) an optimal linear scaling technique and (ii) a nonlinear, locally affine, globally smooth technique. The nonlinear technique was found to offer biomechanically non-trivial advantages over the linear technique, most likely due to non-proportional inter-subject geometry. Specifically, the nonlinear template was able to detect morphological signals in a hallux valgus sample with greater sensitivity than the linear template. Validity of the approach was confirmed by independently assessing left and right feet, through a statistical comparison of local maximal pressures, and also through examination of random subject subsets. The current template, representative of an average healthy foot, could be a valuable resource for automated clinical and scientific analyses of foot morphology and function.

Spatiotemporal visualizations for the measurement of oropharyngeal transit time from videofluoroscopy.

Videofluoroscopy remains one of the mainstay methods for clinical swallowing assessment, yet its interpretation is both complex and subjective. This, in part, reflects the difficulties associated with estimation of bolus transit time through the oral and pharyngeal regions by visual inspection, and problems with consistent repeatability. This paper introduces a software-only framework that automatically determines the time taken for the bolus to cross 1-D anatomical landmarks representing the oral and pharyngeal region boundaries ( Fig. 1). The user-steered delineation algorithm live-wire and straight-line annotators are used to demarcate the landmark on a frame prior to the swallow action. The rate of change of intensity of the pixels in each landmark is used as the detection feature for bolus presence that can be visualized on a spatiotemporal plot. Artifacts introduced by head and neck movement are removed by updating the landmark coordinates using affine parameters optimized by a genetic-algorithm-based registration method. Heuristics are applied to the spatiotemporal plot to identify the frames during which the bolus passes the landmark. Correlation coefficients between three observers visually inspecting twenty-four 5-mL single swallow clips did not exceed 0.42. Yet the same measurements taken using this framework on the same clips had correlation coefficients exceeding 0.87.

A dynamic model of the windlass mechanism of the foot: evidence for early stance phase preloading of the plantar aponeurosis.

In the present study we have estimated the temporal elongation of the plantar aponeurosis (PA) during normal walking using a subject-specific multi-segment rigid-body model of the foot. As previous studies have suggested that muscular forces at the ankle can pre-load the PA prior to heel-strike, the main purpose of the current study was to test, through modelling, whether there is any tension present in the PA during early stance phase. Reflective markers were attached to bony landmarks to track the kinematics of the calcaneus, metatarsus and toes during barefoot walking. Ultrasonography measurements were performed on three subjects to determine both the location of the origin of the PA on the plantar aspect of the calcaneus, and the radii of the metatarsal heads. Starting with the foot in a neutral, unloaded position, inverse kinematics allowed calculation of the tension in the five slips of the PA during the whole duration of the stance phase. The results show that the PA experienced tension significantly above rest during early stance phase in all subjects (P<0.01), thus providing support for the PA-preloading hypothesis. The amount of preloading and the maximum elongation of the slips of the PA decreased from medial to lateral. The mean maximum tension exerted by the PA was 1.5 BW (body weight) over the three subjects.

Simulating acceleration from stereophotogrammetry for medical device design.

When designing a medical device based on lightweight accelerometers, the designer is faced with a number of questions in order to maximize performance while minimizing cost and complexity: Where should the inertial unit be located? How many units are required? How is performance affected if the unit is not correctly located during donning? One way to answer these questions is to use position data from a single trial, captured with a nonportable measurement system (e.g., stereophotogrammetry) to simulate measurements from multiple accelerometers at different locations on the body. In this paper, we undertake a thorough investigation into the applicability of these simulated acceleration signals via a series of interdependent experiments of increasing generality. We measured the dynamics of a reference coordinate frame using stereophotogrammetry over a number of trials. These dynamics were then used to simulate several "virtual" accelerometers at different points on the body segment. We then compared the simulated signals with those directly measured to evaluate the error under a number of conditions. Finally, we demonstrated an example of how simulated signals can be employed in a system design application. In the best case, we may expect an error of 0.028 m/s2 between a derived virtual signal and that directly measured by an accelerometer. In practice, however, using centripetal and tangential acceleration terms (that are poorly estimated) results in an error that is an order of magnitude greater than the baseline. Furthermore, nonrigidity of the limb can increase error dramatically, although the effects can be reduced considerably via careful modeling. We conclude that using simulated signals has definite benefits when an appropriate model of the body segment is applied.

Activity identification using body-mounted sensors--a review of classification techniques.

With the advent of miniaturized sensing technology, which can be body-worn, it is now possible to collect and store data on different aspects of human movement under the conditions of free living. This technology has the potential to be used in automated activity profiling systems which produce a continuous record of activity patterns over extended periods of time. Such activity profiling systems are dependent on classification algorithms which can effectively interpret body-worn sensor data and identify different activities. This article reviews the different techniques which have been used to classify normal activities and/or identify falls from body-worn sensor data. The review is structured according to the different analytical techniques and illustrates the variety of approaches which have previously been applied in this field. Although significant progress has been made in this important area, there is still significant scope for further work, particularly in the application of advanced classification techniques to problems involving many different activities.

Multivoxel fMRI analysis of color tuning in human primary visual cortex.

We use multivoxel pattern analysis (MVPA) to study the spatial clustering of color-selective neurons in the human brain. Our main objective was to investigate whether MVPA reveals the spatial arrangements of color-selective neurons in human primary visual cortex (V1). We measured the distributed fMRI activation patterns for different color stimuli (Experiment 1: cardinal colors (to which the LGN is known to be tuned), Experiment 2: perceptual hues) in V1. Our two main findings were that (i) cone-opponent cardinal color modulations produce highly reproducible patterns of activity in V1, but these were not unique to each color. This suggests that V1 neurons with tuning characteristics similar to those found in LGN are not spatially clustered. (ii) Unique activation patterns for perceptual hues in V1 support current evidence for a spatially clustered hue map. We believe that our work is the first to show evidence of spatial clustering of neurons with similar color preferences in human V1.

Predicting aspiration after hemispheric stroke from timing measures of oropharyngeal bolus flow and laryngeal closure.

Deglutitive aspiration is common after stroke, affecting up to 50% of patients and predisposing them to pneumonia, yet it is virtually impossible to predict those patients at greatest risk. The aim of this study was to develop a robust predictive model for aspiration after stroke. Swallowing was assessed by digital videofluoroscopy (VF) in 90 patients following hemispheric stroke. Lesion characteristics were determined by computerized tomography (CT) brain scan using the Alberta Stroke Programme Early CT Score (ASPECTS). Aspiration severity was measured using a validated penetration-aspiration scale. The probability of aspiration was then determined from measures of swallowing pathophysiology and lesion location by discriminant analysis. Aspiration was observed in 47 (52%) patients, yet despite disrupted swallowing physiology, intrasubject aspiration scores were variable. The best discriminant model combined pharyngeal transit time, swallow response time, and laryngeal closure duration to predict 73.11% of those aspirating (sensitivity = 66.54, specificity = 80.22, p > 0.001). The addition of lesion location did not add anything further to the predictive model. We conclude that the pathophysiology of poststroke aspiration is multifactorial but in most cases can be predicted by three key swallowing measurements. These measurements, if translatable into clinical bedside evaluation, may assist with the development of novel measurement and intervention techniques to detect and treat poststroke aspiration.

The Windkessel model revisited: a qualitative analysis of the circulatory system.

In this paper, we derive a comprehensive computational model to estimate the arterial pressure and the cardiac output of humans, by refining and adapting the well-established equations of the Windkessel theory. The model inputs are based on patient specific factors such as age, sex, smoking and fitness habits as well as the use of specific drugs. The model's outputs correlate very strongly with physiological observations, with a low error of approximately 5% for the arterial pressure.