Pose-based tremor type and level analysis for Parkinson's disease from video.

PURPOSE: Current methods for diagnosis of PD rely on clinical examination. The accuracy of diagnosis ranges between 73 and 84%, and is influenced by the experience of the clinical assessor. Hence, an automatic, effective and interpretable supporting system for PD symptom identification would support clinicians in making more robust PD diagnostic decisions. METHODS: We propose to analyze Parkinson's tremor (PT) to support the analysis of PD, since PT is one of the most typical symptoms of PD with broad generalizability. To realize the idea, we present SPA-PTA, a deep learning-based PT classification and severity estimation system that takes consumer-grade videos of front-facing humans as input. The core of the system is a novel attention module with a lightweight pyramidal channel-squeezing-fusion architecture that effectively extracts relevant PT information and filters noise. It enhances modeling performance while improving system interpretability. RESULTS: We validate our system via individual-based leave-one-out cross-validation on two tasks: the PT classification task and the tremor severity rating estimation task. Our system presents a 91.3% accuracy and 80.0% F1-score in classifying PT with non-PT class, while providing a 76.4% accuracy and 76.7% F1-score in more complex multiclass tremor rating classification task. CONCLUSION: Our system offers a cost-effective PT classification and tremor severity estimation results as warning signs of PD for undiagnosed patients with PT symptoms. In addition, it provides a potential solution for supporting PD diagnosis in regions with limited clinical resources.

Simulation of vertical dispersion and pollution impact of artificial light at night in urban environment.

The use of artificial light at night (ALAN) enables social and commercial activities for urban living. However, the excessive usage of lighting causes nuisance and waste of energy. Light is provided to illuminate target areas on the street level where activities take place, yet light can also cause trespass to residents at the floors above. While regulations are beginning to cover light design, simulation tools for the outdoor environment have also become more popular for assessing the design condition. Simulation tools allow visualisation of the impact of the selected light sources on those who are affected. However, this cause-and-effect relationship is not easy to determine in the complex urban environment. The current work offers a simple methodology that takes site survey results and correlates them with the simulation model to determine lighting impact on the investigated area in 3D. Four buildings in two mixed commercial and residential streets in Hong Kong were studied. Data collection from each residential building requires lengthy work and permission from each household. Therefore, a valid lighting simulation model could help determine the light pollution impact in the area. A light model using DIALux is developed and calibrated by correlating the simulated data with the actual measured data. The correlation value R2 achieved ranged from 0.95 to 0.99, verifying the accuracy of this model and matched from 340 lx to 46 lx for the lower to higher floors of one building and 10 lx to 4 lx for floors of another building. This model can also be applied to human health research, by providing light-level data on residential windows in an area or determining the environmental impact of a development project.

A Two-stream Convolutional Network for Musculoskeletal and Neurological Disorders Prediction.

Musculoskeletal and neurological disorders are the most common causes of walking problems among older people, and they often lead to diminished quality of life. Analyzing walking motion data manually requires trained professionals and the evaluations may not always be objective. To facilitate early diagnosis, recent deep learning-based methods have shown promising results for automated analysis, which can discover patterns that have not been found in traditional machine learning methods. We observe that existing work mostly applies deep learning on individual joint features such as the time series of joint positions. Due to the challenge of discovering inter-joint features such as the distance between feet (i.e. the stride width) from generally smaller-scale medical datasets, these methods usually perform sub-optimally. As a result, we propose a solution that explicitly takes both individual joint features and inter-joint features as input, relieving the system from the need of discovering more complicated features from small data. Due to the distinctive nature of the two types of features, we introduce a two-stream framework, with one stream learning from the time series of joint position and the other from the time series of relative joint displacement. We further develop a mid-layer fusion module to combine the discovered patterns in these two streams for diagnosis, which results in a complementary representation of the data for better prediction performance. We validate our system with a benchmark dataset of 3D skeleton motion that involves 45 patients with musculoskeletal and neurological disorders, and achieve a prediction accuracy of 95.56%, outperforming state-of-the-art methods.

Cerebral Palsy Prediction with Frequency Attention Informed Graph Convolutional Networks.

Early diagnosis and intervention are clinically con-sidered the paramount part of treating cerebral palsy (CP), so it is essential to design an efficient and interpretable automatic prediction system for CP. We highlight a significant difference between CP infants' frequency of human movement and that of the healthy group, which improves prediction performance. However, the existing deep learning-based methods did not use the frequency information of infants' movement for CP prediction. This paper proposes a frequency attention informed graph convolutional network and validates it on two consumer-grade RGB video datasets, namely MINI-RGBD and RVI-38 datasets. Our proposed frequency attention module aids in improving both classification performance and system interpretability. In addition, we design a frequency-binning method that retains the critical frequency of the human joint position data while filtering the noise. Our prediction performance achieves state-of-the-art research on both datasets. Our work demonstrates the effectiveness of frequency information in supporting the prediction of CP non-intrusively and provides a way for supporting the early diagnosis of CP in the resource-limited regions where the clinical resources are not abundant.

Light pollution spatial impact assessment in Hong Kong: Measurement and numerical modelling on commercial lights at street level.

With rapid urbanization, the use of external lighting to illuminate cities for night-time activity is on the rise worldwide. Many studies have suggested the excessive use of external lighting causes light pollution, which harms human health and leads to energy wastage. Although more awareness has been raised, there are not many regulations and guidelines available. As one of the cities most affected by light pollution in the world, Hong Kong has started exploring this issue within the general and business communities. However, studies that quantitatively evaluate the problem of light pollution in this city are lacking. This study aimed to assess light pollution quantitatively through measurement and numerical modelling. To achieve this, measurement protocols were developed, and site measurements were carried out in one of the known problem areas, Sai Yeung Choi Street in Mong Kok district. Through this exercise, both vertical and horizontal illuminances on the street level and the light distribution along the street were determined. An average level of 250 lx for the vertical illuminance was found, which was 3-4 times higher than the recommended brightness for normal activity. The light environment of the measured area was also modelled with the simulation program DIALux. This effort complemented the measurements by providing a means to increase the resolution on the light variation and to visualize light pollution in a 3D environment. The simulation results were verified by correlating the numerical model with measurements. The correlated model was exercised in a subsequent sensitivity study to predict possible outcomes with changing lighting pattern and lighting lumen level. This study serves to quantify this issue, which helps with the further development of effective solutions.

A Pose-Based Feature Fusion and Classification Framework for the Early Prediction of Cerebral Palsy in Infants.

The early diagnosis of cerebral palsy is an area which has recently seen significant multi-disciplinary research. Diagnostic tools such as the General Movements Assessment (GMA), have produced some very promising results. However, the prospect of automating these processes may improve accessibility of the assessment and also enhance the understanding of movement development of infants. Previous works have established the viability of using pose-based features extracted from RGB video sequences to undertake classification of infant body movements based upon the GMA. In this paper, we propose a series of new and improved features, and a feature fusion pipeline for this classification task. We also introduce the RVI-38 dataset, a series of videos captured as part of routine clinical care. By utilising this challenging dataset we establish the robustness of several motion features for classification, subsequently informing the design of our proposed feature fusion framework based upon the GMA. We evaluate our proposed framework's classification performance using both the RVI-38 dataset and the publicly available MINI-RGBD dataset. We also implement several other methods from the literature for direct comparison using these two independent datasets. Our experimental results and feature analysis show that our proposed pose-based method performs well across both datasets. The proposed features afford us the opportunity to include finer detail than previous methods, and further model GMA specific body movements. These new features also allow us to take advantage of additional body-part specific information as a means of improving the overall classification performance, whilst retaining GMA relevant, interpretable, and shareable features.

Spatio-Temporal Manifold Learning for Human Motions via Long-Horizon Modeling.

Data-driven modeling of human motions is ubiquitous in computer graphics and computer vision applications, such as synthesizing realistic motions or recognizing actions. Recent research has shown that such problems can be approached by learning a natural motion manifold using deep learning on a large amount data, to address the shortcomings of traditional data-driven approaches. However, previous deep learning methods can be sub-optimal for two reasons. First, the skeletal information has not been fully utilized for feature extraction. Unlike images, it is difficult to define spatial proximity in skeletal motions in the way that deep networks can be applied for feature extraction. Second, motion is time-series data with strong multi-modal temporal correlations between frames. On the one hand, a frame could be followed by several candidate frames leading to different motions; on the other hand, long-range dependencies exist where a number of frames in the beginning are correlated with a number of frames later. Ineffective temporal modeling would either under-estimate the multi-modality and variance, resulting in featureless mean motion or over-estimate them resulting in jittery motions, which is a major source of visual artifacts. In this paper, we propose a new deep network to tackle these challenges by creating a natural motion manifold that is versatile for many applications. The network has a new spatial component for feature extraction. It is also equipped with a new batch prediction model that predicts a large number of frames at once, such that long-term temporally-based objective functions can be employed to correctly learn the motion multi-modality and variances. With our system, long-duration motions can be predicted/synthesized using an open-loop setup where the motion retains the dynamics accurately. It can also be used for denoising corrupted motions and synthesizing new motions with given control signals. We demonstrate that our system can create superior results comparing to existing work in multiple applications.

3D Object Reconstruction from Imperfect Depth Data Using Extended YOLOv3 Network.

State-of-the-art intelligent versatile applications provoke the usage of full 3D, depth-based streams, especially in the scenarios of intelligent remote control and communications, where virtual and augmented reality will soon become outdated and are forecasted to be replaced by point cloud streams providing explorable 3D environments of communication and industrial data. One of the most novel approaches employed in modern object reconstruction methods is to use a priori knowledge of the objects that are being reconstructed. Our approach is different as we strive to reconstruct a 3D object within much more difficult scenarios of limited data availability. Data stream is often limited by insufficient depth camera coverage and, as a result, the objects are occluded and data is lost. Our proposed hybrid artificial neural network modifications have improved the reconstruction results by 8.53% which allows us for much more precise filling of occluded object sides and reduction of noise during the process. Furthermore, the addition of object segmentation masks and the individual object instance classification is a leap forward towards a general-purpose scene reconstruction as opposed to a single object reconstruction task due to the ability to mask out overlapping object instances and using only masked object area in the reconstruction process.

Interaction-Based Human Activity Comparison.

Traditional methods for motion comparison consider features from individual characters. However, the semantic meaning of many human activities is usually defined by the interaction between them, such as a high-five interaction of two characters. There is little success in adapting interaction-based features in activity comparison, as they either do not have a fixed topology or are in high dimensional. In this paper, we propose a unified framework for activity comparison from the interaction point of view. Our new metric evaluates the similarity of interaction by adapting the Earth Mover's Distance onto a customized geometric mesh structure that represents spatial-temporal interactions. This allows us to compare different classes of interactions and discover their intrinsic semantic similarity. We created five interaction databases of different natures, covering both two-characters (synthetic and real-people) and character-object interactions, which are open for public uses. We demonstrate how the proposed metric aligns well with the semantic meaning of the interaction. We also apply the metric in interaction retrieval and show how it outperforms existing ones. The proposed method can be used for unsupervised activity detection in monitoring systems and activity retrieval in smart animation systems.

Establishing Pose Based Features Using Histograms for the Detection of Abnormal Infant Movements.

The pursuit of early diagnosis of cerebral palsy has been an active research area with some very promising results using tools such as the General Movements Assessment (GMA). In this paper, we conducted a pilot study on extracting important information from video sequences to classify the body movement into two categories, normal and abnormal, and compared the results provided by an independent expert reviewer based on GMA. We present two new pose-based features, Histograms of Joint Orientation 2D (HOJO2D) and Histograms of Joint Displacement 2D (HOJD2D), for the pose-based analysis and classification of infant body movement from video footage. We extract the 2D skeletal joint locations from 2D RGB images using Cao et al.'s method [1]. Using the MINI-RGBD dataset [2], we further segment the body into local regions to extract part specific features. As a result, the pose and the degree of displacement are represented by histograms of normalised data. To demonstrate the effectiveness of the proposed features, we trained several classifiers using combinations of HOJO2D and HOJD2D features and conducted a series of experiments to classify the body movement into categories. The classification algorithms used included k-Nearest Neighbour (kNN, k=1 and k=3), Linear Discriminant Analysis (LDA) and the Ensemble classifier. Encouraging results were attained, with high accuracy (91.67%) obtained using the Ensemble classifier.

A Sampling Approach to Generating Closely Interacting 3D Pose-Pairs from 2D Annotations.

We introduce a data-driven method to generate a large number of plausible, closely interacting 3D human pose-pairs, for a given motion category, e.g., wrestling or salsa dance. With much difficulty in acquiring close interactions using 3D sensors, our approach utilizes abundant existing video data which cover many human activities. Instead of treating the data generation problem as one of reconstruction, either through 3D acquisition or direct 2D-to-3D data lifting from video annotations, we present a solution based on Markov Chain Monte Carlo (MCMC) sampling. Given a motion category and a set of video frames depicting the motion with the 2D pose-pair in each frame annotated, we start the sampling with one or few seed 3D pose-pairs which are manually created based on the target motion category. The initial set is then augmented by MCMC sampling around the seeds, via the Metropolis-Hastings algorithm and guided by a probability density function (PDF) that is defined by two terms to bias the sampling towards 3D pose-pairs that are physically valid and plausible for the motion category. With a focus on efficient sampling over the space of close interactions, rather than pose spaces, we develop a novel representation called interaction coordinates (IC) to encode both poses and their interactions in an integrated manner. Plausibility of a 3D pose-pair is then defined based on the IC and with respect to the annotated 2D pose-pairs from video. We show that our sampling-based approach is able to efficiently synthesize a large volume of plausible, closely interacting 3D pose-pairs which provide a good coverage of the input 2D pose-pairs.

Automatic Musculoskeletal and Neurological Disorder Diagnosis With Relative Joint Displacement From Human Gait.

Musculoskeletal and neurological disorders are common devastating companions of ageing, leading to a reduction in quality of life and increased mortality. Gait analysis is a popular method for diagnosing these disorders. However, manually analyzing the motion data is a labor-intensive task, and the quality of the results depends on the experience of the doctors. In this paper, we propose an automatic framework for classifying musculoskeletal and neurological disorders among older people based on 3D motion data. We also propose two new features to capture the relationship between joints across frames, known as 3D Relative Joint Displacement (3DRJDP) and 6D Symmetric Relative Joint Displacement (6DSymRJDP), such that the relative movement between joints can be analyzed. To optimize the classification performance, we adapt feature selection methods to choose an optimal feature set from the raw feature input. Experimental results show that we achieve a classification accuracy of 84.29% using the proposed relative joint features, outperforming existing features that focus on the movement of individual joints. Considering the limited open motion database for gait analysis focusing on such disorders, we construct a comprehensive, openly accessible 3D full-body motion database from 45 subjects.

Is there an association between fear avoidance beliefs, and pain and disability in patients with orofacial pain?

Numerous psychosocial factors have been shown to contribute to the development and perpetuation of orofacial pain. One well-recognized model for explaining the link between psychosocial factors and chronic pain is the fear avoidance model. To date, this proposed link has not been studied in subjects with orofacial pain. During the initial evaluation of subjects with orofacial pain, we collected data on fear avoidance beliefs using the Fear Avoidance Beliefs Questionnaire, and disability and pain. At between 6 and 8 weeks follow-up, we re-collected these data, as well as data addressing subjects' perceived change in their condition. Data were analyzed using correlation coefficients and linear regression. Fear avoidance beliefs at intake were inversely correlated with intake disability, There were no significant associations between fear avoidance beliefs at initial evaluation or in changes in fear avoidance beliefs during the 6-8 weeks follow-up period; and changes in disability, pain or perceived change in condition at 6-8 weeks follow-up. Of note, fear avoidance beliefs increased over the follow-up period, despite improvements in all outcome measures. There was insufficient evidence to suggest that high levels of fear avoidance beliefs at initial evaluation are associated with higher levels of disability or pain at intake, or with change in disability, pain or perceived change in condition at 6-8 weeks follow-up. Similarly, there was insufficient evidence to suggest that changes in fear avoidance beliefs during treatment are associated with any of these outcome measures.

An Energy-Driven Motion Planning Method for Two Distant Postures.

In this paper, we present a local motion planning algorithm for character animation. We focus on motion planning between two distant postures where linear interpolation leads to penetrations. Our framework has two stages. The motion planning problem is first solved as a Boundary Value Problem (BVP) on an energy graph which encodes penetrations, motion smoothness and user control. Having established a mapping from the configuration space to the energy graph, a fast and robust local motion planning algorithm is introduced to solve the BVP to generate motions that could only previously be computed by global planning methods. In the second stage, a projection of the solution motion onto a constraint manifold is proposed for more user control. Our method can be integrated into current keyframing techniques. It also has potential applications in motion planning problems in robotics.

Real-time posture reconstruction for Microsoft Kinect.

The recent advancement of motion recognition using Microsoft Kinect stimulates many new ideas in motion capture and virtual reality applications. Utilizing a pattern recognition algorithm, Kinect can determine the positions of different body parts from the user. However, due to the use of a single-depth camera, recognition accuracy drops significantly when the parts are occluded. This hugely limits the usability of applications that involve interaction with external objects, such as sport training or exercising systems. The problem becomes more critical when Kinect incorrectly perceives body parts. This is because applications have limited information about the recognition correctness, and using those parts to synthesize body postures would result in serious visual artifacts. In this paper, we propose a new method to reconstruct valid movement from incomplete and noisy postures captured by Kinect. We first design a set of measurements that objectively evaluates the degree of reliability on each tracked body part. By incorporating the reliability estimation into a motion database query during run time, we obtain a set of similar postures that are kinematically valid. These postures are used to construct a latent space, which is known as the natural posture space in our system, with local principle component analysis. We finally apply frame-based optimization in the space to synthesize a new posture that closely resembles the true user posture while satisfying kinematic constraints. Experimental results show that our method can significantly improve the quality of the recognized posture under severely occluded environments, such as a person exercising with a basketball or moving in a small room.

Indexing and retrieving motions of characters in close contact.

Human motion indexing and retrieval are important for animators due to the need to search for motions in the database which can be blended and concatenated. Most of the previous researches of human motion indexing and retrieval compute the Euclidean distance of joint angles or joint positions. Such approaches are difficult to apply for cases in which multiple characters are closely interacting with each other, as the relationships of the characters are not encoded in the representation. In this research, we propose a topology-based approach to index the motions of two human characters in close contact. We compute and encode how the two bodies are tangled based on the concept of rational tangles. The encoded relationships, which we define as TangleList, are used to determine the similarity of the pairs of postures. Using our method, we can index and retrieve motions such as one person piggy-backing another, one person assisting another in walking, and two persons dancing / wrestling. Our method is useful to manage a motion database of multiple characters. We can also produce motion graph structures of two characters closely interacting with each other by interpolating and concatenating topologically similar postures and motion clips, which are applicable to 3D computer games and computer animation.

Prevalence of back symptoms in elders.

OBJECTIVE: The prevalence of back pain has not been well studied in elders in the US. We describe the prevalence of back symptoms in a cohort of elderly subjects residing in the US by age, sex, examination site, and location of pain. METHODS: Data from this study are based on 1037 surviving members of the original Framingham Heart Study cohort aged 68-100 years who participated in the 22nd biennial examination in 1992-93. Subjects were questioned about back pain and timing and location of pain. RESULTS: Prevalence estimates varied, depending on the question used to assess back symptoms and the manner in which the question was asked. For example, back symptoms on most days occurred in 22.3 of 100 elders. Low back symptoms were more prevalent than those in the mid or upper back. Prevalence was higher among women than men, especially for symptoms in the mid or upper back area. Age did not affect the prevalence of back symptoms in this elderly cohort. Back symptoms were also more prevalent among subjects who were examined in their residence than among those who were examined at the examination site. Most subjects who were examined at their residence chose this location for health reasons. CONCLUSION: Back symptoms are highly prevalent in the elderly, although, among elders, they do not increase in prevalence with age. They are more common in women than men. Elders confined mostly to their homes have an especially high prevalence of back symptoms.