Multimodal sensor fusion in the latent representation space.

A new method for multimodal sensor fusion is introduced. The technique relies on a two-stage process. In the first stage, a multimodal generative model is constructed from unlabelled training data. In the second stage, the generative model serves as a reconstruction prior and the search manifold for the sensor fusion tasks. The method also handles cases where observations are accessed only via subsampling i.e. compressed sensing. We demonstrate the effectiveness and excellent performance on a range of multimodal fusion experiments such as multisensory classification, denoising, and recovery from subsampled observations.

A comprehensive ultra-wideband dataset for non-cooperative contextual sensing.

Nowadays, an increasing amount of attention is being devoted towards passive and non-intrusive sensing methods. The prime example is healthcare applications, where on-body sensors are not always an option or in other applications which require the detection and tracking of unauthorized (non-cooperative) targets within a given environment. Therefore, in this paper we present a dataset consisting of measurements obtained from Radio-Frequency (RF) devices. Essentially, the dataset consists of Ultra-Wideband (UWB) data in the form of Channel Impulse Response (CIR), acquired via a Commercial Off-the-Shelf (COTS) UWB equipment. Approximately 1.6 hours of annotated measurements are provided, which are collected in a residential environment. This dataset can be used to passively track a target's location in an indoor environment. Additionally, it can also be used to advance UWB-based Human Activity Recognition (HAR) since three basic human activities were recorded, namely, sitting, standing and walking. We anticipate that such datasets may be utilized to develop novel algorithms and methodologies for healthcare, smart homes and security applications.

OPERAnet, a multimodal activity recognition dataset acquired from radio frequency and vision-based sensors.

This paper presents a comprehensive dataset intended to evaluate passive Human Activity Recognition (HAR) and localization techniques with measurements obtained from synchronized Radio-Frequency (RF) devices and vision-based sensors. The dataset consists of RF data including Channel State Information (CSI) extracted from a WiFi Network Interface Card (NIC), Passive WiFi Radar (PWR) built upon a Software Defined Radio (SDR) platform, and Ultra-Wideband (UWB) signals acquired via commercial off-the-shelf hardware. It also consists of vision/Infra-red based data acquired from Kinect sensors. Approximately 8 hours of annotated measurements are provided, which are collected across two rooms from 6 participants performing 6 daily activities. This dataset can be exploited to advance WiFi and vision-based HAR, for example, using pattern recognition, skeletal representation, deep learning algorithms or other novel approaches to accurately recognize human activities. Furthermore, it can potentially be used to passively track a human in an indoor environment. Such datasets are key tools required for the development of new algorithms and methods in the context of smart homes, elderly care, and surveillance applications.