Cognitive Learning, Monitoring and Assistance of Industrial Workflows Using Egocentric Sensor Networks.

Today, the workflows that are involved in industrial assembly and production activities are becoming increasingly complex. To efficiently and safely perform these workflows is demanding on the workers, in particular when it comes to infrequent or repetitive tasks. This burden on the workers can be eased by introducing smart assistance systems. This article presents a scalable concept and an integrated system demonstrator designed for this purpose. The basic idea is to learn workflows from observing multiple expert operators and then transfer the learnt workflow models to novice users. Being entirely learning-based, the proposed system can be applied to various tasks and domains. The above idea has been realized in a prototype, which combines components pushing the state of the art of hardware and software designed with interoperability in mind. The emphasis of this article is on the algorithms developed for the prototype: 1) fusion of inertial and visual sensor information from an on-body sensor network (BSN) to robustly track the user's pose in magnetically polluted environments; 2) learning-based computer vision algorithms to map the workspace, localize the sensor with respect to the workspace and capture objects, even as they are carried; 3) domain-independent and robust workflow recovery and monitoring algorithms based on spatiotemporal pairwise relations deduced from object and user movement with respect to the scene; and 4) context-sensitive augmented reality (AR) user feedback using a head-mounted display (HMD). A distinguishing key feature of the developed algorithms is that they all operate solely on data from the on-body sensor network and that no external instrumentation is needed. The feasibility of the chosen approach for the complete action-perception-feedback loop is demonstrated on three increasingly complex datasets representing manual industrial tasks. These limited size datasets indicate and highlight the potential of the chosen technology as a combined entity as well as point out limitations of the system.

Innovative system for real-time ergonomic feedback in industrial manufacturing.

This work presents a system that permits a real-time ergonomic assessment of manual tasks in an industrial environment. First, a biomechanical model of the upper body has been developed by using inertial sensors placed at different locations on the upper body. Based on this model, a computerized RULA ergonomic assessment was implemented to permit a global risk assessment of musculoskeletal disorders in real-time. Furthermore, local scores were calculated per segment, e.g. the neck region, and gave information on the local risks for musculoskeletal disorders. Visual information was fed back to the user by using a see-through head mounted display. Additional visual highlighting and auditory warnings were provided when some predefined thresholds were exceeded. In a user study (N = 12 participants) a group with the RULA feedback was compared to a control group. Results demonstrate that the real-time ergonomic feedback significantly decreased the outcome of both globally as well as locally hazardous RULA values that are associated with increased risk for musculoskeletal disorders. Task execution time did not differ between groups. The real-time ergonomic tool introduced in this study has the potential to considerably reduce the risk of musculoskeletal disorders in industrial settings. Implications for ergonomics in manufacturing and user feedback modalities are further discussed.

IBES: a tool for creating instructions based on event segmentation.

Receiving informative, well-structured, and well-designed instructions supports performance and memory in assembly tasks. We describe IBES, a tool with which users can quickly and easily create multimedia, step-by-step instructions by segmenting a video of a task into segments. In a validation study we demonstrate that the step-by-step structure of the visual instructions created by the tool corresponds to the natural event boundaries, which are assessed by event segmentation and are known to play an important role in memory processes. In one part of the study, 20 participants created instructions based on videos of two different scenarios by using the proposed tool. In the other part of the study, 10 and 12 participants respectively segmented videos of the same scenarios yielding event boundaries for coarse and fine events. We found that the visual steps chosen by the participants for creating the instruction manual had corresponding events in the event segmentation. The number of instructional steps was a compromise between the number of fine and coarse events. Our interpretation of results is that the tool picks up on natural human event perception processes of segmenting an ongoing activity into events and enables the convenient transfer into meaningful multimedia instructions for assembly tasks. We discuss the practical application of IBES, for example, creating manuals for differing expertise levels, and give suggestions for research on user-oriented instructional design based on this tool.

Toward a better understanding of psychological well-being in dementia caregivers: the link between marital communication and depression.

Dementia research has frequently documented high rates of caregiver depression and distress in spouses providing care for a partner suffering from dementia. However, the role of marital communication in understanding caregiver distress has not been examined sufficiently. Studies with healthy couples demonstrated an association between marital communication and the partners' psychological well-being, depressiveness, respectively (e.g., Heene, Buysee, & Van Oost, 2005). The current study investigates the relationship between caregiver depression and communication in 37 couples in which the wives care for their partners with dementia. Nonsequential and sequential analyses revealed significant correlations between caregiver depression and marital communication quality. Caregivers whose husbands used more positive communication reported less depression and distress. Additionally, caregiver depression was negatively correlated with rates of positive reciprocal communication indicating dependence between the couples' interaction patterns. This study is one of the first to illustrate the relevance of spousal communication in understanding caregiver distress and depression.