Investigating Ensemble Learning and Classifier Generalization in a Hybrid, Passive Brain-Computer Interface for Assessing Cognitive Workload.

Hybrid, passive brain-computer (h/pBCI) interfaces have received much attention in regards to measuring various mental states. A high classification rate of operator workload state is necessary in order to be able to enhance operator performance. Physiological measures have been used with machine learning algorithms to classify workload state, however, these measures are hypothesized to suffer from inherent nonstationarity. To attain a more generalizable classifier, a prior solution has been to use a multi-day learning paradigm to train classifier models. In earlier work, we have shown that increasing the number of unique data sessions used to form a learning set can improve the accuracy of classifying mental workload where improved generalizability is partly attributable to the multi-day paradigm. To further investigate methods that produce more generalizable classifiers, we look to ensemble learning. Here we implement ensemble learning to increase accuracies, reduce variance, and leverage theoretical performance of the ensemble as compared to observed to make inference about generalization. An adaptive boosting method (AdaBoost) is used to train a "base learning algorithm" multiple times, adaptively adjusting to errors and forming a vote out of the resulting hypotheses using three different base learning algorithms: an artificial neural network (ANN), a support vector machine (SVM), and linear discriminant analysis (LDA). We observed that the ensemble converged on theoretical performance with respect to error and variance only when the training sets were formed using the multi-day paradigm. These results indicate that ensemble learning approaches can be used in examples of pBCI such as those designed here, but they are also affected by theorized nonstationarity in physiological response. The observation of ensemble convergence on theoretical performance may be used to make inference about generalizability when simple accuracy of detection can be misleading.

The effect of propensity to trust and perceptions of trustworthiness on trust behaviors in dyads.

Research on trust has burgeoned in the last few decades. Despite the growing interest in trust, little is known about trusting behaviors in non-dichotomous trust games. The current study explored propensity to trust, trustworthiness, and trust behaviors in a new computer-mediated trust relevant task. We used multivariate multilevel survival analysis (MMSA) to analyze behaviors across time. Results indicated propensity to trust did not influence trust behaviors. However, trustworthiness perceptions influenced initial trust behaviors and trust behaviors influenced subsequent trustworthiness perceptions. Indeed, behaviors fully mediated the relationship of trustworthiness perceptions over time. The study demonstrated the utility of MMSA and the new trust game, Checkmate, as viable research methods and stimuli for assessing the loci of trust.

Day-to-day variability in hybrid, passive brain-computer interfaces: comparing two studies assessing cognitive workload.

As hybrid, passive brain-computer interface systems become more advanced, it is important to grow our understanding of how to produce generalizable pattern classifiers of physiological data. One of the most difficult problems in applying machine learning algorithms to these data types is nonstationarity, which can evolve over the course of hours and days, and is more susceptible to changes resulting from complex cognitive function in comparison to simple, stimulus-based processes. This nonstationarity, referenced as day-to-day variability, results in the inability of many learning algorithms to generalize to new data. In previous work, we have shown that increasing the number of unique testing sessions used to form a learning set can improve the accuracy of classifying mental workload in a binary state paradigm. While this result was very promising, we did not address whether the additional discriminability was the result of a larger learning set or the uniqueness contributed by the testing sessions being spread over multiple days. Further, the simulation task used in this prior analysis was low-fidelity with respect to the task it attempted to model; whether these methods extend to more realistic task simulation environments has not been comparatively investigated. In this work, we compare these previous results to a second study, with a similar multi-day paradigm, that required participants to perform a more realistic simulation task. Comparative analysis of these two studies reveals that the improved generalization of the multi-day learning set is attributable, in large part, to the uniqueness of the multi-day paradigm. Further, this multi-day effect was also observed in the higher fidelity simulation study. These results help to validate the use of the multi-day learning set approach for improving overall system classification accuracy. Future studies should consider the use of multi-day designs for improving generalizability over other interesting dimensions.

An assessment of non-stationarity in physiological cognitive state assessment using artificial neural networks.

With increased attention toward physiological cognitive state assessment as a component in the larger field of applied neuroscience, the need to develop methods for robust, stable assessment of cognitive state has been expressed as critical to designing effective augmented human-machine systems. The technique of cognitive state assessment, as well as its benefits, has been demonstrated by many research groups. In an effort to move closer toward a realized system, efforts must now be focused on critical issues that remain unsolved, namely instability of pattern classifiers over the course of hours and days. This work, as part of the Cognitive State Assessment Competition 2011, seeks to explore methods for 'learning' non-stationarity as a mitigation for more generalized patterns that are stable over time courses that are not widely discussed in the literature.