Exploring the effect of automation failure on the human's trustworthiness in human-agent teamwork.

Introduction: Collaboration in teams composed of both humans and automation has an interdependent nature, which demands calibrated trust among all the team members. For building suitable autonomous teammates, we need to study how trust and trustworthiness function in such teams. In particular, automation occasionally fails to do its job, which leads to a decrease in a human's trust. Research has found interesting effects of such a reduction of trust on the human's trustworthiness, i.e., human characteristics that make them more or less reliable. This paper investigates how automation failure in a human-automation collaborative scenario affects the human's trust in the automation, as well as a human's trustworthiness towards the automation. Methods: We present a 2 × 2 mixed design experiment in which the participants perform a simulated task in a 2D grid-world, collaborating with an automation in a "moving-out" scenario. During the experiment, we measure the participants' trustworthiness, trust, and liking regarding the automation, both subjectively and objectively. Results: Our results show that automation failure negatively affects the human's trustworthiness, as well as their trust in and liking of the automation. Discussion: Learning the effects of automation failure in trust and trustworthiness can contribute to a better understanding of the nature and dynamics of trust in these teams and improving human-automation teamwork.

Normative uncertainty and societal preferences: the problem with evaluative standards.

Many technological systems these days interact with their environment with increasingly little human intervention. This situation comes with higher stakes and consequences that society needs to manage. No longer are we dealing with 404 pages: AI systems today may cause serious harm. To address this, we wish to exert a kind of control over these systems, so that they can adhere to our moral beliefs. However, given the plurality of values in our societies, which "oughts" ought these machines to adhere to? In this article, we examine Borda voting as a way to maximize expected choice-worthiness among individuals through different possible "implementations" of ethical principles. We use data from the Moral Machine experiment to illustrate the effectiveness of such a voting system. Although it appears to be effective on average, the maximization of expected choice-worthiness is heavily dependent on the formulation of principles. While Borda voting may be a good way of ensuring outcomes that are preferable to many, the larger problems in maximizing expected choice-worthiness, such as the capacity to formulate credences well, remain notoriously difficult; hence, we argue that such mechanisms should be implemented with caution and that other problems ought to be solved first.

Drivers of partially automated vehicles are blamed for crashes that they cannot reasonably avoid.

People seem to hold the human driver to be primarily responsible when their partially automated vehicle crashes, yet is this reasonable? While the driver is often required to immediately take over from the automation when it fails, placing such high expectations on the driver to remain vigilant in partially automated driving is unreasonable. Drivers show difficulties in taking over control when needed immediately, potentially resulting in dangerous situations. From a normative perspective, it would be reasonable to consider the impact of automation on the driver's ability to take over control when attributing responsibility for a crash. We, therefore, analyzed whether the public indeed considers driver ability when attributing responsibility to the driver, the vehicle, and its manufacturer. Participants blamed the driver primarily, even though they recognized the driver's decreased ability to avoid the crash. These results portend undesirable situations in which users of partially driving automation are the ones held responsible, which may be unreasonable due to the detrimental impact of driving automation on human drivers. Lastly, the outcome signals that public awareness of such human-factors issues with automated driving should be improved.

A Unifying Framework for Reinforcement Learning and Planning.

Sequential decision making, commonly formalized as optimization of a Markov Decision Process, is a key challenge in artificial intelligence. Two successful approaches to MDP optimization are reinforcement learning and planning, which both largely have their own research communities. However, if both research fields solve the same problem, then we might be able to disentangle the common factors in their solution approaches. Therefore, this paper presents a unifying algorithmic framework for reinforcement learning and planning (FRAP), which identifies underlying dimensions on which MDP planning and learning algorithms have to decide. At the end of the paper, we compare a variety of well-known planning, model-free and model-based RL algorithms along these dimensions. Altogether, the framework may help provide deeper insight in the algorithmic design space of planning and reinforcement learning.

What values should an agent align with?: An empirical comparison of general and context-specific values.

The pursuit of values drives human behavior and promotes cooperation. Existing research is focused on general values (e.g., Schwartz) that transcend contexts. However, context-specific values are necessary to (1) understand human decisions, and (2) engineer intelligent agents that can elicit and align with human values. We propose Axies, a hybrid (human and AI) methodology to identify context-specific values. Axies simplifies the abstract task of value identification as a guided value annotation process involving human annotators. Axies exploits the growing availability of value-laden text corpora and Natural Language Processing to assist the annotators in systematically identifying context-specific values. We evaluate Axies in a user study involving 80 human subjects. In our study, six annotators generate value lists for two timely and important contexts: Covid-19 measures and sustainable Energy. We employ two policy experts and 72 crowd workers to evaluate Axies value lists and compare them to a list of general (Schwartz) values. We find that Axies yields values that are (1) more context-specific than general values, (2) more suitable for value annotation than general values, and (3) independent of the people applying the methodology. Supplementary Information: The online version contains supplementary material available at 10.1007/s10458-022-09550-0.

Volunteers in the Smart City: Comparison of Contribution Strategies on Human-Centered Measures.

Provision of smart city services often relies on users contribution, e.g., of data, which can be costly for the users in terms of privacy. Privacy risks, as well as unfair distribution of benefits to the users, should be minimized as they undermine user participation, which is crucial for the success of smart city applications. This paper investigates privacy, fairness, and social welfare in smart city applications by means of computer simulations grounded on real-world data, i.e., smart meter readings and participatory sensing. We generalize the use of public good theory as a model for resource management in smart city applications, by proposing a design principle that is applicable across application scenarios, where provision of a service depends on user contributions. We verify its applicability by showing its implementation in two scenarios: smart grid and traffic congestion information system. Following this design principle, we evaluate different classes of algorithms for resource management, with respect to human-centered measures, i.e., privacy, fairness and social welfare, and identify algorithm-specific trade-offs that are scenario independent. These results could be of interest to smart city application designers to choose a suitable algorithm given a scenario-specific set of requirements, and to users to choose a service based on an algorithm that matches their privacy preferences.

Microlearning mApp raises health competence: hybrid service design.

Work place health support interventions can help support our aging work force, with mApps offering cost-effectiveness opportunities. Previous research shows that health support apps should offer users enough newness and relevance each time they are used. Otherwise the 'eHealth law of attrition' applies: 90 % of users are lost prematurely. Our research study builds on this prior research with further investigation on whether a mobile health quiz provides added value for users within a hybrid service mix and whether it promotes long term health? We developed a hybrid health support intervention solution that uses a mix of electronic and physical support services for improving health behaviours, including a mobile micro-learning health quiz. This solution was evaluated in a multiple-case study at three work sites with 86 users. We find that both our mobile health quiz and the overall hybrid solution contributed to improvements in health readiness, -behaviour and -competence. Users indicated that the micro-learning health quiz courses provided new and relevant information. Relatively high utilization rates of the health quiz were observed. Participants indicated that health insights were given that directly influenced every day health perceptions, -choices, coping and goal achievement strategies, plus motivation and self-norms. This points to increased user health self-management competence. Moreover, even after 10 months they indicated to still have improved health awareness, -motivation and -behaviours (food, physical activity, mental recuperation). A design analysis was conducted regarding service mix efficacy; the mobile micro-learning health quiz helped fulfil a set of key requirements that exist for designing ICT-enabled lifestyle interventions, largely in the way it was anticipated.

User validation of an empathic virtual buddy against cyberbullying.

People are able to comfort others by talking about their problems. In our research, we are exploring whether computers can provide social support in a similar manner. Recently, we proposed a design for an empathic virtual buddy that supports victims of cyberbullying. To validate our approach in providing social support and to gather feedback from potential users, we performed an experiment (N = 30) to compare interaction with the buddy to reading a text. Both the buddy and the text received high scores; scores for the buddy were consistently higher. The difference was significant for the extent to which feelings were taken into account. These results indicate that participants liked to interact with the buddy and that they recognized the emotional cues emitted by the buddy, thus validating our approach in comforting users.

Formalization of Damasio's theory of emotion, feeling and core consciousness.

This paper contributes an analysis and formalization of Damasio's theory on core consciousness. Three important concepts in this theory are 'emotion', 'feeling' and 'feeling a feeling' (or core consciousness). In particular, a simulation model is described of the dynamics of basic mechanisms leading via emotion and feeling to core consciousness, and dynamic properties are formally specified that hold for these dynamics at a more global level. These properties have been automatically checked for the simulation model. Moreover, a formal analysis is made of relevant notions of representation used by Damasio. As part of this analysis, specifications of representation relations have been verified and confirmed against the simulation model.

Formalization and analysis of reasoning by assumption.

This article introduces a novel approach for the analysis of the dynamics of reasoning processes and explores its applicability for the reasoning pattern called reasoning by assumption. More specifically, for a case study in the domain of a Master Mind game, it is shown how empirical human reasoning traces can be formalized and automatically analyzed against dynamic properties they fulfill. To this end, for the pattern of reasoning by assumption a variety of dynamic properties have been specified, some of which are considered characteristic for the reasoning pattern, whereas some other properties can be used to discriminate among different approaches to the reasoning. These properties have been automatically checked for the traces acquired in experiments undertaken. The approach turned out to be beneficial from two perspectives. First, checking characteristic properties contributes to the empirical validation of a theory on reasoning by assumption. Second, checking discriminating properties allows the analyst to identify different classes of human reasoners.

Putting intentions into cell biochemistry: an artificial intelligence perspective.

The living cell exists by virtue of thousands of nonlinearly interacting processes. This complexity greatly impedes its understanding. The standard approach to the calculation of the behaviour of the living cell, or part thereof, integrates all the rate equations of the individual processes. If successful extremely intensive calculations often lead the calculation of coherent, apparently simple, cellular "decisions" taken in response to a signal: the complexity of the behavior of the cell is often smaller than it might have been. The "decisions" correspond to the activation of entire functional units of molecular processes, rather than individual ones. The limited complexity of signal and response suggests that there might be a simpler way to model at least some important aspects of cell function. In the field of Artificial Intelligence, such simpler modelling methods for complex systems have been developed. In this paper, it is shown how the Artificial Intelligence description method for deliberative agents functioning on the basis of beliefs, desires and intentions as known in Artificial Intelligence, can be used successfully to describe essential aspects of cellular regulation. This is demonstrated for catabolite repression and substrate induction phenomena in the bacterium Escherichia coli. The method becomes highly efficient when the computation is automated in a Prolog implementation. By defining in a qualitative way the food supply of the bacterium, the make-up of its catabolic pathways is readily calculated for cases that are sufficiently complex to make the traditional human reasoning tedious and error prone.