Approaches Adopted by Researchers to Measure the Quality of the Experience of People Working from Home: a Scoping Review.

Working from home (WFH) remotely is a modality of working that requires the careful design of systems of rules and tools to enable people to exchange information and perform actions. WFH is expected to expand after the COVID-19 pandemic. How to assess and compare in a reliable way the experience of workers with different (sociotechnical) systems of WFH is a central point to supporting the diffusion of acceptable modalities of working. However, the concept of experience and how it can be measured in the domain in WFH is yet to be clearly characterized. Following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) methodology for scoping reviews, we systematically map the approaches used by researchers to assess WFH, identify which aspects are usually investigated, and examine how such aspects are usually measured in terms of questions and tools. Literature is collected using Scopus and Web of Science. Thirty-four records out of 323 focusing either on validating a scale, presenting theoretically the experience of workers or testing this empirically are included in the qualitative synthesis. The results highlight a lack of unified terminology and tools, with assessments of workers' experience mainly characterized by survey approaches and qualitative questions. Clustering together the most investigated aspects in the literature and reviewing how these aspects are assessed, we propose a list of 10 relevant overarching dimensions and attempt to define workers' experience in the domain of WFH remotely. This definition can be used as a tool by researchers aiming to assess the experience of workers in order to inform the design or redesign of the sociotechnical systems that enable WFH. Supplementary Information: The online version contains supplementary material available at 10.1007/s41347-022-00264-4.

Inside pandora's box: a systematic review of the assessment of the perceived quality of chatbots for people with disabilities or special needs.

Introduction: People with disabilities or special needs can benefit from AI-based conversational agents (i.e., chatbots) that are used for competence training and well-being management. Assessing the quality of interactions with these chatbots is key to being able to reduce dissatisfaction with them and to understanding their potential long-term benefit. This in turn will help to increase adherence to their use, thereby improving the quality of life of the large population of end-users that they are able to serve.Methods: Following Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) methodology, we systematically reviewed the literature on methods of assessing the perceived quality of interactions with chatbots using the from Scopus and the Web of Science electronic databases. Using the Boolean operators (AND/OR) the keywords chatbot*, conversational agent*, special needs, disability were combined.Results: Revealed that only 15 of 192 papers on this topic included people with disabilities or special needs in their assessments. The results also highlighted the lack of a shared theoretical framework for assessing the perceived quality of interactions with chatbots.Conclusion: Systematic procedures based on reliable and valid methodologies continue to be needed in this field. The current lack of reliable tools and systematic methods to assess chatbots for people with disabilities and special needs is concerning, and ultimately, it may also lead to unreliable systems entering the market with disruptive consequences for people.Implications for rehabilitationChatbots applied in rehabilitation are mainly tested in terms of clinical effectiveness and validity with a minimal focus on measuring the quality of the interactionThe usability and interactive properties of chatbots applied in rehabilitation are not comparable as each tool is measured in different wayThe lack of a common framework to assess chatbots exposes people with disability and special needs to the risk of using unreliable tools.

Believing is seeing: fixation duration predicts implicit negative attitudes.

A prototypical finding of social cognition is that social experiences influence later performance even though those experiences are not introspectively available. Building on social cognition research on implicit attitudes, we evaluate whether ethnic category/attribute pairs influence eye movements during the Implicit Association Test (IAT, Greenwald, McGhee, & Schwartz 1998). Results show that fixation duration predicted implicit attitudes such that when the category/attribute pairs disconfirmed one's implicit negative attitude fixation duration toward that pair increased. The present research provides evidence that eye movements and implicit processes inherent in the IAT are more broadly connected than previously thought.

Gaze and eye-tracking solutions for psychological research.

Eye-tracking technology is a growing field used to detect eye movements and analyze human processing of visual information for interactive and diagnostic applications. Different domains in scientific research such as neuroscience, experimental psychology, computer science and human factors can benefit from eye-tracking methods and techniques to unobtrusively investigate the quantitative evidence underlying visual processes. In order to meet the experimental requirements concerning the variety of application fields, different gaze- and eye-tracking solutions using high-speed cameras are being developed (e.g., eye-tracking glasses, head-mounted or desk-mounted systems), which are also compatible with other analysis devices such as magnetic resonance imaging. This work presents an overview of the main application fields of eye-tracking methodology in psychological research. In particular, two innovative solutions will be shown: (1) the SMI RED-M eye-tracker, a high performance portable remote eye-tracker suitable for different settings, that requires maximum mobility and flexibility; (2) a wearable mobile gaze-tracking device--the SMI eye-tracking glasses--which is suitable for real-world and virtual environment research. For each kind of technology, the functions and different possibilities of application in experimental psychology will be described by focusing on some examples of experimental tasks (i.e., visual search, reading, natural tasks, scene viewing and other information processing) and theoretical approaches (e.g., embodied cognition).

A psychotechnological review on eye-tracking systems: towards user experience.

PURPOSE: The aim of the present work is to show a critical review of the international literature on eye-tracking technologies by focusing on those features that characterize them as 'psychotechnologies'. METHOD: A critical literature review was conducted through the main psychology, engineering, and computer sciences databases by following specific inclusion and exclusion criteria. A total of 46 matches from 1998 to 2010 were selected for content analysis. Results have been divided into four broad thematic areas. RESULTS: We found that, although there is a growing attention to end-users, most of the studies reviewed in this work are far from being considered as adopting holistic human-computer interaction models that include both individual differences and needs of users. User is often considered only as a measurement object of the functioning of the technological system and not as a real alter-ego of the intrasystemic interaction. CONCLUSION: In order to fully benefit from the communicative functions of gaze, the research on eye-tracking must emphasize user experience. Eye-tracking systems would become an effective assistive technology for integration, adaptation and neutralization of the environmental barrier only when a holistic model can be applied for both design processes and assessment of the functional components of the interaction.

Usability evaluation with screen reader users: a video presentation of the PCTA's experimental setting and rules.

In the study entitled "Web usability evaluation with screen reader users: Implementation of the Partial Concurrent Thinking Aloud technique" (Federici et al. 2010), we have proposed a modified protocol of usability evaluation technique for blind users, which integrates the features of the concurrent and the retrospective techniques. This new technique, called partial concurrent thinking aloud (PCTA), while respecting the properties of classic verbal protocols, overcomes the structural interference and the limits of concurrent and retrospective protocols when used with screen reader users. In order to facilitate understanding and acquisition of the PCTA's technique for practitioners and researchers, we have video recorded three different verbal protocols by visualizing five experimental sections. In the first two videos, we have compared a concurrent with a retrospective's verbal protocol of a sighted user, showing the difference of the verbalizations provided by the user in these two conditions. The third video shows the structural interference of the screen reader, during a blind user concurrent thinking aloud. In the last two videos, we show the difference of a blind user behaviour when PCTA or retrospective protocol is adopted. The videos clearly visualize the advantage of the PCTA use in respect of the two other protocols. In conclusion, the visualization of the PCTA technique confirms that this new verbal protocol promotes and guarantees a more user-driven usability assessment with disabled people, by better involving screen reader users, overcoming the structural interference and the limits of the concurrent and retrospective protocols.