Effectiveness of embodied evaluation of mobile applications: A qualitative study.

As a consequence of technological advances, the number of devices has increased, with the emergence of different smartphones, tablets, and smartwatches present in both personal and professional activities. As a result, mobile applications have been developed, and with them comes the need for techniques and methods that provide support for conducting evaluation and prototyping since the current approaches are limited and cannot support the complexity and the need for understanding the context. The overall goal of the study is to evaluate the applicability of adopting a situated and embodied approach to mobile application usability testing. The aspects of postmodern and phenomenological approaches were taken into consideration. The study was conducted using the technique of digital ethnography, in particular the re-enactment technique, combined with qualitative research techniques to capture the evidence that the situated and embodied approach allows for capturing the perceptions and experience of the participants about the use of the application under evaluation. As motivation, there is the theoretical and methodological evolution and contribution and the proof that the postmodern and situated approach allows us to evaluate the usability of mobile applications in a complete way, considering the context and the user experience. The results show aspects of experience, reflections, perceptions, contingencies, practices, and meanings that go beyond the complexity of interactions and context with the use of the application, reinforcing the effectiveness of the use of the situated prototyping approach.

Benefits, challenges, and usability evaluation of DeloreanJS: a back-in-time debugger for JavaScript.

JavaScript Web applications are a common product in industry. As with most applications, Web applications can acquire software flaws (known as bugs), whose symptoms are seen during the development stage and, even worse, in production. The use of debuggers is beneficial for detecting bugs. Unfortunately, most JavaScript debuggers (1) only support the "step into/through" feature in an execution program to detect a bug, and (2) do not allow developers to go back-in-time at the application execution to take actions to detect the bug accurately. For example, the second limitation does not allow developers to modify the value of a variable to fix a bug while the application is running or test if the same bug is triggered with other values of that variable. Using concepts such as continuations and static analysis, this article presents a usable debugger for JavaScript, named DeloreanJS, which enables developers to go back-in-time in different execution points and resume the execution of a Web application to improve the understanding of a bug, or even experiment with hypothetical scenarios around the bug. Using an online and available version, we illustrate the benefits of DeloreanJS through five examples of bugs in JavaScript. Although DeloreanJS is developed for JavaScript, a dynamic prototype-based object model with side effects (mutable variables), we discuss our proposal with the state-of-art/practice of debuggers in terms of features. For example, modern browsers like Mozilla Firefox include a debugger in their distribution that only support for the breakpoint feature. However DeloreanJS uses a graphical user interface that considers back-in-time features. The aim of this study is to evaluate and compare the usability of DeloreanJS and Mozilla Firefox's debugger using the system usability scale approach. We requested 30 undergraduate students from two computer science programs to solve five tasks. Among the findings, we highlight two results. First, we found that 100% (15) of participants recommended DeloreanJS, and only 53% (eight) recommended Firefox's debugger to complete the tasks. Second, whereas the average score for DeloreanJS is 71.6 ("Good"), the average score for Firefox's debugger is 55.8 ("Acceptable").

Multi-part quality evaluation of a customized mobile application for monitoring elderly patients with functional loss and helping caregivers.

BACKGROUND: The challenges faced by caregivers of the elderly with chronic diseases are always complex. In this context, mobile technologies have been used with promising results, but often have restricted functionality, or are either difficult to use or do not provide the necessary support to the caregiver - which leads to declining usage over time. Therefore, we developed the Mobile System for Elderly Monitoring, SMAI. The purpose of SMAI is to monitor patients with functional loss and to improve the support to caregivers' communication with the health team professionals, informing them the data related to the patients' daily lives, while providing the health team better tools. METHOD: SMAI is composed of mobile applications developed for the caregivers and health team, and a web portal that supports management activities. Caregivers use an Android application to send information and receive care advice and feedback from the health team. The system was constructed using a refinement stage approach. Each stage involved caregivers and the health team in prototype release-test-assessment-refinement cycles. SMAI was evaluated during 18 months. We studied which features were being used the most, and their use pattern throughout the week. We also studied the users' qualitative perceptions. Finally, the caregiver application was also evaluated for usability. RESULTS: SMAI functionalities showed to be very useful or useful to caregivers and health professionals. The Focus Group interviews reveled that among caregivers the use of the application gave them the sensation of being connected to the health team. The usability evaluation identified that the interface design and associated tasks were easy to use and the System Usability Scale, SUS, presented very good results. CONCLUSIONS: In general, the use of SMAI represented a positive change for the family caregivers and for the NAI health team. The overall qualitative results indicate that the approach used to construct the system was appropriate to achieve the objectives.

A Human⁻Machine Interface Based on Eye Tracking for Controlling and Monitoring a Smart Home Using the Internet of Things.

People with severe disabilities may have difficulties when interacting with their home devices due to the limitations inherent to their disability. Simple home activities may even be impossible for this group of people. Although much work has been devoted to proposing new assistive technologies to improve the lives of people with disabilities, some studies have found that the abandonment of such technologies is quite high. This work presents a new assistive system based on eye tracking for controlling and monitoring a smart home, based on the Internet of Things, which was developed following concepts of user-centered design and usability. With this system, a person with severe disabilities was able to control everyday equipment in her residence, such as lamps, television, fan, and radio. In addition, her caregiver was able to monitor remotely, by Internet, her use of the system in real time. Additionally, the user interface developed here has some functionalities that allowed improving the usability of the system as a whole. The experiments were divided into two steps. In the first step, the assistive system was assembled in an actual home where tests were conducted with 29 participants without disabilities. In the second step, the system was tested with online monitoring for seven days by a person with severe disability (end-user), in her own home, not only to increase convenience and comfort, but also so that the system could be tested where it would in fact be used. At the end of both steps, all the participants answered the System Usability Scale (SUS) questionnaire, which showed that both the group of participants without disabilities and the person with severe disabilities evaluated the assistive system with mean scores of 89.9 and 92.5, respectively.

Human-centered design of a personal health record system for metabolic syndrome management based on the ISO 9241-210:2010 standard.

BACKGROUND: Previous studies have demonstrated the effectiveness of information and communication technologies to support healthy lifestyle interventions. In particular, personal health record systems (PHR-Ss) empower self-care, essential to support lifestyle changes. Approaches such as the user-centered design (UCD), which is already a standard within the software industry (ISO 9241-210:2010), provide specifications and guidelines to guarantee user acceptance and quality of eHealth systems. However, no single PHR-S for metabolic syndrome (MS) developed following the recommendations of the ISO 9241-210:2010 specification has been found in the literature. OBJECTIVE: The aim of this study was to describe the development of a PHR-S for the management of MS according to the principles and recommendations of the ISO 9241-210 standard. METHODS: The proposed PHR-S was developed using a formal software development process which, in addition to the traditional activities of any software process, included the principles and recommendations of the ISO 9241-210 standard. To gather user information, a survey sample of 1,187 individuals, eight interviews, and a focus group with seven people were performed. Throughout five iterations, three prototypes were built. Potential users of each system evaluated each prototype. The quality attributes of efficiency, effectiveness, and user satisfaction were assessed using metrics defined in the ISO/IEC 25022 standard. RESULTS: The following results were obtained: 1) a technology profile from 1,187 individuals at risk for MS from the city of Popayan, Colombia, identifying that 75.2% of the people use the Internet and 51% had a smartphone; 2) a PHR-S to manage MS developed (the PHR-S has the following five main functionalities: record the five MS risk factors, share these measures with health care professionals, and three educational modules on nutrition, stress management, and a physical activity); and 3) usability tests on each prototype obtaining the following results: 100% effectiveness, 100% efficiency, and 84.2 points in the system usability scale. CONCLUSION: The software development methodology used was based on the ISO 9241-210 standard, which allowed the development team to maintain a focus on user's needs and requirements throughout the project, which resulted in an increased satisfaction and acceptance of the system. Additionally, the establishment of a multidisciplinary team allowed the application of considerations not only from the disciplines of software engineering and health sciences but also from other disciplines such as graphical design and media communication. Finally, usability testing allowed the observation of flaws in the designs, which helped to improve the solution.

Usability Evaluation Methods for Gesture-Based Games: A Systematic Review.

BACKGROUND: Gestural interaction systems are increasingly being used, mainly in games, expanding the idea of entertainment and providing experiences with the purpose of promoting better physical and/or mental health. Therefore, it is necessary to establish mechanisms for evaluating the usability of these interfaces, which make gestures the basis of interaction, to achieve a balance between functionality and ease of use. OBJECTIVE: This study aims to present the results of a systematic review focused on usability evaluation methods for gesture-based games, considering devices with motion-sensing capability. We considered the usability methods used, the common interface issues, and the strategies adopted to build good gesture-based games. METHODS: The research was centered on four electronic databases: IEEE, Association for Computing Machinery (ACM), Springer, and Science Direct from September 4 to 21, 2015. Within 1427 studies evaluated, 10 matched the eligibility criteria. As a requirement, we considered studies about gesture-based games, Kinect and/or Wii as devices, and the use of a usability method to evaluate the user interface. RESULTS: In the 10 studies found, there was no standardization in the methods because they considered diverse analysis variables. Heterogeneously, authors used different instruments to evaluate gesture-based interfaces and no default approach was proposed. Questionnaires were the most used instruments (70%, 7/10), followed by interviews (30%, 3/10), and observation and video recording (20%, 2/10). Moreover, 60% (6/10) of the studies used gesture-based serious games to evaluate the performance of elderly participants in rehabilitation tasks. This highlights the need for creating an evaluation protocol for older adults to provide a user-friendly interface according to the user's age and limitations. CONCLUSIONS: Through this study, we conclude this field is in need of a usability evaluation method for serious games, especially games for older adults, and that the definition of a methodology and a test protocol may offer the user more comfort, welfare, and confidence.

Does the use of structured reporting improve usability? A comparative evaluation of the usability of two approaches for findings reporting in a large-scale telecardiology context.

One of the main reasons that leads to a low adoption rate of telemedicine systems is poor usability. An aspect that influences usability during the reporting of findings is the input mode, e.g., if a free-text (FT) or a structured report (SR) interface is employed. The objective of our study is to compare the usability of FT and ST telemedicine systems, specifically in terms of user satisfaction, efficiency and general usability. We comparatively evaluate the usability of these two input modes in a telecardiology system for issuing electrocardiography reports in the context of a statewide telemedicine system in Brazil with more than 350.000 performed tele-electrocardiography examinations. We adopted a multiple method research strategy, applying three different kinds of usability evaluations: user satisfaction was evaluated through interviews with seven medical professionals using the System Usability Scale (SUS) questionnaire and specific questions related to adequacy and user experience. Efficiency was evaluated by estimating execution time using the Keystroke-Level Model (KLM). General usability was assessed based on the conformity of the systems to a set of e-health specific usability heuristics. The results of this comparison provide a first indication that a structured report (SR) input mode for such a system is more satisfactory and efficient with a larger conformity to usability heuristics than free-text (FT) input. User satisfaction using the SUS questionnaire has been scored in average with 58.8 and 77.5 points for the FT and SR system, respectively, which means that the SR system was rated 18.65 points higher than the FT system. In terms of efficiency, the completion of a findings report using the SR mode is estimated to take 8.5s, 3.74 times faster than using the FT system (31.8s). The SR system also demonstrated less violations to usability heuristics (8 points) in comparison to 14 points observed in the FT system. These results provide a first indication that the usage of structured reporting as an input mode in telecardiology systems may enhance usability. This also seems to confirm the advantages of the usage of structured reporting, as already described in the literature for other areas such as teleradiology.

Usabilidad de los sitios Web, los métodos y las técnicas para la evaluación / Usability of Web sites, methods and evaluation techniques

En la sociedad actual es incuestionable la relevancia de la Web, y existe una gran variedad de sitios web que brindan servicio a los usuarios. En este contexto, la usabilidad juega un papel primordial en el proceso de desarrollo de sitios web de éxito. En este artículo se revisan diferentes definiciones sobre la disciplina usabilidad, su incorporación en el proceso de ingeniería (ingeniería de la usabilidad) y su relación con la ingeniería de software, sus atributos y métodos de evaluación. Se presentan las consideraciones sobre el sistema de evaluación de la usabilidad Web orientado al usuario y basado en la determinación de tareas críticas (SIRIUS revisado por las autoras)(AU)

In the present society, the relevance of the Web is unquestionable and there is a great variety of Web sites that offer services to users. In this context, usability plays a fundamental role in the process of developing successful Web sites. In this article we review different definitions about the discipline of usability, its incorporation in the engineering process (usability engineering) and its relation to software engineer, as well as its attributes and evaluation methods. The considerations about the evaluation system of the Web usability are also presented, advising the user and basing it in the establishment of critical tasks (SIRIUS, reviewed by the authors)(AU)

Usabilidad de los sitios Web, los métodos y las técnicas para la evaluación / Usability of Web sites, methods and evaluation techniques

En la sociedad actual es incuestionable la relevancia de la Web, y existe una gran variedad de sitios web que brindan servicio a los usuarios. En este contexto, la usabilidad juega un papel primordial en el proceso de desarrollo de sitios web de éxito. En este artículo se revisan diferentes definiciones sobre la disciplina usabilidad, su incorporación en el proceso de ingeniería (ingeniería de la usabilidad) y su relación con la ingeniería de software, sus atributos y métodos de evaluación. Se presentan las consideraciones sobre el sistema de evaluación de la usabilidad Web orientado al usuario y basado en la determinación de tareas críticas (SIRIUS revisado por las autoras)

In the present society, the relevance of the Web is unquestionable and there is a great variety of Web sites that offer services to users. In this context, usability plays a fundamental role in the process of developing successful Web sites. In this article we review different definitions about the discipline of usability, its incorporation in the engineering process (usability engineering) and its relation to software engineer, as well as its attributes and evaluation methods. The considerations about the evaluation system of the Web usability are also presented, advising the user and basing it in the establishment of critical tasks (SIRIUS, reviewed by the authors)