Understanding Pitfalls and Opportunities of Applying Heuristic Evaluation Methods to VR Training Systems: An Empirical Study.

The usability of virtual reality (VR) training applications is crucial for their success, but examining the usability in the early development stages remains challenging. A realistic and plausible solution would be revisiting and reconciling Heuristics Evaluation (HE) methods among the most widely used usability inspection methods in the human-computer interaction (HCI) domain. While research on studying and using HE methods is growing within the VR domain, few studies have considered the novel VR environment challenges new requirements for fitting HE methods to the context and applying them effectively. To this end, we conducted a user study with 14 evaluators using the standard HE methods to complete two HE sessions for a VR training application. We identified five critical challenges that evaluators encountered in the HE process by observing and interviewing them. Based on our findings, we discuss the importance of considering an easy-to-use heuristic set, how we can facilitate the HE procedures in the VR context, and the opportunities for developing HE-supporting tools.

Heuristics used for evaluating the usability of mobile health applications: A systematic literature review.

Objective: Mobile health applications hold immense potential for enhancing health outcomes. Usability is one of the main factors for the adoption and use of mobile health applications. However, despite the growing importance of mHealth applications, clear standards for their evaluation remain elusive. The present study aimed to determine heuristics for the usability evaluation of health-related applications. Methods: We systematically searched multiple databases for relevant papers published between January 2008 and April 2021. Articles were reviewed, and data were extracted and categorized from those meeting inclusion criteria by two authors independently. Heuristics were identified based on statements, words, and concepts expressed in the studies. These heuristics were first mapped to Nielsen's heuristics based on their differences or similarities. The remaining heuristics that were very important for mobile applications were categorized into new heuristics. Results: Seventeen studies met the eligibility criteria. Seventy-nine heuristics were extracted from the papers. After combining the items with the same concepts and removing irrelevant items based on the exclusion criteria, 20 heuristics remained. Common heuristics such as "Visibility of system status" and "Flexibility and efficiency of use" were categorized into 10 previously established heuristics and new heuristics like "Navigation" and "User engagement" were recognized as new ones. Conclusions: In our study, we have meticulously identified 20 heuristics that hold promise for evaluating and designing mHealth applications. These heuristics can be used by the researchers for the development of robust tools for heuristic evaluation. These tools, when adapted or tailored for health domain applications, have the potential to significantly enhance the quality of mHealth applications. Ultimately, this improvement in quality translates to enhanced patient safety. Protocol Registration: (10.17605/OSF.IO/PZJ7H).

A systematic search, heuristic evaluation and analysis of dental trauma mobile applications.

BACKGROUND: Despite the exponential growth of mobile device applications in promoting awareness on traumatic dental injuries (TDIs), the overall validity of existing applications in content, engagement, usability, functionality, esthetics, and user perception remains largely uncertain. AIMS: The primary aim was to systematically search and assess the quality, functionality, perceived impact and usability of available applications related to TDIs using the User Version of the Mobile Application Rating Scale (uMARS) and the Coventry, Aberdeen, and London-Revised (CA-LO-RE) scale. A heuristic evaluation was additionally performed. The secondary aim was to categorize existing mobile applications according to their target users, including general population, dentists, dental students, medical healthcare professionals, and researchers. METHODS: A systematic search for dental trauma applications (free and paid) was conducted using the AppStore and Google Play store. Two reviewers independently performed data extraction and quality assessment for all the included applications using uMARS and CA-LO-RE. A user experience architect conducted a heuristic evaluation. RESULTS: Only 6 of the 3061 initially screened applications were eligible for qualitative and quantitative assessments. These applications' average star uMARS rating was 12.5 (standard deviation [SD] 2.6). The mean and SD for the uMARS objective items were as follows: A-engagement (mean 12.1 [SD 3.3]), B-functionality (mean 15.5 [SD 1.4]), C-esthetics (mean 9.5 [SD 2.5]), and D-information (mean 13 [SD 4.5]). The severity rating for the heuristic evaluation criteria was low to moderate for all the included applications, excluding tooth emergencies with a high severity score. CONCLUSION: Despite moderate quality, functionality, user perception, behavioural impact and usability scores, the inclusion of advanced features to "engagement" and "usability" in existing trauma based apps will enhance their purpose. A design-thinking approach coupled with interactive features is recommended for future mobile applications on TDIs.

Design of a Self-Measuring Device Based on Bioelectrical Impedance Analysis for Regular Monitoring of Rheumatoid Arthritis.

Rheumatoid arthritis (RA) is a chronic disease, in which permanent joint deformation is largely preventable with the timely introduction of appropriate treatment strategies. However, there is no consensus for patients with RA to monitor their progress and communicate it to the rheumatologist till the condition progresses to remission. In response to this unmet need, we proposed the design of a self-measuring device based on bioelectrical impedance analysis (BIA) for regular monitoring of inflammation levels. Twenty joints of both hands were measured to monitor trends in inflammation levels. Three electrodes were used to measure two joints of each finger. A central electrode was used for two consecutive measurements. A suitable form factor for the device was proposed for the vertical placement of the hand. To ensure the stability of measurements, an air cushion was incorporated into the back of the hand, hand containers were designed on both sides, and a mobile application was designed. We conducted a convergence-assessment experiment with five air pressures to validate the consistency and convergence of bioimpedance measurements. A heuristic evaluation of the usability around the product and mobile application was conducted in parallel by six subject matter experts and validated the design. This study underscores the significance of considering patients' disease activity during intervals between hospital visits and introduces a novel approach to self-RA care.

Interdisciplinary Learning Health System Response to Public Vaccine Concerns.

In Victoria, Australia, jurisdictional vaccine safety service is conducted by SAEFVIC (Surveillance of Adverse Events Following Vaccination in the Community). SAEFVIC developed a public Vaccine Safety Report (saefvic.online/vaccinesafety) to present key surveillance information. This study applies an interdisciplinary learning health system approach to evaluate the report, taking into consideration public expressions of concern on social media.

Development of a new set of Heuristics for the evaluation of Human-Robot Interaction in industrial settings: Heuristics Robots Experience (HEUROBOX).

Humans and robots will increasingly have to work together in the new industrial context. Therefore, it is necessary to improve the User Experience, Technology Acceptance, and overall wellbeing to achieve a smoother and more satisfying interaction while obtaining the maximum performance possible out of it. For this reason, it is essential to analyze these interactions to enhance User Experience. The heuristic evaluation is an easy-to-use, low-cost method that can be applied at different stages of a design process in an iterative manner. Despite these advantages, there is rarely a list of heuristics in the current literature that evaluates Human-Robot interactions both from a User Experience, Technology Acceptance, and Human-Centered approach. Such an approach should integrate key aspects like safety, trust, and perceived safety, ergonomics and workload, inclusivity, and multimodality, as well as robot characteristics and functionalities. Therefore, a new set of heuristics, namely, the HEUROBOX tool, is presented in this work in the form of the HEUROBOX tool to help practitioners and researchers in the assessment of human-robot systems in industrial environments. The HEUROBOX tool clusters design guidelines and methodologies as a logic list of heuristics for human-robot interaction and comprises four categories: Safety, Ergonomics, Functionality, and Interfaces. They include 84 heuristics in the basic evaluation, while the advanced evaluation lists a total of 228 heuristics in order to adapt the tool to the evaluation of different industrial requirements. Finally, the set of new heuristics has been validated by experts using the System Usability Scale (SUS) questionnaire and the categories has been prioritized in order of their importance in the evaluation of Human-Robot Interaction through the Analytic Hierarchy Process (AHP).

Exploring usability problems of mHealth applications developed for cervical cancer: An empirical study.

Objectives: Nowadays, mobile health applications are developed to raise awareness and facilitate screening and treatment of cervical cancer, while a very few studies have been conducted focusing on the measurement and assurance of usability and exploring the acceptable user experience of such applications. Usability issues become a crucial concern for such cervical-cancer-related applications because users with diverse backgrounds in terms of education, information technology literacy, and geographic reasons are required to access those applications. The objective of this research is to evaluate the usability of mobile health applications developed for cervical cancer patients. Methods: Two evaluation studies were conducted following the expert evaluation and a questionnaire-based user study. A total of four cervical-cancer-related applications that are focusing on the Awareness and Diagnosis theme were selected and each of the applications was evaluated by four usability experts. Then, a user study (n = 80) based on the Goal Question Metric was conducted to reveal the usability problems of four selected applications. Finally, findings of both evaluations were aggregated and analyzed. Results: Both approaches showed that all applications suffer from several usability problems while "Cervical Cancer Guide" performs better and "Cervical Cancer Tracker" showed the least in performance from the usability perspective. Again, the Goal Question Metric performs noticeably better in assessing the learnability of the applications, while the analytical heuristic evaluation performs better in identifying the issues that cause user annoyance. Conclusion: The methodology adopted and the usability problems revealed through this study can be well utilized by the information technology professionals or user interface designers for designing, evaluating, and developing the cervical-cancer-related applications with enhanced usability and user experience.

Evaluating narrative visualization: a survey of practitioners.

Narrative visualization is characterized by the integration of data visualization and storytelling techniques. These characteristics provide challenges in its evaluation. Little is known about how these evaluation challenges are addressed by narrative visualization practitioners. We surveyed experienced narrative visualization practitioners to investigate their methods of evaluation. To gain deeper insight we conducted a series of semi-structured interviews with practitioners. We found that there is usually an informal approach to narrative visualization evaluation, where practitioners rely on prior experience and their peers for evaluation. Our study also revealed novel approaches to evaluation. We introduce a practice-led heuristic framework to aid practitioners to evaluate narrative visualization systematically. Our practice-led heuristic framework couples first-hand practitioner experience with recent research literature. This work sheds light on how to address narrative visualization evaluation to better inform both academic research and practice.

Evaluating the Usability of an Emergency Department After Visit Summary: Staged Heuristic Evaluation.

BACKGROUND: Heuristic evaluations, while commonly used, may inadequately capture the severity of identified usability issues. In the domain of health care, usability issues can pose different levels of risk to patients. Incorporating diverse expertise (eg, clinical and patient) in the heuristic evaluation process can help assess and address potential negative impacts on patient safety that may otherwise go unnoticed. One document that should be highly usable for patients-with the potential to prevent adverse outcomes-is the after visit summary (AVS). The AVS is the document given to a patient upon discharge from the emergency department (ED), which contains instructions on how to manage symptoms, medications, and follow-up care. OBJECTIVE: This study aims to assess a multistage method for integrating diverse expertise (ie, clinical, an older adult care partner, and health IT) with human factors engineering (HFE) expertise in the usability evaluation of the patient-facing ED AVS. METHODS: We conducted a three-staged heuristic evaluation of an ED AVS using heuristics developed for use in evaluating patient-facing documentation. In stage 1, HFE experts reviewed the AVS to identify usability issues. In stage 2, 6 experts of varying expertise (ie, emergency medicine physicians, ED nurses, geriatricians, transitional care nurses, and an older adult care partner) rated each previously identified usability issue on its potential impact on patient comprehension and patient safety. Finally, in stage 3, an IT expert reviewed each usability issue to identify the likelihood of successfully addressing the issue. RESULTS: In stage 1, we identified 60 usability issues that violated a total of 108 heuristics. In stage 2, 18 additional usability issues that violated 27 heuristics were identified by the study experts. Impact ratings ranged from all experts rating the issue as "no impact" to 5 out of 6 experts rating the issue as having a "large negative impact." On average, the older adult care partner representative rated usability issues as being more significant more of the time. In stage 3, 31 usability issues were rated by an IT professional as "impossible to address," 21 as "maybe," and 24 as "can be addressed." CONCLUSIONS: Integrating diverse expertise when evaluating usability is important when patient safety is at stake. The non-HFE experts, included in stage 2 of our evaluation, identified 23% (18/78) of all the usability issues and, depending on their expertise, rated those issues as having differing impacts on patient comprehension and safety. Our findings suggest that, to conduct a comprehensive heuristic evaluation, expertise from all the contexts in which the AVS is used must be considered. Combining those findings with ratings from an IT expert, usability issues can be strategically addressed through redesign. Thus, a 3-staged heuristic evaluation method offers a framework for integrating context-specific expertise efficiently, while providing practical insights to guide human-centered design.

A Tablet-Based Aphasia Assessment System "STELA": Feasibility and Validation Study.

BACKGROUND: There is an extensive library of language tests, each with excellent psychometric properties; however, many of the tests available take considerable administration time, possibly bearing psychological strain on patients. The Short and Tailored Evaluation of Language Ability (STELA) is a simplified, tablet-based language ability assessment system developed to address this issue, with a reduced number of items and automated testing process. OBJECTIVE: The aim of this paper is to assess the administration time, internal consistency, and validity of the STELA. METHODS: The STELA consists of a tablet app, a microphone, and an input keypad for clinician's use. The system is designed to assess language ability with 52 questions grouped into 2 comprehension modalities (auditory comprehension and reading comprehension) and 3 expression modalities (naming and sentence formation, repetition, and reading aloud). Performance in each modality was scored as the correct answer rate (0-100), and overall performance expressed as the sum of modality scores (out of 500 points). RESULTS: The time taken to complete the STELA was significantly less than the time for the WAB (mean 16.2, SD 9.4 vs mean 149.3, SD 64.1 minutes; P<.001). The STELA's total score was strongly correlated with the WAB Aphasia Quotient (r=0.93, P<.001), supporting the former's concurrent validity concerning the WAB, which is a gold-standard aphasia assessment. Strong correlations were also observed at the subscale level; STELA auditory comprehension versus WAB auditory comprehension (r=0.75, P<.001), STELA repetition versus WAB repetition (r=0.96, P<.001), STELA naming and sentence formation versus WAB naming and word finding (r=0.81, P<.001), and the sum of STELA reading comprehension or reading aloud versus WAB reading (r=0.82, P<.001). Cronbach α obtained for each modality was .862 for auditory comprehension, .872 for reading comprehension, .902 for naming and sentence formation, .787 for repetition, and .892 for reading aloud. Global Cronbach α was .961. The average of the values of item-total correlation to each subscale was 0.61 (SD 0.17). CONCLUSIONS: Our study confirmed significant time reduction in the assessment of language ability and provided evidence for good internal consistency and validity of the STELA tablet-based aphasia assessment system.

A model for design of online health professions education faculty development courses in sub-Saharan Africa.

The design of faculty development courses requires a contextually aware theoretical foundation. Therefore, the study aimed to determine the feasibility of a specific conjecture map as a model for designing online health professions education faculty development courses in sub-Saharan Africa through a heuristic evaluation approach. The authors hypothesise that using heuristic evaluation strengthens an educational model's theoretical basis and feasibility.Twenty-five health professions educators from nine sub-Saharan African countries participated in this multi-phased study. In the first phase, the participants completed a survey evaluating the model and the accompanying author-generated heuristics. Then, participants' recommendations were used to make revisions. In the subsequent phase, participants reviewed these revisions and commented on the feasibility of the model within their local context.All six heuristics were revised following the initial phase, where 80 problems had been identified. Upon revision, the model was deemed feasible by all except one participant. There was a strong relationship and inter-rater agreement of feasibility between 0.84 and 0.95 regarding the model's practicality, demand, acceptability, and adaptability. Revisions to the final model and guidance documents incorporated all changes the participants recommended, confirming the model's feasibility.Theoretical models are often developed through a top-down approach, omitting the practice-based considerations that could change the formulation of the model. This study demonstrates the convergence of a theoretical and process model with theoretical, expert and end-user data triangulation. Further research is needed to test this empirically developed model.

Usability evaluation of circRNA identification tools: Development of a heuristic-based framework and analysis.

BACKGROUND AND OBJECTIVE: Circular RNAs (circRNAs) are endogenous molecules of non-coding RNA that form a covalently closed loop at the 3' and 5' ends. Recently, the role of these molecules in the regulation of gene expression and their involvement in several human pathologies has gained notoriety. The identification of circRNAs is highly dependent on computational methods for analyzing RNA sequencing data. However, bioinformatics software is known to be problematic in terms of usability. Evidence points out that tools for identifying circRNAs can have such problems, negatively impacting researchers in this field. Here we present a heuristic-based framework for evaluating the usability of command-line circRNA identification software. METHODS: We used heuristics evaluation to comprehensively identify the usability issues in a sample of circRNA identification tools. RESULTS: We identified 46 usability issues presented individually in four tools. Most of the issues had cosmetic or minor severity. These are unlikely to challenge experienced users but may cause inconvenience for novice users. We also identified severe issues with the potential to harm users regardless of their experience. The areas most affected were the documentation and the installability of the tools. CONCLUSIONS: With the proposed framework, we formally describe, for the first time, the usability problems that can affect users in this area of circRNA research. We hope that our framework can help researchers evaluate their software's usability during development.

Comparison of usability evaluation methods for a health information system: heuristic evaluation versus cognitive walkthrough method.

BACKGROUND: There are differences of opinion regarding the selection of the most practical usability evaluation method among different methods. The present study aimed to compare two expert-based evaluation methods in order to assess a nursing module as the most widely used module of a Hospital Information System (HIS). METHODS: Five independent evaluators used the Heuristic Evaluation (HE) and Cognitive Walkthrough (CW) methods to evaluate the nursing module of Shafa HIS. In this regard, the number and severity of the recognized problems according to the usability attributes were compared using two evaluation methods. RESULTS: The HE and CW evaluation methods resulted in the identification of 104 and 24 unique problems, respectively, of which 33.3% of recognized problems in the CW evaluation method overlapped with the HE method. The average severity of the recognized problems was considered to be minor (2.34) in the HE method and major (2.77) in the CW evaluation method. There was a significant difference in terms of the total number and average severity of the recognized problems by these methods (P < 0.001). Based on the usability attribute, the HE method identified a larger number of problems concerning all usability attributes, and a significant difference was observed in terms of the number of recognized problems in both methods for all attributes except 'memorability'. Also, there was a significant difference between the two methods based on the average severity of recognized problems only in terms of 'learnability'. CONCLUSION: The HE method identified more problems with lower average severity while the CW was able to recognize fewer problems with higher average severity. Regarding the evaluation goal, the HE method was able to be used to improve the effectiveness and satisfaction of the HIS. Furthermore, the CW evaluation method is recommended to identify usability problems with the highest average severity, especially in terms of 'learnability'.

Serious game design model for language learning in the cultural context.

Digital learning environments have been gaining prominence during the last few years. In particular, the rising usage of mobile devices, including smartphones and tabs, has invited researchers to design and develop learning applications and games for such platforms. Mobile applications and games have been developed for learning languages like many other domains. However, most of these games are fun-based and lack a holistic design and development approach. Therefore, as a principal contribution, this research presents a theoretical model for designing language learning games in a cultural context. The proposed model combines the elements of sociocultural theory with the concepts and elements of gamification, keeping in view the requirements and educational settings, including level and mode of education, etc., to ensure the effectiveness and usability of the developed game. Subsequently, based on the proposed model, a Language Learning Game (LLG) has been designed and developed through a systematic process that involves game design, low-fidelity, and high-fidelity prototyping and its validation. The LLG has been evaluated comprehensively at different stages by incorporating standard methods. Whereby this research augments the existing set of heuristics by proposing a number of specialized heuristics for the evaluation of serious games to gauge their conformance to the cultural context. The evaluation results show that the game has overall usability scores of 90%. While the quasi-experiment-based pre-test and post-test have been conducted, the results reveal that the results obtained by LLG are statistically significantly better than adopted mobile application and traditional group.

Enriching the Value of Patient Experience Feedback: Web-Based Dashboard Development Using Co-design and Heuristic Evaluation.

BACKGROUND: There is an abundance of patient experience data held within health care organizations, but stakeholders and staff are often unable to use the output in a meaningful and timely way to improve care delivery. Dashboards, which use visualized data to summarize key patient experience feedback, have the potential to address these issues. OBJECTIVE: The aim of this study is to develop a patient experience dashboard with an emphasis on Friends and Family Test (FFT) reporting, as per the national policy drive. METHODS: A 2-stage approach was used-participatory co-design involving 20 co-designers to develop a dashboard prototype, followed by iterative dashboard testing. Language analysis was performed on free-text patient experience data from the FFT, and the themes and sentiments generated were used to populate the dashboard with associated FFT metrics. Heuristic evaluation and usability testing were conducted to refine the dashboard and assess user satisfaction using the system usability score. RESULTS: The qualitative analysis from the co-design process informed the development of the dashboard prototype with key dashboard requirements and a significant preference for bubble chart display. The heuristic evaluation revealed that most cumulative scores had no usability problems (18/20, 90%), had cosmetic problems only (7/20, 35%), or had minor usability problems (5/20, 25%). The mean System Usability Scale score was 89.7 (SD 7.9), suggesting an excellent rating. CONCLUSIONS: The growing capacity to collect and process patient experience data suggests that data visualization will be increasingly important in turning feedback into improvements to care. Through heuristic usability, we demonstrated that very large FFT data can be presented in a thematically driven, simple visual display without the loss of the nuances and still allow for the exploration of the original free-text comments. This study establishes guidance for optimizing the design of patient experience dashboards that health care providers find meaningful, which in turn drives patient-centered care.

Application of Anti-Stigma Design Heuristics for Usability Inspection.

User interface evaluation has become important in developing usable health care technologies. Although usability engineering methods have been applied in the design and evaluation of health care software, available heuristics focus on task-work aspects and do not address stigma associated with many health conditions. We used a previous set of heuristics and propose a new set of anti-stigma heuristics to evaluate stigmatization in health care websites. The extended set of heuristics were concurrently applied in a heuristic evaluation and a cognitive walkthrough to evaluate an endometriosis and sexual pain website. The walkthrough involved 5 tasks that required 21 actions to execute. Twenty-six usability problems were identified and recommendations for re-design were made to the design team before end-user testing. The anti-stigma heuristics received worse ratings than the traditional heuristics, resulting in several design changes that might otherwise have been missed. Thus, the new anti-stigma heuristics were a valuable contribution.

Usability of Telemedicine Mobile Applications during COVID-19 in Saudi Arabia: A Heuristic Evaluation of Patient User Interfaces.

The coronavirus disease 2019 (COVID-19) pandemic has impacted the use of telemedicine application (apps), which has seen an uprise. This study evaluated the usability of the user interface design of telemedicine apps deployed during the COVID-19 pandemic in Saudi Arabia. It also explored changes to the apps' usability based on the pandemic timeline. METHODS: We screened ten mHealth apps published by the National Digital Transformation Unit and selected three telemedicine apps: (1) governmental "Seha"® app, (2) stand-alone "Cura"® app, and (3) private "Dr. Sulaiman Alhabib"®app. We conducted the evaluations in April 2020 and in June 2021 by identifying positive app features, using Nielsen's ten usability heuristics with a five-point severity rating scale, and documenting redesign recommendations. RESULTS: We identified 54 user interface usability issues during both evaluation periods: 18 issues in "Seha" 14 issues in "Cura", and 22 issues in "Dr. Sulaiman Alhabib". The two most heuristic items violated in "Seha", were "user control and freedom" and "recognition rather than recall". In "Cura", the three most heuristic items violated were "consistency and adherence to standards", "esthetic and minimalist design", and "help and documentation" In "Dr. Sulaiman Alhabib" the most heuristic item violated was "error prevention". Ten out of the thirty usability issues identified from our first evaluation were no longer identified during our second evaluation. CONCLUSIONS: our findings indicate that all three apps have a room for improving their user interface designs to improve the overall user experience and to ensure the continuity of these services beyond the pandemic.

Usability of state public health department websites for communication during a pandemic: A heuristic evaluation.

Study aims were to investigate how usable COVID-19 dedicated state public health websites in the US were, and whether case counts in different geographical regions in the US were related to website usability. 16 state websites representing the 2 highest and the 2 lowest case count states in each region were selected. Five experts used a heuristic evaluation procedure to independently rate all 16 websites on a severity scale of 0-4. Usability criteria published by the US Department of Health and Human Services and criteria on risk communication and data dashboards were used. Analyses involved cross tabulation of usability criteria with case counts, comparison of usability scores using Mood's median tests, test of differences in average usability scores using ANOVA and post-hoc tests, and identification of correlations between case counts and usability scores. Results from the Mood's median test showed that the median usability scores for the states were significantly different from each other at the 5% level of significance (df = 15, chi-square = 38.40; p = 0.001). ANOVA showed statistically significant differences between the mean usability scores for the states at the 5% level of significance (F = 6.33, p < 0.05). Although not statistically significant, results from a correlation analysis between case count and usability scores showed a negative correlation (r = -0.209, p = 0.437) indicating that the higher the case count, the better the usability score. Overall, the websites fared well on usability, but many websites were used as an information and data repository. These websites must communicate infection risk better. RELEVANCE TO INDUSTRY: The study applies to public health agency websites that communicate essential information during a pandemic.

Mobile Health for Smoking Cessation Among Disadvantaged Young Women During and After Pregnancy: User-Centered Design and Usability Study.

BACKGROUND: Smoking prevalence during and after pregnancy remains high among socioeconomically disadvantaged women. Mobile health (mHealth) apps with game and social support elements seem promising to support smoking cessation. OBJECTIVE: This study aims to describe the user-centered design and usability evaluation of Kindle, an mHealth app with game and social support elements, to support disadvantaged young women during and after pregnancy through the first stages of smoking cessation. METHODS: Disadvantaged women (n=9), members of their social networks (n=4), and nurses supporting these women (n=51) were informants throughout the iterative prototype development of Kindle according to the International Organization for Standardization 9241-11:2018. Specific phases included understanding the context of use through secondary analysis of qualitative interview data (phase 1), establishing the user and organizational requirements (phase 2), production of design solutions (phase 3), and usability inspection of the prototype through a heuristic evaluation (3 experts) along with user testing by a think aloud method (5 disadvantaged women and 5 nurses; phase 4). Usability problems were categorized according to the principles of the Healthcare Information and Management Systems Society. RESULTS: Phase 1 resulted in an understanding of the VoorZorg program and the needs of VoorZorg nurses and clients (eg, focus on early stages of change and building new supportive networks to aid clients in smoking cessation). In phase 2, we established requirements (n=22; eg, mHealth app, secure communication between nurses and clients, easy-to-use interfaces, inclusion of game elements, and tailoring at early stages of change in smoking cessation). Phase 3 resulted in a prototype of Kindle, combining the interface for nurses and clients, including the following functionalities: personal goal setting with earning points; secured chat function between nurses and other clients; and tips, diary, and profile creation. The heuristic evaluation and thinking aloud method in phase 4 revealed 78 usability problems in the interfaces. Most usability problems concerned simplicity (eg, unclear clickable button) and naturalness (eg, unclear icon). CONCLUSIONS: The user-centered design and usability testing of the mHealth app Kindle yielded useful insights. The involvement of end users, specifically socioeconomically disadvantaged women during and after their pregnancy, resulted in a prototype that met their needs and requirements (eg, mHealth app, secure communication between nurses and clients, easy-to-use interfaces, inclusion of game elements, and tailoring to the early stages of change in smoking cessation) to achieve readiness for smoking cessation. Moreover, the usability evaluation by end users and experts revealed unique usability problems for this population. These insights allow for further optimization of Kindle and encourage future studies to engage disadvantaged populations in all phases of mHealth intervention design and usability testing.

Evaluación de usabilidad de sitios web de bibliotecas de universidades públicas chilenas a partir del método SIRIUS / Usability evaluation of chilean academic public library web sites using the SIRIUS method

Resumen Este artículo presenta los resultados de una evaluación heurística de usabilidad aplicada a 17 sitios web de bibliotecas de universidades públicas chilenas, a través del método SIRIUS, "Sistema de Evaluación de la Usabilidad Web Orientado al Usuario y Basado en la Determinación de Tareas Críticas". Para la recolección de datos se utilizó la pauta fija de SIRIUS que contiene 10 heurísticas subdivididas en un total de 83 sub-heurísticas evaluadas por cinco personas expertas con conocimiento en sitios web de bibliotecas y usabilidad. Los resultados indican que las bibliotecas universitarias mejor evaluadas presentan buenos mecanismos de rotulado, layout (diseño) y facilidad de interacción, mientras que, en su conjunto, los sitios web carecen de ayudas de navegación para las personas usuarias. Finalmente, se concluye que la evaluación heurística es un método válido, rápido, fácil y aplicable en sitios web de bibliotecas universitarias, siempre y cuando se definan adecuadamente las personas expertas y las heurísticas. Asimismo, SIRIUS resulta una herramienta adecuada al contar con una pauta fija que permite obtener evaluaciones con rapidez y facilitar la comparación de sus resultados.

Abstract This article presents the results of a heuristic usability evaluation applied to 17 websites of libraries of public chilean universities, using the SIRIUS method "System of Web Usability Evaluation, User Oriented and Based on the Determination of Critical Tasks" For data collection, SIRIUS fixed guideline was used, which contains 10 heuristics that are subdivided into a total of 83 sub-heuristics evaluated by 5 experts with knowledge about libraries and usability websites. The results indicate that the best evaluated university libraries have good labeling mechanisms, layout and ease of interaction, while, as a whole, the websites lack navigation aids for their users. Finally, it is concluded that heuristic evaluation is a valid, fast, easy and applicable method in university library websites, as long as experts and heuristics are properly defined. Likewise, SIRIUS is an adequate tool since it has a fixed guideline that allows evaluations to be obtained quickly and to facilitate the comparison of its results.