MyCOVIDrisk: User Experience Study Of COVID-19 Risk Assessment and Mitigation Application

ABSTRACT

Background: MyCOVIDRisk app is a free, publicly available COVID-19 risk estimation and mitigation tool. The MyCOVIDRisk app has since been accessed almost 1.3 million times since launch, demonstrating the acceptability of a free and simple web-based mobile application to estimate risk of COVID-19 transmission. Little is known about how mobile apps influence assessments of risk. User experience (“UX”) studies are a key strategy for examining usability and influence of digital technology. Study Objectives: Our primary objectives were to (1) describe how the app informs interpretation of COVID-19 risk, (2) describe motivations for use and patterns of use, (3) to inform future app design. Methods: This UX study consisted of two parts. Part 1 focused on new users’ experiences, particularly navigation of the user interface across various operating systems and devices. Part 2 focused on repeat users’ experiences, particularly how they interpreted risk, what motivated their use of the app, and whether it modified behavior. Participants were recruited remotely via social media advertisements on Facebook, Instagram, and Twitter. To reach existing users, Part 2 additionally included emailing MyCOVIDRisk users who previously sent unsolicited feedback on the app. Study participants were entered into raffles to win one of two $50 Amazon gift codes. For Part 1, users completed a series of tasks during a semi-structured 30-minute video interview using the app while sharing first impressions, likes, and dislikes. Live notes taken tracked common user errors, points of confusion, and other insights. Part 2 consisted of 40-minute semi-structured video interviews with repeat users. Participants shared their personal pandemic experiences, related health decisionmaking processes, and their experience with the MyCOVIDRisk app. Interviews were audio recorded, transcribed, and analyzed to find common themes and subthemes. Results: Recruitment continued until thematic saturation was reached. Part 1 and Part 2 included 8 and 5 unique participants, respectively. Participants varied in terms of background (age range 21-73 years), geography, prior use of the app, and goals of usage. Key use cases were as a teaching tool, source of authority and objectivity, and resource for personal decisionmaking (Fig 1). Nearly all pointed to simplicity and ease of use as key design strengths. Repeat users highlighted the interactive nature and ability to change parameters, eg, “when it [MyCOVIDRisk App] came out, I used it you know like a hundred times-like what if I do this or what if I do that? Like almost like a video game!” Challenges included difficulty in estimating number of people present and percent that would be masked, and desire for more complex activities than the prepopulated options. Users nearly unanimously suggested integration of vaccine status as an input parameter. Other common recommendations included more customization options (eg, ability to change font size), having more information when hovering over icons, and options to send inputs and risk score results to others or to print them for documentation. Conclusion: This UX testing of a COVID-19 risk assessment and mitigation app confirmed key principles for design: clear imagery, interactivity, and interpretable science. Future work should incorporate new data in real-time and improve customizability. [Formula presented] Figure 1. Primary cases for MyCOVIDRisk app based on interview responses from repeat users.