ABSTRACT
BACKGROUND: In the past years, a mobile health (mHealth) app called the Dutch Talking Touch Screen Questionnaire (DTTSQ) was developed in The Netherlands. The aim of development was to enable Dutch physical therapy patients to autonomously complete a health-related questionnaire regardless of their level of literacy and digital skills. OBJECTIVE: The aim of this study was to evaluate the usability (defined as the effectiveness, efficiency, and satisfaction) of the prototype of the DTTSQ for Dutch physical therapy patients with diverse levels of experience in using mobile technology. METHODS: The qualitative Three-Step Test-Interview method, including both think-aloud and retrospective probing techniques, was used to gain insight into the usability of the DTTSQ. A total of 24 physical therapy patients were included. The interview data were analyzed using a thematic content analysis approach aimed at analyzing the accuracy and completeness with which participants completed the questionnaire (effectiveness), the time it took the participants to complete the questionnaire (efficiency), and the extent to which the participants were satisfied with the ease of use of the questionnaire (satisfaction). The problems encountered by the participants in this study were given a severity rating that was used to provide a rough estimate of the need for additional usability efforts. RESULTS: All participants within this study were very satisfied with the ease of use of the DTTSQ. Overall, 9 participants stated that the usability of the app exceeded their expectations. The group of 4 average-/high-experienced participants encountered only 1 problem in total, whereas the 11 little-experienced participants encountered an average of 2 problems per person and the 9 inexperienced participants an average of 3 problems per person. A total of 13 different kind of problems were found during this study. Of these problems, 4 need to be addressed before the DTTSQ will be released because they have the potential to negatively influence future usage of the tool. The other 9 problems were less likely to influence future usage of the tool substantially. CONCLUSIONS: The usability of the DTTSQ needs to be improved before it can be released. No problems were found with satisfaction or efficiency during the usability test. The effectiveness needs to be improved by (1) making it easier to navigate through screens without the possibility of accidentally skipping one, (2) enabling the possibility to insert an answer by tapping on the text underneath a photograph instead of just touching the photograph itself, and (3) making it easier to correct wrong answers. This study shows the importance of including less skilled participants in a usability study when striving for inclusive design and the importance of measuring not just satisfaction but also efficiency and effectiveness during such studies.
ABSTRACT
BACKGROUND: Low-educated patients are disadvantaged in using questionnaires within the health care setting because most health-related questionnaires do not take the educational background of patients into account. The Dutch Talking Touch Screen Questionnaire (DTTSQ) was developed in an attempt to meet the needs of low-educated patients by using plain language and adding communication technology to an existing paper-based questionnaire. For physical therapists to use the DTTSQ as part of their intake procedure, it needs to generate accurate information from all of their patients, independent of educational level. OBJECTIVE: The aim of this study was to get a first impression of the information that is generated by the DTTSQ. To achieve this goal, response processes of physical therapy patients with diverse levels of education were analyzed. METHODS: The qualitative Three-Step Test-Interview method was used to collect observational data on actual response behavior of 24 physical therapy patients with diverse levels of education. The interviews included both think-aloud and retrospective probing techniques. RESULTS: Of the 24 respondents, 20 encountered one or more problems during their response process. The use of plain language and information and communication technology (ICT) appeared to have a positive effect on the comprehensibility of the DTTSQ. However, it also had some negative effects on the interpretation, retrieval, judgment, and response selection within the response processes of the participants in this study. No educational group in this research population stood out from the rest in the kind or number of problems that arose. All respondents recognized themselves in the outcomes of the questionnaire. CONCLUSIONS: The use of plain language and ICT within the DTTSQ had both positive and negative effects on the response processes of its target population. The results of this study emphasize the importance of earlier recommendations to accompany any adaption of any questionnaire to a new mode of delivery by demonstrating the difference and equivalence between the two different modes and to scientifically evaluate the applicability of the newly developed mode of the questionnaire in its intended setting. This is especially important in a digital era in which the use of plain language within health care is increasingly being advocated.
