RESUMO
The evaluation of clinical utility is essential for the successful adoption of new technology in clinical practice. An approach to evaluating clinical utility is presented here using the example of digitized measurement instruments.
Assuntos
Assistência ao Paciente , Humanos , Assistência ao Paciente/normasRESUMO
Background: The Box and Block Test (BBT) is a clinical tool used to measure hand dexterity, which is often used for tracking disease progression or the effectiveness of therapy, particularly benefiting older adults and those with neurological conditions. Digitizing the measurement of hand function may enhance the quality of data collection. We have developed and validated a prototype that digitizes this test, known as the digital BBT (dBBT), which automatically measures time and determines and displays the test result. Objective: This study aimed to investigate the clinical utility and usability of the newly developed dBBT and to collect suggestions for future improvements. Methods: A total of 4 occupational therapists participated in our study. To evaluate the clinical utility, we compared the dBBT to the BBT across dimensions such as acceptance, portability, energy and effort, time, and costs. We observed therapists using the dBBT as a dexterity measurement tool and conducted a quantitative usability questionnaire using the System Usability Scale (SUS), along with a focus group. Evaluative, structured, and qualitative content analysis was used for the qualitative data, whereas quantitative analysis was applied to questionnaire data. The qualitative and quantitative data were merged and analyzed using a convergent mixed methods approach. Results: Overall, the results of the evaluative content analysis suggested that the dBBT had a better clinical utility than the original BBT, with ratings of all collected participant statements for the dBBT being 45% (45/99) equal to, 48% (48/99) better than, and 6% (6/99) lesser than the BBT. Particularly in the subcategories "acceptance," "time required for evaluation," and "purchase costs," the dBBT was rated as being better than the original BBT. The dBBT achieved a mean SUS score of 83 (95% CI 76-96). Additionally, several suggested changes to the system were identified. Conclusions: The study demonstrated an overall positive evaluation of the clinical utility and usability of the dBBT. Valuable insights were gathered for future system iterations. These pioneering results highlight the potential of digitizing hand dexterity assessments.
RESUMO
BACKGROUND: The Box and Block Test (BBT) measures unilateral gross manual dexterity and is widely used in clinical settings with a wide range of populations, including older people and clients with neurological disorders. OBJECTIVE: In this study, we present a newly developed digitized version of the BBT, called the digital BBT (dBBT). The physical design is similar to the original BBT, but the dBBT contains digital electronics that automate the test procedure, timing, and score measurement. The aim of this study is to investigate the validity and reliability of the dBBT. METHODS: We performed measurements at 2 time points for 29 healthy participants. BBT and dBBT were used at the first measurement time point, and dBBT was used again at the second measurement time point. Concurrent validity was assessed using the correlation between BBT and dBBT, the paired t test, and the Bland-Altman analysis. Test-retest reliability and interrater reliability were examined using the interclass correlation coefficient (ICC) by repeated measures with the dBBT within an interval of 10 days. RESULTS: Our results showed moderate concurrent validity (r=0.48, P=.008), moderate test-retest reliability (ICC 0.72, P<.001), a standard error of measurement of 3.1 blocks, and the smallest detectable change at a 95% CI of 8.5 blocks. Interrater reliability was moderate with an ICC of 0.67 (P=.02). The Bland-Altman analysis showed sufficient accuracy of the dBBT in comparison with the conventional BBT. CONCLUSIONS: The dBBT can contribute to objectifying the measurement of gross hand dexterity without losing its important characteristics and is simple to implement.