Empowering Deaf-Hearing Communication: Exploring Synergies between Predictive and Generative AI-Based Strategies towards (Portuguese) Sign Language Interpretation.

Communication between Deaf and hearing individuals remains a persistent challenge requiring attention to foster inclusivity. Despite notable efforts in the development of digital solutions for sign language recognition (SLR), several issues persist, such as cross-platform interoperability and strategies for tokenizing signs to enable continuous conversations and coherent sentence construction. To address such issues, this paper proposes a non-invasive Portuguese Sign Language (Língua Gestual Portuguesa or LGP) interpretation system-as-a-service, leveraging skeletal posture sequence inference powered by long-short term memory (LSTM) architectures. To address the scarcity of examples during machine learning (ML) model training, dataset augmentation strategies are explored. Additionally, a buffer-based interaction technique is introduced to facilitate LGP terms tokenization. This technique provides real-time feedback to users, allowing them to gauge the time remaining to complete a sign, which aids in the construction of grammatically coherent sentences based on inferred terms/words. To support human-like conditioning rules for interpretation, a large language model (LLM) service is integrated. Experiments reveal that LSTM-based neural networks, trained with 50 LGP terms and subjected to data augmentation, achieved accuracy levels ranging from 80% to 95.6%. Users unanimously reported a high level of intuition when using the buffer-based interaction strategy for terms/words tokenization. Furthermore, tests with an LLM-specifically ChatGPT-demonstrated promising semantic correlation rates in generated sentences, comparable to expected sentences.

The recognition of web pages' hyperlinks by people with intellectual disabilities: an evaluation study.

BACKGROUND: One of the most mentioned problems of web accessibility, as recognized in several different studies, is related to the difficulty regarding the perception of what is or is not clickable in a web page. In particular, a key problem is the recognition of hyperlinks by a specific group of people, namely those with intellectual disabilities. MATERIALS AND METHODS: This experiment investigated a methodology based on the direct observation, video recording, interview and data obtained by an eye tracker device. Ten participants took part in this study. They were divided into two groups and asked to perform two tasks: 'Sing a song' and 'Listen to a story' in two websites. These websites were developed to include specific details. The first website presented an image navigation menu (INM), whereas the other one showed a text navigation menu (TNM). RESULTS: There was a general improvement regarding the participants' performance when using INMs. CONCLUSION: The referred analysis indeed shows that not only did these specific participants gain a better understanding of the demanding task, but also they showed an improved perception concerning the content of the navigation menu that included hyperlinks with images.