Real-time multilingual speech recognition and speaker diarization system based on Whisper segmentation.

This research presents the development of a cutting-edge real-time multilingual speech recognition and speaker diarization system that leverages OpenAI's Whisper model. The system specifically addresses the challenges of automatic speech recognition (ASR) and speaker diarization (SD) in dynamic, multispeaker environments, with a focus on accurately processing Mandarin speech with Taiwanese accents and managing frequent speaker switches. Traditional speech recognition systems often fall short in such complex multilingual and multispeaker contexts, particularly in SD. This study, therefore, integrates advanced speech recognition with speaker diarization techniques optimized for real-time applications. These optimizations include handling model outputs efficiently and incorporating speaker embedding technology. The system was evaluated using data from Taiwanese talk shows and political commentary programs, featuring 46 diverse speakers. The results showed a promising word diarization error rate (WDER) of 2.68% in two-speaker scenarios and 11.65% in three-speaker scenarios, with an overall WDER of 6.96%. This performance is comparable to that of non-real-time baseline models, highlighting the system's ability to adapt to various complex conversational dynamics, a significant advancement in the field of real-time multilingual speech processing.

Migraine headache (MH) classification using machine learning methods with data augmentation.

Migraine headache, a prevalent and intricate neurovascular disease, presents significant challenges in its clinical identification. Existing techniques that use subjective pain intensity measures are insufficiently accurate to make a reliable diagnosis. Even though headaches are a common condition with poor diagnostic specificity, they have a significant negative influence on the brain, body, and general human function. In this era of deeply intertwined health and technology, machine learning (ML) has emerged as a crucial force in transforming every aspect of healthcare, utilizing advanced facilities ML has shown groundbreaking achievements related to developing classification and automatic predictors. With this, deep learning models, in particular, have proven effective in solving complex problems spanning computer vision and data analytics. Consequently, the integration of ML in healthcare has become vital, especially in developing countries where limited medical resources and lack of awareness prevail, the urgent need to forecast and categorize migraines using artificial intelligence (AI) becomes even more crucial. By training these models on a publicly available dataset, with and without data augmentation. This study focuses on leveraging state-of-the-art ML algorithms, including support vector machine (SVM), K-nearest neighbors (KNN), random forest (RF), decision tree (DST), and deep neural networks (DNN), to predict and classify various types of migraines. The proposed models with data augmentations were trained to classify seven various types of migraine. The proposed models with data augmentations were trained to classify seven various types of migraine. The revealed results show that DNN, SVM, KNN, DST, and RF achieved an accuracy of 99.66%, 94.60%, 97.10%, 88.20%, and 98.50% respectively with data augmentation highlighting the transformative potential of AI in enhancing migraine diagnosis.

An approach to the dermatological classification of histopathological skin images using a hybridized CNN-DenseNet model.

This research addresses the challenge of automating skin disease diagnosis using dermatoscopic images. The primary issue lies in accurately classifying pigmented skin lesions, which traditionally rely on manual assessment by dermatologists and are prone to subjectivity and time consumption. By integrating a hybrid CNN-DenseNet model, this study aimed to overcome the complexities of differentiating various skin diseases and automating the diagnostic process effectively. Our methodology involved rigorous data preprocessing, exploratory data analysis, normalization, and label encoding. Techniques such as model hybridization, batch normalization and data fitting were employed to optimize the model architecture and data fitting. Initial iterations of our convolutional neural network (CNN) model achieved an accuracy of 76.22% on the test data and 75.69% on the validation data. Recognizing the need for improvement, the model was hybridized with DenseNet architecture and ResNet architecture was implemented for feature extraction and then further trained on the HAM10000 and PAD-UFES-20 datasets. Overall, our efforts resulted in a hybrid model that demonstrated an impressive accuracy of 95.7% on the HAM10000 dataset and 91.07% on the PAD-UFES-20 dataset. In comparison to recently published works, our model stands out because of its potential to effectively diagnose skin diseases such as melanocytic nevi, melanoma, benign keratosis-like lesions, basal cell carcinoma, actinic keratoses, vascular lesions, and dermatofibroma, all of which rival the diagnostic accuracy of real-world clinical specialists but also offer customization potential for more nuanced clinical uses.

TranStutter: A Convolution-Free Transformer-Based Deep Learning Method to Classify Stuttered Speech Using 2D Mel-Spectrogram Visualization and Attention-Based Feature Representation.

Stuttering, a prevalent neurodevelopmental disorder, profoundly affects fluent speech, causing involuntary interruptions and recurrent sound patterns. This study addresses the critical need for the accurate classification of stuttering types. The researchers introduce "TranStutter", a pioneering Convolution-free Transformer-based DL model, designed to excel in speech disfluency classification. Unlike conventional methods, TranStutter leverages Multi-Head Self-Attention and Positional Encoding to capture intricate temporal patterns, yielding superior accuracy. In this study, the researchers employed two benchmark datasets: the Stuttering Events in Podcasts Dataset (SEP-28k) and the FluencyBank Interview Subset. SEP-28k comprises 28,177 audio clips from podcasts, meticulously annotated into distinct dysfluent and non-dysfluent labels, including Block (BL), Prolongation (PR), Sound Repetition (SR), Word Repetition (WR), and Interjection (IJ). The FluencyBank subset encompasses 4144 audio clips from 32 People Who Stutter (PWS), providing a diverse set of speech samples. TranStutter's performance was assessed rigorously. On SEP-28k, the model achieved an impressive accuracy of 88.1%. Furthermore, on the FluencyBank dataset, TranStutter demonstrated its efficacy with an accuracy of 80.6%. These results highlight TranStutter's significant potential in revolutionizing the diagnosis and treatment of stuttering, thereby contributing to the evolving landscape of speech pathology and neurodevelopmental research. The innovative integration of Multi-Head Self-Attention and Positional Encoding distinguishes TranStutter, enabling it to discern nuanced disfluencies with unparalleled precision. This novel approach represents a substantial leap forward in the field of speech pathology, promising more accurate diagnostics and targeted interventions for individuals with stuttering disorders.

Data augmentation and deep neural networks for the classification of Pakistani racial speakers recognition.

Speech emotion recognition (SER) systems have evolved into an important method for recognizing a person in several applications, including e-commerce, everyday interactions, law enforcement, and forensics. The SER system's efficiency depends on the length of the audio samples used for testing and training. However, the different suggested models successfully obtained relatively high accuracy in this study. Moreover, the degree of SER efficiency is not yet optimum due to the limited database, resulting in overfitting and skewing samples. Therefore, the proposed approach presents a data augmentation method that shifts the pitch, uses multiple window sizes, stretches the time, and adds white noise to the original audio. In addition, a deep model is further evaluated to generate a new paradigm for SER. The data augmentation approach increased the limited amount of data from the Pakistani racial speaker speech dataset in the proposed system. The seven-layer framework was employed to provide the most optimal performance in terms of accuracy compared to other multilayer approaches. The seven-layer method is used in existing works to achieve a very high level of accuracy. The suggested system achieved 97.32% accuracy with a 0.032% loss in the 75%:25% splitting ratio. In addition, more than 500 augmentation data samples were added. Therefore, the proposed approach results show that deep neural networks with data augmentation can enhance the SER performance on the Pakistani racial speech dataset.

Multi-label emotion classification of Urdu tweets.

Urdu is a widely used language in South Asia and worldwide. While there are similar datasets available in English, we created the first multi-label emotion dataset consisting of 6,043 tweets and six basic emotions in the Urdu Nastalíq script. A multi-label (ML) classification approach was adopted to detect emotions from Urdu. The morphological and syntactic structure of Urdu makes it a challenging problem for multi-label emotion detection. In this paper, we build a set of baseline classifiers such as machine learning algorithms (Random forest (RF), Decision tree (J48), Sequential minimal optimization (SMO), AdaBoostM1, and Bagging), deep-learning algorithms (Convolutional Neural Networks (1D-CNN), Long short-term memory (LSTM), and LSTM with CNN features) and transformer-based baseline (BERT). We used a combination of text representations: stylometric-based features, pre-trained word embedding, word-based n-grams, and character-based n-grams. The paper highlights the annotation guidelines, dataset characteristics and insights into different methodologies used for Urdu based emotion classification. We present our best results using micro-averaged F1, macro-averaged F1, accuracy, Hamming loss (HL) and exact match (EM) for all tested methods.

Multi-class sentiment analysis of urdu text using multilingual BERT.

Sentiment analysis (SA) is an important task because of its vital role in analyzing people's opinions. However, existing research is solely based on the English language with limited work on low-resource languages. This study introduced a new multi-class Urdu dataset based on user reviews for sentiment analysis. This dataset is gathered from various domains such as food and beverages, movies and plays, software and apps, politics, and sports. Our proposed dataset contains 9312 reviews manually annotated by human experts into three classes: positive, negative and neutral. The main goal of this research study is to create a manually annotated dataset for Urdu sentiment analysis and to set baseline results using rule-based, machine learning (SVM, NB, Adabbost, MLP, LR and RF) and deep learning (CNN-1D, LSTM, Bi-LSTM, GRU and Bi-GRU) techniques. Additionally, we fine-tuned Multilingual BERT(mBERT) for Urdu sentiment analysis. We used four text representations: word n-grams, char n-grams,pre-trained fastText and BERT word embeddings to train our classifiers. We trained these models on two different datasets for evaluation purposes. Finding shows that the proposed mBERT model with BERT pre-trained word embeddings outperformed deep learning, machine learning and rule-based classifiers and achieved an F1 score of 81.49%.

Effect on speech emotion classification of a feature selection approach using a convolutional neural network.

Speech emotion recognition (SER) is a challenging issue because it is not clear which features are effective for classification. Emotionally related features are always extracted from speech signals for emotional classification. Handcrafted features are mainly used for emotional identification from audio signals. However, these features are not sufficient to correctly identify the emotional state of the speaker. The advantages of a deep convolutional neural network (DCNN) are investigated in the proposed work. A pretrained framework is used to extract the features from speech emotion databases. In this work, we adopt the feature selection (FS) approach to find the discriminative and most important features for SER. Many algorithms are used for the emotion classification problem. We use the random forest (RF), decision tree (DT), support vector machine (SVM), multilayer perceptron classifier (MLP), and k-nearest neighbors (KNN) to classify seven emotions. All experiments are performed by utilizing four different publicly accessible databases. Our method obtains accuracies of 92.02%, 88.77%, 93.61%, and 77.23% for Emo-DB, SAVEE, RAVDESS, and IEMOCAP, respectively, for speaker-dependent (SD) recognition with the feature selection method. Furthermore, compared to current handcrafted feature-based SER methods, the proposed method shows the best results for speaker-independent SER. For EMO-DB, all classifiers attain an accuracy of more than 80% with or without the feature selection technique.

LSTM-based sentiment analysis for stock price forecast.

Investing in stocks is an important tool for modern people's financial management, and how to forecast stock prices has become an important issue. In recent years, deep learning methods have successfully solved many forecast problems. In this paper, we utilized multiple factors for the stock price forecast. The news articles and PTT forum discussions are taken as the fundamental analysis, and the stock historical transaction information is treated as technical analysis. The state-of-the-art natural language processing tool BERT are used to recognize the sentiments of text, and the long short term memory neural network (LSTM), which is good at analyzing time series data, is applied to forecast the stock price with stock historical transaction information and text sentiments. According to experimental results using our proposed models, the average root mean square error (RMSE ) has 12.05 accuracy improvement.

Comparative analysis on Facebook post interaction using DNN, ELM and LSTM.

This study presents a novel research approach to predict user interaction for social media post using machine learning algorithms. The posts are converted to vector form using word2vec and doc2vec model. These two methods are used to analyse the best approach for generating word embeddings. The generated word embeddings of post combined with other attributes like post published time, type of post and total interactions are used to train machine learning algorithms. Deep neural network (DNN), Extreme Learning Machine (ELM) and Long Short-Term Memory (LSTM) are used to compare the prediction of total interaction for a particular post. For word2vec, the word vectors are created using both continuous bag-of-words (CBOW) and skip-gram models. Also the pre-trained word vectors provided by google is used for the analysis. For doc2vec, the word embeddings are created using both the Distributed Memory model of Paragraph Vectors (PV-DM) and Distributed Bag of Words model of Paragraph Vectors (PV-DBOW). A word embedding is also created using PV-DBOW combined with skip-gram.

Determinants of user acceptance of a specific social platform for older adults: An empirical examination of user interface characteristics and behavioral intention.

The use of the Internet and social applications has many benefits for the elderly, but numerous investigations have shown that the elderly do not perceive online social networks as a friendly social environment. Therefore, TreeIt, a social application specifically designed for the elderly, was developed for this study. In the TreeIt application, seven mechanisms promoting social interaction were designed to allow older adults to use social networking sites (SNSs) to increase social connection, maintain the intensity of social connections and strengthen social experience. This study's main objective was to investigate how user interface design affects older people's intention and attitude related to using SNSs. Fourteen user interface evaluation heuristics proposed by Zhang et al. were adopted as the criteria to assess user interface usability and further grouped into three categories: system support, user interface design and navigation. The technology acceptance model was adopted to assess older people's intention and attitude related to using SNSs. One hundred and one elderly persons were enrolled in this study as subjects, and the results showed that all of the hypotheses proposed in this study were valid: system support and perceived usefulness had a significant effect on behavioral intention; user interface design and perceived ease of use were positively correlated with perceived usefulness; and navigation exerted an influence on perceived ease of use. The results of this study are valuable for the future development of social applications for the elderly.

The Design and Validation of a Child Developmental e-Screening System.

An effective screening test could significantly impact identification of developmental delays at an early age. However, many studies have shown that delay screenings still use text-based screening survey questionnaires. Unfortunately, the traditional text-based screening method tends to be fairly passive. In addition, the advantages of using an interactive system and animation have been shown to lead to positive effects on learning in medical research. Therefore, a multimedia screening system is necessary. This study constructs a system architecture to develop an e-screening system for child developmental delays. To validate the system after development, this study conducted an experiment and employed a questionnaire to survey users. Five experts and 120 subjects participated in the experiment. After the experiment, the results of the system evaluation revealed excellent agreement between the text-based and multimedia version of Taipei II. A total of 118 (98%) participants preferred the multimedia version or had no preference, and only 2 (2%) preferred the paper version. Regular text-based screening sometimes excludes those with low literacy and those whose native language is different from the text. In addition, text-based screening tools lose users' attention easily. The current study successfully developed a multimedia text-based screening system. Feedback from the participants showed that the e-screening system was well accepted and more easily accessible than the original. In this study, a child developmental delays e-screening system was developed. After the experiment, the subjects indicated that the developmental delay e-screening system increased their comprehension and kept them interested in the screening.

A Database as a Service for the Healthcare System to Store Physiological Signal Data.

Wearable devices that measure physiological signals to help develop self-health management habits have become increasingly popular in recent years. These records are conducive for follow-up health and medical care. In this study, based on the characteristics of the observed physiological signal records- 1) a large number of users, 2) a large amount of data, 3) low information variability, 4) data privacy authorization, and 5) data access by designated users-we wish to resolve physiological signal record-relevant issues utilizing the advantages of the Database as a Service (DaaS) model. Storing a large amount of data using file patterns can reduce database load, allowing users to access data efficiently; the privacy control settings allow users to store data securely. The results of the experiment show that the proposed system has better database access performance than a traditional relational database, with a small difference in database volume, thus proving that the proposed system can improve data storage performance.

mCAF: a multi-dimensional clustering algorithm for friends of social network services.

In recent years, social network services have grown rapidly. The number of friends of each user using social network services has also increased significantly and is so large that clustering and managing these friends has become difficult. In this paper, we propose an algorithm called mCAF that automatically clusters friends. Additionally, we propose methods that define the distance between different friends based on different sets of measurements. Our proposed mCAF algorithm attempts to reduce the effort and time required for users to manage their friends in social network services. The proposed algorithm could be more flexible and convenient by implementing different privacy settings for different groups of friends. According to our experimental results, we find that the improved ratios between mCAF and SCAN are 35.8 % in similarity and 84.9 % in F 1 score.

Perceptions of a Specific Family Communication Application among Grandparents and Grandchildren: An Extension of the Technology Acceptance Model.

Many studies have noted that the use of social networks sites (SNSs) can enhance social interaction among the elderly and that the motivation for the elderly to use SNSs is to keep in contact with remote friends and family or the younger generation. Memotree is designed to promote intergenerational family communication. The system incorporates the Family Tree design concept and provides family communication mechanisms based on the Family Communication Scale. In addition, the system optimizes hardware and interface use to conform to the specific needs of older and substantially younger individuals. Regarding the impact of variables on SNS with respect to the interaction of usability variables in the construction of a cross-generational communication platform, we adopted the TAM model and Chung et al.'s suggestions to promote user acceptance of the proposed Memotree system. A total of 39 grandchildren and 39 grandparents met the criteria and were included in the study. The elderly and young respondents revealed substantial willingness to use and/or satisfaction with using the Memotree system. Empirical results indicate that technology affordances and perceived ease of use have a positive impact on perceived usefulness, while perceived ease of use is affected by technology affordances. Internet self-efficacy and perceived usefulness have a positive impact on the user's behavioral intention toward the system. In addition, this study investigated age as a moderating variable in the model. The results indicate that grandchildren have a larger significant effect on the path between perceived usefulness and behavioral intention than grandparents. This study proposes a more complete framework for investigating the user's behavioral intention and provides a more appropriate explanation of related services for cross-generational interaction with SNS services.

ATIPS: Automatic Travel Itinerary Planning System for Domestic Areas.

Leisure travel has become a topic of great interest to Taiwanese residents in recent years. Most residents expect to be able to relax on a vacation during the holidays; however, the complicated procedure of travel itinerary planning is often discouraging and leads them to abandon the idea of traveling. In this paper, we design an automatic travel itinerary planning system for the domestic area (ATIPS) using an algorithm to automatically plan a domestic travel itinerary based on user intentions that allows users to minimize the process of trip planning. Simply by entering the travel time, the departure point, and the destination location, the system can automatically generate a travel itinerary. According to the results of the experiments, 70% of users were satisfied with the result of our system, and 82% of users were satisfied with the automatic user preference learning mechanism of ATIPS. Our algorithm also provides a framework for substituting modules or weights and offers a new method for travel planning.

IoT Big-Data Centred Knowledge Granule Analytic and Cluster Framework for BI Applications: A Case Base Analysis.

The current rapid growth of Internet of Things (IoT) in various commercial and non-commercial sectors has led to the deposition of large-scale IoT data, of which the time-critical analytic and clustering of knowledge granules represent highly thought-provoking application possibilities. The objective of the present work is to inspect the structural analysis and clustering of complex knowledge granules in an IoT big-data environment. In this work, we propose a knowledge granule analytic and clustering (KGAC) framework that explores and assembles knowledge granules from IoT big-data arrays for a business intelligence (BI) application. Our work implements neuro-fuzzy analytic architecture rather than a standard fuzzified approach to discover the complex knowledge granules. Furthermore, we implement an enhanced knowledge granule clustering (e-KGC) mechanism that is more elastic than previous techniques when assembling the tactical and explicit complex knowledge granules from IoT big-data arrays. The analysis and discussion presented here show that the proposed framework and mechanism can be implemented to extract knowledge granules from an IoT big-data array in such a way as to present knowledge of strategic value to executives and enable knowledge users to perform further BI actions.

Cognitive inference device for activity supervision in the elderly.

Human activity, life span, and quality of life are enhanced by innovations in science and technology. Aging individual needs to take advantage of these developments to lead a self-regulated life. However, maintaining a self-regulated life at old age involves a high degree of risk, and the elderly often fail at this goal. Thus, the objective of our study is to investigate the feasibility of implementing a cognitive inference device (CI-device) for effective activity supervision in the elderly. To frame the CI-device, we propose a device design framework along with an inference algorithm and implement the designs through an artificial neural model with different configurations, mapping the CI-device's functions to minimise the device's prediction error. An analysis and discussion are then provided to validate the feasibility of CI-device implementation for activity supervision in the elderly.

An e-caring chair for physiological signal measurement and recording.

There is an increasing awareness among the populace of the need for regular health check-up to detect diseases in their early stages and thereby administer treatments in a timely fashion. However, commercially available physiological signal monitoring devices, which may offer clues on the onset of diseases, are time-consuming, far from user friendly and limited in their applications. We design an e-caring chair that combines six modular physiological signal measurement instruments into a single unit, enabling users to simultaneously measure the blood pressure, body temperature, heart rate, height, weight and body fat percentage, and display the results and simple diagnoses in real time. The e-caring chair further allows for easy integration of additional physiological signal measuring devices, speedy measurements and long term monitoring of any trends that may emerge, making it easier for users to be alerted to physiological changes in the body without the need to enlist assistance from medical personnel. In this paper, we describe how this e-caring chair can be placed in several different environments for different purposes.

Technology acceptance for an Intelligent Comprehensive Interactive Care (ICIC) system for care of the elderly: a survey-questionnaire study.

The key components of caring for the elderly are diet, living, transportation, education, and safety issues, and telemedical systems can offer great assistance. Through the integration of personal to community information technology platforms, we have developed a new Intelligent Comprehensive Interactive Care (ICIC) system to provide comprehensive services for elderly care. The ICIC system consists of six items, including medical care (physiological measuring system, Medication Reminder, and Dr. Ubiquitous), diet, living, transportation, education (Intelligent Watch), entertainment (Sharetouch), and safety (Fall Detection). In this study, we specifically evaluated the users' intention of using the Medication Reminder, Dr. Ubiquitous, Sharetouch, and Intelligent Watch using a modified technological acceptance model (TAM). A total of 121 elderly subjects (48 males and 73 females) were recruited. The modified TAM questionnaires were collected after they had used these products. For most of the ICIC units, the elderly subjects revealed great willingness and/or satisfaction in using this system. The elderly users of the Intelligent Watch showed the greatest willingness and satisfaction, while the elderly users of Dr. Ubiquitous revealed fair willingness in the dimension of perceived ease of use. The old-old age group revealed greater satisfaction in the dimension of result demonstrability for the users of the Medication Reminder as compared to the young-old and oldest-old age groups. The women revealed greater satisfaction in the dimension of perceived ease of use for the users of Dr. Ubiquitous as compared to the men. There were no statistically significant differences in terms of gender, age, and education level in the other dimensions. The modified TAM showed its effectiveness in evaluating the acceptance and characteristics of technologic products for the elderly user. The ICIC system offers a user-friendly solution in telemedical care and improves the quality of care for the elderly.