Human vs machine towards neonatal pain assessment: A comprehensive analysis of the facial features extracted by health professionals, parents, and convolutional neural networks.

Neonates are not able to verbally communicate pain, hindering the correct identification of this phenomenon. Several clinical scales have been proposed to assess pain, mainly using the facial features of the neonate, but a better comprehension of these features is yet required, since several related works have shown the subjectivity of these scales. Meanwhile, computational methods have been implemented to automate neonatal pain assessment and, although performing accurately, these methods still lack the interpretability of the corresponding decision-making processes. To address this issue, we propose in this work a facial feature extraction framework to gather information and investigate the human and machine neonatal pain assessments, comparing the visual attention of the facial features perceived by health-professionals and parents of neonates with the most relevant ones extracted by eXplainable Artificial Intelligence (XAI) methods, considering the VGG-Face and N-CNN deep learning architectures. Our experimental results show that the information extracted by the computational methods are clinically relevant to neonatal pain assessment, but yet do not agree with the facial visual attention of health-professionals and parents, suggesting that humans and machines can learn from each other to improve their decision-making processes. We believe that these findings might advance our understanding of how humans and machines code and decode neonatal facial responses to pain, enabling further improvements in clinical scales widely used in practical situations and in face-based automatic pain assessment tools as well.

Pediatricians' focus of sight at pain assessment during a neonatal heel puncture / Foco do olhar do pediatra na avaliação da dor neonatal durante uma punção de calcanhar

ABSTRACT Objective: To evaluate the focus of pediatricians' gaze during the heel prick of neonates. Methods: Prospective study in which pediatricians wearing eye tracker glasses evaluated neonatal pain before/after a heel prtick. Pediatricians scored the pain they perceived in the neonate in a verbal analogue numerical scale (0=no pain; 10=maximum pain). The outcomes measured were number and time of visual fixations in upper face, lower face, and hands, in two 10-second periods, before (pre) and after the puncture (post). These outcomes were compared between the periods, and according to pediatricians' pain perception: absent/mild (score: 0-5) and moderate/intense (score: 6-10). Results: 24 pediatricians (31 years old, 92% female) evaluated 24 neonates. The median score attributed to neonatal pain during the heel prick was 7.0 (Interquartile range: 5-8). Compared to pre-, in the post-periods, more pediatricians fixed their gaze on the lower face (63 vs. 92%; p=0.036) and the number of visual fixations was greater on the lower face (2.0 vs. 5.0; p=0.018). There was no difference in the number and time of visual fixations according to the intensity of pain. Conclusions: At bedside, pediatricians change their focus of attention on the neonatal face after a painful procedure, focusing mainly on the lower part of the face.

RESUMO Objetivo: Avaliar o foco do olhar do pediatra durante a punção do calcanhar de neonatos. Métodos: Estudo prospectivo no qual pediatras, utilizando óculos de rastreamento visual, avaliaram a dor neonatal antes/depois de uma punção de calcanhar. Os pediatras pontuaram a dor de acordo com a sua percepção por meio de uma escala analógica verbal (0=sem dor; 10=dor máxima). Os desfechos analisados foram o número e o tempo das fixações visuais na face superior, face inferior e mãos, em dois períodos de 10 segundos, antes (PRÉ) e depois da punção (PÓS). Os resultados foram comparados entre os períodos e segundo a percepção da dor do pediatra: ausente/leve (escore: 0-5) e moderada/grave (escore: 6-10). Resultados: Vinte e quatro pediatras (31 anos, 92% sexo feminino) avaliaram 24 neonatos. A mediana do escore atribuído à dor do recém-nascido durante a punção do calcanhar foi 7,0 (intervalo interquartil: 5-8). Comparado ao período PRÉ, no período PÓS, o maior número de pediatras fixou o olhar na face inferior (63 vs. 92%; p=0,036) e o número de fixações visuais foi maior na face inferior (2,0 vs. 5,0; p=0,018). Não houve diferença no número e no tempo das fixações visuais de acordo com a intensidade da dor. Conclusões: À beira do leito, os pediatras mudam seu foco de atenção visual na face do recém-nascido após um procedimento doloroso, focando o olhar principalmente na parte inferior da face.

Pediatricians' focus of sight at pain assessment during a neonatal heel puncture.

OBJECTIVE: To evaluate the focus of pediatricians' gaze during the heel prick of neonates. METHODS: Prospective study in which pediatricians wearing eye tracker glasses evaluated neonatal pain before/after a heel prtick. Pediatricians scored the pain they perceived in the neonate in a verbal analogue numerical scale (0=no pain; 10=maximum pain). The outcomes measured were number and time of visual fixations in upper face, lower face, and hands, in two 10-second periods, before (pre) and after the puncture (post). These outcomes were compared between the periods, and according to pediatricians' pain perception: absent/mild (score: 0-5) and moderate/intense (score: 6-10). RESULTS: 24 pediatricians (31 years old, 92% female) evaluated 24 neonates. The median score attributed to neonatal pain during the heel prick was 7.0 (Interquartile range: 5-8). Compared to pre-, in the post-periods, more pediatricians fixed their gaze on the lower face (63 vs. 92%; p=0.036) and the number of visual fixations was greater on the lower face (2.0 vs. 5.0; p=0.018). There was no difference in the number and time of visual fixations according to the intensity of pain. CONCLUSIONS: At bedside, pediatricians change their focus of attention on the neonatal face after a painful procedure, focusing mainly on the lower part of the face.

What Facial Features Does the Pediatrician Look to Decide That a Newborn Is Feeling Pain?

OBJECTIVE: The study aimed to analyze the gaze fixation of pediatricians during the decision process regarding the presence/absence of pain in pictures of newborn infants. STUDY DESIGN: Experimental study, involving 38 pediatricians (92% females, 34.6 ± 9.0 years, 22 neonatologists) who evaluated 20 pictures (two pictures of each newborn: one at rest and one during a painful procedure), presented in random order for each participant. The Tobii-TX300 equipment tracked eye movements in four areas of interest of each picture (AOI): mouth, eyes, forehead, and nasolabial furrow. Pediatricians evaluated the intensity of pain with a verbal analogue score from 0 to 10 (0 = no pain; 10 = maximum pain). The number of pictures in which pediatricians fixed their gaze, the number of gaze fixations, and the total and average time of gaze fixations were compared among the AOI by analysis of variance (ANOVA). The visual-tracking parameters of the pictures' evaluations were also compared by ANOVA according to the pediatricians' perception of pain presence: moderate/severe (score = 6-10), mild (score = 3-5), and absent (score = 0-2). The association between the total time of gaze fixations in the AOI and pain perception was assessed by logistic regression. RESULTS: In the 20 newborn pictures, the mean number of gaze fixations was greater in the mouth, eyes, and forehead than in the nasolabial furrow. Also, the average total time of gaze fixations was greater in the mouth and forehead than in the nasolabial furrow. Controlling for the time of gaze fixation in the AOI, each additional second in the time of gaze fixation in the mouth (odds ratio [OR]: 1.26; 95% confidence interval [CI]: 1.08-1.46) and forehead (OR: 1.16; 95% CI: 1.02-1.33) was associated with an increase in the chance of moderate/severe pain presence in the neonatal facial picture. CONCLUSION: When challenged to say whether pain is present in pictures of newborn infants' faces, pediatricians fix their gaze preferably in the mouth. The longer duration of gaze fixation in the mouth and forehead is associated with an increase perception that moderate/severe pain is present. KEY POINTS: · Neonatal pain assessment is intrinsically subjective.. · Visual tracking identifies the focus of attention of individuals.. · Adults' gaze in neonates' mouth and forehead is associated with pain perception..

Looking at neonatal facial features of pain: do health and non-health professionals differ?

Abstract Objective: To analyze the regions that trigger the attention of adults' gaze when assessing pain in newborn infants' pictures and to verify if there are differences between health and non-health professionals. Method: Experimental study with 84 health professionals and 59 non-health professionals, who evaluated two images of 10 neonates, one at rest and the other during a painful procedure. Each image was shown for 7 seconds on a computer screen, while eye movements were tracked by the Tobii TX300 EyeTracker. After evaluating each image, participants gave a score from 0 (absent pain) to 10 (maximum pain), according to their perception of neonatal pain. For each image, the number and total time of gaze fixations in the forehead, eyes, nasolabial furrow, and mouth were studied. Comparisons between both groups of adults were made by an intraclass correlation coefficient, Student's t-test, and Bland Altman graphic. Results: Health professionals (93% female; 34 ± 9 years old), compared to non-health professionals (64% female; 35 ± 11 years old), gave lower scores for images at rest (0.81 ± 0.50 vs. 1.59 ± 0.76; p = 0.010), with no difference for those obtained during the painful procedure (6.98 ± 1.08 vs. 6.73 ± 0.82). There was a strong or almost perfect correlation for the number of fixations in the mouth, eyes, forehead, and for the total fixation time in the eyes and forehead. Conclusions: Adults, irrespective of their profession, showed a homogeneous gaze pattern when evaluating pictures of neonates at rest or during a painful procedures.

Looking at neonatal facial features of pain: do health and non-health professionals differ?

OBJECTIVE: To analyze the regions that trigger the attention of adults' gaze when assessing pain in newborn infants' pictures and to verify if there are differences between health and non-health professionals. METHOD: Experimental study with 84 health professionals and 59 non-health professionals, who evaluated two images of 10 neonates, one at rest and the other during a painful procedure. Each image was shown for 7 seconds on a computer screen, while eye movements were tracked by the Tobii TX300 EyeTracker. After evaluating each image, participants gave a score from 0 (absent pain) to 10 (maximum pain), according to their perception of neonatal pain. For each image, the number and total time of gaze fixations in the forehead, eyes, nasolabial furrow, and mouth were studied. Comparisons between both groups of adults were made by an intraclass correlation coefficient, Student's t-test, and Bland Altman graphic. RESULTS: Health professionals (93% female; 34 ± 9 years old), compared to non-health professionals (64% female; 35 ± 11 years old), gave lower scores for images at rest (0.81 ± 0.50 vs. 1.59 ± 0.76; p = 0.010), with no difference for those obtained during the painful procedure (6.98 ± 1.08 vs. 6.73 ± 0.82). There was a strong or almost perfect correlation for the number of fixations in the mouth, eyes, forehead, and for the total fixation time in the eyes and forehead. CONCLUSIONS: Adults, irrespective of their profession, showed a homogeneous gaze pattern when evaluating pictures of neonates at rest or during a painful procedures.

Identification of pain in neonates: the adults' visual perception of neonatal facial features.

OBJECTIVE: To verify the visual attention of adults when assessing neonatal pain. STUDY DESIGN: 143 adults (59% health professionals) evaluated 20 pictures (2 pictures of 10 neonates' faces: at rest; during a painful procedure). Tobii-TX300 tracked the participants' eyes movement. For each picture, adults scored pain intensity (0 = no pain; 10 = maximum). Latent classes analysis was applied by cognitive diagnosis models-GDINA with two attributes (knowledge of pain presence/absence). Variables associated with belonging to the class of adults that correctly identified pictures of newborns with/without pain were identified by logistic regression. RESULTS: To identify neonatal pain, adults look at the mouth, eyes, and forehead in facial pictures. The latent class analysis identified four classes of adults: those that identify painful/painless neonates (YY-Class; n = 80); only painful neonates (n = 28); only painless neonates (n = 34) and none (n = 1). Being a health professional (OR: 2.29; 95% CI: 1.16-4.51), and each look at the nasolabial furrow (2.07; 1.19-3.62) increased the chance of belonging to the YY-class. CONCLUSIONS: Being a health professional and the visual fixation at the nasolabial furrow helped to identify the presence/absence of neonatal pain.

Maximum-uncertainty linear discrimination analysis of first-episode schizophrenia subjects.

Recent techniques of image analysis brought the possibility to recognize subjects based on discriminative image features. We performed a magnetic resonance imaging (MRI)-based classification study to assess its usefulness for outcome prediction of first-episode schizophrenia patients (FES). We included 39 FES patients and 39 healthy controls (HC) and performed the maximum-uncertainty linear discrimination analysis (MLDA) of MRI brain intensity images. The classification accuracy index (CA) was correlated with the Positive and Negative Syndrome Scale (PANSS) and the Global Assessment of Functioning scale (GAF) at 1-year follow-up. The rate of correct classifications of patients with poor and good outcomes was analyzed using chi-square tests. MLDA classification was significantly better than classification by chance. Leave-one-out accuracy was 72%. CA correlated significantly with PANSS and GAF scores at the 1-year follow-up. Moreover, significantly more patients with poor outcome than those with good outcome were classified correctly. MLDA of brain MR intensity features is, therefore, able to correctly classify a significant number of FES patients, and the discriminative features are clinically relevant for clinical presentation 1 year after the first episode of schizophrenia. The accuracy of the current approach is, however, insufficient to be used in clinical practice immediately. Several methodological issues need to be addressed to increase the usefulness of this classification approach.

Evaluating SVM and MLDA in the extraction of discriminant regions for mental state prediction.

Pattern recognition methods have been successfully applied in several functional neuroimaging studies. These methods can be used to infer cognitive states, so-called brain decoding. Using such approaches, it is possible to predict the mental state of a subject or a stimulus class by analyzing the spatial distribution of neural responses. In addition it is possible to identify the regions of the brain containing the information that underlies the classification. The Support Vector Machine (SVM) is one of the most popular methods used to carry out this type of analysis. The aim of the current study is the evaluation of SVM and Maximum uncertainty Linear Discrimination Analysis (MLDA) in extracting the voxels containing discriminative information for the prediction of mental states. The comparison has been carried out using fMRI data from 41 healthy control subjects who participated in two experiments, one involving visual-auditory stimulation and the other based on bi-manual fingertapping sequences. The results suggest that MLDA uses significantly more voxels containing discriminative information (related to different experimental conditions) to classify the data. On the other hand, SVM is more parsimonious and uses less voxels to achieve similar classification accuracies. In conclusion, MLDA is mostly focused on extracting all discriminative information available, while SVM extracts the information which is sufficient for classification.

Multivariate gene expression analysis reveals functional connectivity changes between normal/tumoral prostates.

BACKGROUND: Prostate cancer is a leading cause of death in the male population, therefore, a comprehensive study about the genes and the molecular networks involved in the tumoral prostate process becomes necessary. In order to understand the biological process behind potential biomarkers, we have analyzed a set of 57 cDNA microarrays containing approximately 25,000 genes. RESULTS: Principal Component Analysis (PCA) combined with the Maximum-entropy Linear Discriminant Analysis (MLDA) were applied in order to identify genes with the most discriminative information between normal and tumoral prostatic tissues. Data analysis was carried out using three different approaches, namely: (i) differences in gene expression levels between normal and tumoral conditions from an univariate point of view; (ii) in a multivariate fashion using MLDA; and (iii) with a dependence network approach. Our results show that malignant transformation in the prostatic tissue is more related to functional connectivity changes in their dependence networks than to differential gene expression. The MYLK, KLK2, KLK3, HAN11, LTF, CSRP1 and TGM4 genes presented significant changes in their functional connectivity between normal and tumoral conditions and were also classified as the top seven most informative genes for the prostate cancer genesis process by our discriminant analysis. Moreover, among the identified genes we found classically known biomarkers and genes which are closely related to tumoral prostate, such as KLK3 and KLK2 and several other potential ones. CONCLUSION: We have demonstrated that changes in functional connectivity may be implicit in the biological process which renders some genes more informative to discriminate between normal and tumoral conditions. Using the proposed method, namely, MLDA, in order to analyze the multivariate characteristic of genes, it was possible to capture the changes in dependence networks which are related to cell transformation.

Hyperplane navigation: a method to set individual scores in fMRI group datasets.

Recent studies have demonstrated that spatial patterns of fMRI BOLD activity distribution over the brain may be used to classify different groups or mental states. These studies are based on the application of advanced pattern recognition approaches and multivariate statistical classifiers. Most published articles in this field are focused on improving the accuracy rates and many approaches have been proposed to accomplish this task. Nevertheless, a point inherent to most machine learning methods (and still relatively unexplored in neuroimaging) is how the discriminative information can be used to characterize groups and their differences. In this work, we introduce the Maximum Uncertainty Linear Discrimination Analysis (MLDA) and show how it can be applied to infer groups' patterns by discriminant hyperplane navigation. In addition, we show that it naturally defines a behavioral score, i.e., an index quantifying the distance between the states of a subject from predefined groups. We validate and illustrate this approach using a motor block design fMRI experiment data with 35 subjects.