Detecting implicit and explicit facial emotions at different ages.

Emotions are processed in the brain through a cortical route, responsible for detailed-conscious recognition and mainly based on image High Spatial Frequencies (HSF), and a subcortical route, responsible for coarse-unconscious processing and based on Low SF (LSF). However, little is known about possible changes in the functioning of the two routes in ageing. In the present go/no-go online task, 112 younger adults and 111 older adults were asked to press a button when a happy or angry face appeared (go) and to inhibit responses for neutral faces (no-go). Facial stimuli were presented unfiltered (broadband image), filtered at HSF and LSF, and hybrids (LSF of an emotional expression superimposed to the HSF of the same face with a neutral expression). All stimuli were also presented rotated on the vertical axis (upside-down) to investigate the global analysis of faces in ageing. Results showed an overall better performance of younger compared to older participants for all conditions except for hybrid stimuli. The expected face-inversion effect was confirmed in both age groups. We conclude that, besides an overall worsening of the perceptual skill with ageing, no specific impairment in the functioning of both the cortical and the subcortical route emerged.

Suicide attempt risk predicts inconsistent self-reported suicide attempts: A machine learning approach using longitudinal data.

INTRODUCTION: Inconsistent self-reports of lifetime suicide attempts (LSAs) are a major obstacle for accurate assessment of suicidal behavior. This study is the first to posit that adolescents at higher risk report LSAs more consistently than those at lower risk, revealing a link between suicide attempt risk and consistent reporting. METHODS: A machine learning model was trained with 70 % of the baseline assessment data of a longitudinal sample of Norwegian adolescents (n = 10,739). The model was used to estimate the LSA risk score for the remaining 30 % of the testing dataset. The relationship between these baseline risk scores and the consistency of reporting LSAs was assessed using a 2-year follow-up reassessment of the testing dataset. RESULTS: Internalizing problems, optimism about the future, conduct problems, substance use, and disordered eating were important factors associated with suicide attempt risk. Of the participants, 63.41 % had inconsistent self-reports at the two-year follow-up. Adolescents who consistently reported LSAs had significantly higher scores of suicide attempt risk at baseline. Two logistic regression analyses confirmed an association between suicide attempt risk and inconsistent self-reported LSAs and showed that sex (being male), and lower levels of depression and conduct problems significantly predicted such inconsistencies. Those who inconsistently reported LSAs were more likely than the others to be classified by the model as false negatives at the baseline risk assessment due to their lower estimated risk scores. LIMITATIONS: Suicide attempts were measured with a single item in this study. CONCLUSION: These risk factors support the theory of adolescent suicidality (TAS) and could improve suicide attempt risk assessment. Inconsistent self-reported LSAs signal lower suicide attempt risk.

Oscillatory attention in groove.

Attention is not constant but rather fluctuates over time and these attentional fluctuations may prioritize the processing of certain events over others. In music listening, the pleasurable urge to move to music (termed 'groove' by music psychologists) offers a particularly convenient case study of oscillatory attention because it engenders synchronous and oscillatory movements which also vary predictably with stimulus complexity. In this study, we simultaneously recorded pupillometry and scalp electroencephalography (EEG) from participants while they listened to drumbeats of varying complexity that they rated in terms of groove afterwards. Using the intertrial phase coherence of the beat frequency, we found that while subjects were listening, their pupil activity became entrained to the beat of the drumbeats and this entrained attention persisted in the EEG even as subjects imagined the drumbeats continuing through subsequent silent periods. This entrainment in both the pupillometry and EEG worsened with increasing rhythmic complexity, indicating poorer sensory precision as the beat became more obscured. Additionally, sustained pupil dilations revealed the expected, inverted U-shaped relationship between rhythmic complexity and groove ratings. Taken together, this work bridges oscillatory attention to rhythmic complexity in relation to musical groove.

From pre-processing to advanced dynamic modeling of pupil data.

The pupil of the eye provides a rich source of information for cognitive scientists, as it can index a variety of bodily states (e.g., arousal, fatigue) and cognitive processes (e.g., attention, decision-making). As pupillometry becomes a more accessible and popular methodology, researchers have proposed a variety of techniques for analyzing pupil data. Here, we focus on time series-based, signal-to-signal approaches that enable one to relate dynamic changes in pupil size over time with dynamic changes in a stimulus time series, continuous behavioral outcome measures, or other participants' pupil traces. We first introduce pupillometry, its neural underpinnings, and the relation between pupil measurements and other oculomotor behaviors (e.g., blinks, saccades), to stress the importance of understanding what is being measured and what can be inferred from changes in pupillary activity. Next, we discuss possible pre-processing steps, and the contexts in which they may be necessary. Finally, we turn to signal-to-signal analytic techniques, including regression-based approaches, dynamic time-warping, phase clustering, detrended fluctuation analysis, and recurrence quantification analysis. Assumptions of these techniques, and examples of the scientific questions each can address, are outlined, with references to key papers and software packages. Additionally, we provide a detailed code tutorial that steps through the key examples and figures in this paper. Ultimately, we contend that the insights gained from pupillometry are constrained by the analysis techniques used, and that signal-to-signal approaches offer a means to generate novel scientific insights by taking into account understudied spectro-temporal relationships between the pupil signal and other signals of interest.

Unveiling Adolescent Suicidality: Holistic Analysis of Protective and Risk Factors Using Multiple Machine Learning Algorithms.

Adolescent suicide attempts are on the rise, presenting a significant public health concern. Recent research aimed at improving risk assessment for adolescent suicide attempts has turned to machine learning. But no studies to date have examined the performance of stacked ensemble algorithms, which are more suitable for low-prevalence conditions. The existing machine learning-based research also lacks population-representative samples, overlooks protective factors and their interplay with risk factors, and neglects established theories on suicidal behavior in favor of purely algorithmic risk estimation. The present study overcomes these shortcomings by comparing the performance of a stacked ensemble algorithm with a diverse set of algorithms, performing a holistic item analysis to identify both risk and protective factors on a comprehensive data, and addressing the compatibility of these factors with two competing theories of suicide, namely, The Interpersonal Theory of Suicide and The Strain Theory of Suicide. A population-representative dataset of 173,664 Norwegian adolescents aged 13 to 18 years (mean = 15.14, SD = 1.58, 50.5% female) with a 4.65% rate of reported suicide attempt during the past 12 months was analyzed. Five machine learning algorithms were trained for suicide attempt risk assessment. The stacked ensemble model significantly outperformed other algorithms, achieving equal sensitivity and a specificity of 90.1%, AUC of 96.4%, and AUCPR of 67.5%. All algorithms found recent self-harm to be the most important indicator of adolescent suicide attempt. Exploratory factor analysis suggested five additional risk domains, which we labeled internalizing problems, sleep disturbance, disordered eating, lack of optimism regarding future education and career, and victimization. The identified factors provided stronger support for The Interpersonal Theory of Suicide than for The Strain Theory of Suicide. An enhancement to The Interpersonal Theory based on the risk and protective factors identified by holistic item analysis is presented.

Are false positives in suicide classification models a risk group? Evidence for "true alarms" in a population-representative longitudinal study of Norwegian adolescents.

Introduction: False positives in retrospective binary suicide attempt classification models are commonly attributed to sheer classification error. However, when machine learning suicide attempt classification models are trained with a multitude of psycho-socio-environmental factors and achieve high accuracy in suicide risk assessment, false positives may turn out to be at high risk of developing suicidal behavior or attempting suicide in the future. Thus, they may be better viewed as "true alarms," relevant for a suicide prevention program. In this study, using large population-based longitudinal dataset, we examine three hypotheses: (1) false positives, compared to the true negatives, are at higher risk of suicide attempt in future, (2) the suicide attempts risk for the false positives increase as a function of increase in specificity threshold; and (3) as specificity increases, the severity of risk factors between false positives and true positives becomes more similar. Methods: Utilizing the Gradient Boosting algorithm, we used a sample of 11,369 Norwegian adolescents, assessed at two timepoints (1992 and 1994), to classify suicide attempters at the first time point. We then assessed the relative risk of suicide attempt at the second time point for false positives in comparison to true negatives, and in relation to the level of specificity. Results: We found that false positives were at significantly higher risk of attempting suicide compared to true negatives. When selecting a higher classification risk threshold by gradually increasing the specificity cutoff from 60% to 97.5%, the relative suicide attempt risk of the false positive group increased, ranging from minimum of 2.96 to 7.22 times. As the risk threshold increased, the severity of various mental health indicators became significantly more comparable between false positives and true positives. Conclusion: We argue that the performance evaluation of machine learning suicide classification models should take the clinical relevance into account, rather than focusing solely on classification error metrics. As shown here, the so-called false positives represent a truly at-risk group that should be included in suicide prevention programs. Hence, these findings should be taken into consideration when interpreting machine learning suicide classification models as well as planning future suicide prevention interventions for adolescents.

Recognition of brief sounds in rapid serial auditory presentation.

Two experiments were conducted to test the role of participant factors (i.e., musical sophistication, working memory capacity) and stimulus factors (i.e., sound duration, timbre) on auditory recognition using a rapid serial auditory presentation paradigm. Participants listened to a rapid stream of very brief sounds ranging from 30 to 150 milliseconds and were tested on their ability to distinguish the presence from the absence of a target sound selected from various sound sources placed amongst the distracters. Experiment 1a established that brief exposure to stimuli (60 to 150 milliseconds) does not necessarily correspond to impaired recognition. In Experiment 1b we found evidence that 30 milliseconds of exposure to the stimuli significantly impairs recognition of single auditory targets, but the recognition for voice and sine tone targets impaired the least, suggesting that the lower limit required for successful recognition could be lower than 30 milliseconds for voice and sine tone targets. Critically, the effect of sound duration on recognition completely disappeared when differences in musical sophistication were controlled for. Participants' working memory capacities did not seem to predict their recognition performances. Our behavioral results extend the studies oriented to understand the processing of brief timbres under temporal constraint by suggesting that the musical sophistication may play a larger role than previously thought. These results can also provide a working hypothesis for future research, namely, that underlying neural mechanisms for the processing of various sound sources may have different temporal constraints.

Using Automated Speech Processing for Repeated Measurements in a Clinical Setting of the Behavioral Variability in the Stroop Task.

The Stroop interference task is indispensable to current neuropsychological practice. Despite this, it is limited in its potential for repeated administration, its sensitivity and its demands on professionals and their clients. We evaluated a digital Stroop deployed using a smart device. Spoken responses were timed using automated speech recognition. Participants included adult nonpatients (N = 113; k = 5 sessions over 5 days) and patients with psychiatric diagnoses (N = 85; k = 3-4 sessions per week over 4 weeks). Traditional interference (difference in response time between color incongruent words vs. color neutral words; M = 0.121 s) and facilitation (neutral vs. color congruent words; M = 0.085 s) effects were robust and temporally stable over testing sessions (ICCs 0.50-0.86). The performance showed little relation to clinical symptoms for a two-week window for either nonpatients or patients but was related to self-reported concentration at the time of testing for both groups. Performance was also related to treatment outcomes in patients. The duration of response word utterances was longer in patients than in nonpatients. Measures of intra-individual variability showed promise for understanding clinical state and treatment outcome but were less temporally stable than measures based solely on average response time latency. This framework of remote assessment using speech processing technology enables the fine-grained longitudinal charting of cognition and verbal behavior. However, at present, there is a problematic lower limit to the absolute size of the effects that can be examined when using voice in such a brief 'out-of-the-laboratory condition' given the temporal resolution of the speech-to-text detection system (in this case, 10 ms). This resolution will limit the parsing of meaningful effect sizes.

Caressed by music: Related preferences for velocity of touch and tempo of music?

Given that both hearing and touch are 'mechanical senses' that respond to physical pressure or mechanical energy and that individuals appear to have a characteristic internal or spontaneous tempo, individual preferences in musical and touch rhythms might be related. We explored this in two experiments probing individual preferences for tempo in the tactile and auditory modalities. Study 1 collected ratings of received stroking on the forearm and measured the velocity the participants used for stroking a fur. Music tempo preferences were assessed as mean beats per minute of individually selected music pieces and via the adjustment of experimenter-selected music to a preferred tempo. Heart rate was recorded to measure levels of physiological arousal. We found that the preferred tempo of favorite (self-selected) music correlated positively with the velocity with which each individual liked to be touched. In Study 2, participants rated videos of repeated touch on someone else's arm and videos of a drummer playing with brushes on a snare drum, both at a variety of tempos. We found that participants with similar rating patterns for the different stroking speeds did not show similar rating patterns for the different music beats. The results suggest that there may be a correspondence between preferences for favorite music and felt touch, but this is either weak or it cannot be evoked effectively with vicarious touch and/or mere drum beats. Thus, if preferences for touch and music are related, this is likely to be dependent on the specific type of stimulation.

The lateralized flash-lag illusion: A psychophysical and pupillometry study.

The flash-lag illusion (FLI) is a visual phenomenon where a flashed object, either co-localized or in physical alignment with another continuously moving object, is perceived to lag behind the path of the moving object. In the present study, we reveal an anisotropy of the FLI between the lateral visual fields that was expressed psychophysically as different points of subjective equality, depending on the hemifield in which the stimuli appeared. Specifically, the study confirmed that, as seen in two previous studies, the FLI was significantly larger in the left visual field (LVF) than in the right (RVF). In addition, pupil dilations were larger in the RVF than in the LVF as well as returning to baseline levels more rapidly in the LVF. We interpret these findings as converging on revealing more efficient spatial and attentional processing and, in turn, extrapolation of motion in the LVF/right hemisphere than in the RVF/left hemisphere.

Beat alignment ability is associated with formal musical training not current music playing.

The ability to perceive the beat in music is crucial for both music listeners and players with expert musicians being notably skilled at noticing fine deviations in the beat. However, it is unclear whether this beat perception ability is enhanced in trained musicians who continue to practice relative to musicians who no longer play. Thus, we investigated this by comparing active musicians', inactive musicians', and nonmusicians' beat alignment ability scores on the Computerized Adaptive Beat Alignment Test (CA-BAT). 97 adults with diverse musical experience participated in the study, reporting their years of formal musical training, number of instruments played, hours of weekly music playing, and hours of weekly music listening, in addition to their demographic information. While initial tests between groups indicated active musicians outperformed inactive musicians and nonmusicians on the CA-BAT, a generalized linear regression analysis showed that there was no significant difference once differences in musical training had been accounted for. To ensure that our results were not impacted by multicollinearity between music-related variables, nonparametric and nonlinear machine learning regressions were employed and confirmed that years of formal musical training was the only significant predictor of beat alignment ability. These results suggest that expertly perceiving fine differences in the beat is not a use-dependent ability that degrades without regular maintenance through practice or musical engagement. Instead, better beat alignment appears to be associated with more musical training regardless of continued use.

Human voices escape the auditory attentional blink: Evidence from detections and pupil responses.

Attentional selection of a second target in a rapid stream of stimuli embedding two targets tends to be briefly impaired when two targets are presented in close temporal proximity, an effect known as an attentional blink (AB). Two target sounds (T1 and T2) were embedded in a rapid serial auditory presentation of environmental sounds with a short (Lag 3) or long lag (Lag 9). Participants were to first identify T1 (bell or sine tone) and then to detect T2 (present or absent). Individual stimuli had durations of either 30 or 90 ms, and were presented in streams of 20 sounds. The T2 varied in category: human voice, cello, or dog sound. Previous research has introduced pupillometry as a useful marker of the intensity of cognitive processing and attentional allocation in the visual AB paradigm. Results suggest that the interplay of stimulus factors is critical for target detection accuracy and provides support for the hypothesis that the human voice is the least likely to show an auditory AB (in the 90 ms condition). For the other stimuli, accuracy for T2 was significantly worse at Lag 3 than at Lag 9 in the 90 ms condition, suggesting the presence of an auditory AB. When AB occurred (at Lag 3), we observed smaller pupil dilations, time-locked to the onset of T2, compared to Lag 9, reflecting lower attentional processing when 'blinking' during target detection. Taken together, these findings support the conclusion that human voices escape the AB and that the pupillary changes are consistent with the so-called T2 attentional deficit. In addition, we found some indication that salient stimuli like human voices could require a less intense allocation of attention, or noradrenergic potentiation, compared to other auditory stimuli.

Pupil drift rate indexes groove ratings.

Groove, understood as an enjoyable compulsion to move to musical rhythms, typically varies along an inverted U-curve with increasing rhythmic complexity (e.g., syncopation, pickups). Predictive coding accounts posit that moderate complexity drives us to move to reduce sensory prediction errors and model the temporal structure. While musicologists generally distinguish the effects of pickups (anacruses) and syncopations, their difference remains unexplored in groove. We used pupillometry as an index to noradrenergic arousal while subjects listened to and rated drumbeats varying in rhythmic complexity. We replicated the inverted U-shaped relationship between rhythmic complexity and groove and showed this is modulated by musical ability, based on a psychoacoustic beat perception test. The pupil drift rates suggest that groovier rhythms hold attention longer than ones rated less groovy. Moreover, we found complementary effects of syncopations and pickups on groove ratings and pupil size, respectively, discovering a distinct predictive process related to pickups. We suggest that the brain deploys attention to pickups to sharpen subsequent strong beats, augmenting the predictive scaffolding's focus on beats that reduce syncopations' prediction errors. This interpretation is in accordance with groove envisioned as an embodied resolution of precision-weighted prediction error.

The Eye Pupil Adjusts to Illusorily Expanding Holes.

Some static patterns evoke the perception of an illusory expanding central region or "hole." We asked observers to rate the magnitudes of illusory motion or expansion of black holes, and these predicted the degree of dilation of the pupil, measured with an eye tracker. In contrast, when the "holes" were colored (including white), i.e., emitted light, these patterns constricted the pupils, but the subjective expansions were also weaker compared with the black holes. The change rates of pupil diameters were significantly related to the illusory motion phenomenology only with the black holes. These findings can be accounted for within a perceiving-the-present account of visual illusions, where both the illusory motion and the pupillary adjustments represent compensatory mechanisms to the perception of the next moment, based on shared experiences with the ecological regularities of light.

Environmental risks to humans, the first database of valence and arousal ratings for images of natural hazards.

Due to their relevance for the entire society, environmental hazards have largely been investigated in terms of their psychological effects. However, a complete image database comprising different categories of catastrophes has not been proposed yet. We selected 200 photographs of the most frequent natural disasters with the aim to collect the emotional reactions of observers. In particular, 20 stimuli were selected for each of the following 10 categories: earthquake, volcanic activity, lightning, hailstorm, drought, fire, landslide, epidemic, and neutral and positive images as control categories. A sample of 605 participants completed an online survey, in which they were asked to rate either the valence or the arousal of each stimulus, by using a Self-Assessment Manikin. The Environmental Risk to Humans database associates the emotional reactions to these visual stimuli, together with the demographics of the sample (e.g., gender, age, marital status, income, previous experience of natural disasters). The database constitutes a tool to explore human reactions to natural hazards, providing a controlled set of stimuli for different types of catastrophes.

Substituting facial movements in singers changes the sounds of musical intervals.

Cross-modal integration is ubiquitous within perception and, in humans, the McGurk effect demonstrates that seeing a person articulating speech can change what we hear into a new auditory percept. It remains unclear whether cross-modal integration of sight and sound generalizes to other visible vocal articulations like those made by singers. We surmise that perceptual integrative effects should involve music deeply, since there is ample indeterminacy and variability in its auditory signals. We show that switching videos of sung musical intervals changes systematically the estimated distance between two notes of a musical interval so that pairing the video of a smaller sung interval to a relatively larger auditory led to compression effects on rated intervals, whereas the reverse led to a stretching effect. In addition, after seeing a visually switched video of an equally-tempered sung interval and then hearing the same interval played on the piano, the two intervals were judged often different though they differed only in instrument. These findings reveal spontaneous, cross-modal, integration of vocal sounds and clearly indicate that strong integration of sound and sight can occur beyond the articulations of natural speech.

Musical and Bodily Predictors of Mental Effort in String Quartet Music: An Ecological Pupillometry Study of Performers and Listeners.

Music performance can be cognitively and physically demanding. These demands vary across the course of a performance as the content of the music changes. More demanding passages require performers to focus their attention more intensity, or expend greater "mental effort." To date, it remains unclear what effect different cognitive-motor demands have on performers' mental effort. It is likewise unclear how fluctuations in mental effort compare between performers and perceivers of the same music. We used pupillometry to examine the effects of different cognitive-motor demands on the mental effort used by performers and perceivers of classical string quartet music. We collected pupillometry, motion capture, and audio-video recordings of a string quartet as they performed a rehearsal and concert (for live audience) in our lab. We then collected pupillometry data from a remote sample of musically-trained listeners, who heard the audio recordings (without video) that we captured during the concert. We used a modelling approach to assess the effects of performers' bodily effort (head and arm motion; sound level; performers' ratings of technical difficulty), musical complexity (performers' ratings of harmonic complexity; a score-based measure of harmonic tension), and expressive difficulty (performers' ratings of expressive difficulty) on performers' and listeners' pupil diameters. Our results show stimulating effects of bodily effort and expressive difficulty on performers' pupil diameters, and stimulating effects of expressive difficulty on listeners' pupil diameters. We also observed negative effects of musical complexity on both performers and listeners, and negative effects of performers' bodily effort on listeners, which we suggest may reflect the complex relationships that these features share with other aspects of musical structure. Looking across the concert, we found that both of the quartet violinists (who exchanged places halfway through the concert) showed more dilated pupils during their turns as 1st violinist than when playing as 2nd violinist, suggesting that they experienced greater arousal when "leading" the quartet in the 1st violin role. This study shows how eye tracking and motion capture technologies can be used in combination in an ecological setting to investigate cognitive processing in music performance.

'Defrosting' music chills with naltrexone: The role of endogenous opioids for the intensity of musical pleasure.

The endogenous opioid system has been implicated during experiences of pleasure (i.e., from food or sex). Music can elicit intense emotional and bodily sensations of pleasure, called 'Chills'. We investigated the effects of an opioid antagonist (50 mg naltrexone) or placebo (40 µg d3-vitamin) while listening to self-selected music or other 'control' music selected by another participant. We used a novel technique of continuous measurement of pleasantness with an eye tracker system, where participants shifted their eyes along a visual analogue scale, in the semblance of a thermometer so that, as the music unfolded, gaze positions indicated the self-reported hedonic experience. Simultaneously, we obtained pupil diameters. Self-reported pleasure remained unchanged by naltrexone, which - however - selectively decreased pupillary diameters during 'Chills'. Hence, the endogenous µ-opioid signaling is not necessary for subjective enjoyment of music but an opioid blockade dampens pupil responses to peak pleasure, consistent with decreased arousal to the music.

"Mickey Mousing" in the Brain: Motion-Sound Synesthesia and the Subcortical Substrate of Audio-Visual Integration.

Motion-sound synesthesia is characterized by illusory auditory sensations linked to the pattern and rhythms of motion (dubbed "Mickey Mousing" as in cinema) of visually experienced but soundless object, like an optical flow array, a ball bouncing or a horse galloping. In an MRI study with a group of three synesthetes and a group of eighteen control participants, we found structural changes in the brains of synesthetes in the subcortical multisensory areas of the superior and inferior colliculi. In addition, functional magnetic resonance imaging data showed activity in motion-sensitive regions, as well as temporal and occipital areas, and the cerebellum. However, the synesthetes had a higher activation within the left and right cuneus, with stronger activations when viewing optical flow stimuli. There was also a general difference in connectivity of the colliculi with the above mentioned regions between the two groups. These findings implicate low-level mechanisms within the human neuroaxis as a substrate for local connectivity and cross activity between perceptual processes that are "distant" in terms of cortical topography. The present findings underline the importance of considering the role of subcortical systems and their connectivity to multimodal regions of the cortex and they strengthen a parsimonious account of synesthesia, at the least of the visual-auditory type.

More mental rotation time does not imply more mental effort: Pupillary diameters do not change with angular distance.

The ability to mentally rotate objects in space is a fundamental cognitive capacity. Previous studies showed that the time to rotate the image of a figure to match another increases progressively with angular disparity. It remains unclear whether this increase in response time with angular disparity could reflect increased processing operations or more cognitive effort instead of a sustained use of a 'rotate' mechanism without a change in workload. We collected response times as well as pupillary responses that index cognitive workload and activity in the brainstem's locus coeruleus, from a sample of 38 young adults performing a chronometric mental rotations task. The results showed the expected increase in response times but no increase in pupil diameters between 60, 120, and 180 degrees of rotation, suggesting no significant changes in arousal levels when rotating figures near and far. This indicates that during mental rotation the load on cognitive resources remains constant irrespective of angular distance.