Cross-Modal Interaction and Integration Through Stimulus-Specific Adaptation in the Thalamic Reticular Nucleus of Rats / 神经科学通报·英文版

Stimulus-specific adaptation (SSA), defined as a decrease in responses to a common stimulus that only partially generalizes to other rare stimuli, is a widespread phenomenon in the brain that is believed to be related to novelty detection. Although cross-modal sensory processing is also a widespread phenomenon, the interaction between the two phenomena is not well understood. In this study, the thalamic reticular nucleus (TRN), which is regarded as a hub of the attentional system that contains multi-modal neurons, was investigated. The results showed that SSA existed in an interactive oddball stimulation, which mimics stimulation changes from one modality to another. In the bimodal integration, SSA to bimodal stimulation was stronger than to visual stimulation alone but similar to auditory stimulation alone, which indicated a limited integrative effect. Collectively, the present results provide evidence for independent cross-modal processing in bimodal TRN neurons.

Ocular fixations modulate audiovisual semantic congruency when standing in an upright position / Las fijaciones oculares modulan la congruencia semántica audiovisual cuando el cuerpo está en una posición erguida

Abstract Introduction: Multisensory audiovisual semantic congruency is the process by which visual information is perceived as integrated to auditory stimuli, because both coincide in terms of simultaneity and semantic correspondence. This study was aimed at establishing whether visual percepts, which semantically correspond to auditory stimuli, are associated with ocular fixations in modulating bottom-up areas while keeping a body posture alignment between the up-direction and the idiotropic axes, as well as in another orientation corresponding to a vectorial opposition between the up-direction and the head idiotropic axis. Method: Two groups (one for each position) were selected from a sample of 88 people. A bistable image was presented on a screen of a fixed 120 Hz eye-tracker device, providing background auditory stimuli so as to establish semantic congruencies and their relations to ocular fixations. Results: It was found that audiovisual semantic congruency is associated with fixations when idiotropic vectors are aligned with the up direction. Fixations manifested in bottom-up modulating areas are not associated with multisensory audiovisual semantic congruency when the head idiotropic vector is parallel with the gravity vector. Eye fixations decrease significantly if the head idiotropic axis is aligned with the gravity vector. Conclusion: It is concluded that body position can affect visual perceptual processes involved in the occurrence of semantic congruency.

Resumen Introducción: La congruencia semántica audiovisual es el proceso por el cual información de una modalidad sensorial visual se percibe como integrada a estímulos auditivos, porque coinciden en términos de simultaneidad y correspondencia semántica. Este estudio tuvo por objeto establecer si los perceptos visuales que se corresponden semánticamente a estímulos auditivos están asociados con las fijaciones oculares realizadas en áreas de modulación bottom-up, tanto en una postura corporal definida por la alineación entre la dirección vertical hacia arriba y los ejes vectoriales idiotrópicos, como en otra orientación definida por una oposición vectorial entre la vertical hacia arriba y el eje idiotrópico de la cabeza. Método: Dos grupos fueron seleccionados (uno por cada posición), tomados de una muestra de 88 personas. Los datos se obtuvieron por medio de un dispositivo fijo de registro de movimientos oculares de 120 Hz. Una imagen biestable se presentó, proporcionando estímulos auditivos de fondo para producir congruencias semánticas y establecer su relación con las fijaciones oculares. Resultados: Se encontró que la congruencia semántica audiovisual está asociada con áreas de fijación ocular cuando los vectores idiotrópicos están alineados con la dirección vertical ascendente. Las fijaciones oculares en áreas de modulación bottom-up no están asociadas con la congruencia semántica audiovisual cuando el vector idiotrópico de la cabeza está alineado con el vector gravitacional. En esta última posición, la cantidad de fijaciones oculares es significativamente menor. Conclusión: La posición del cuerpo puede afectar procesos perceptuales visuales que, a su vez, inciden en el efecto de congruencia semántica.

Research progress of MR imaging for prediction of CT imaging / 中国辐射卫生

Medical images can provide clinicans with accurate and comprehensive patients’ information. Morphological or functional abnormalities caused by various diseases can be manifested in many aspects. Although MR images and CT images can highlight the medical image data of different tissue structures of patients, single MR images or CT images cannot fully reflect the complexity of diseases. Using MR image to predict CT image is one of the cross-modal prediction of medical images. In this paper, the methods of MR image prediction for CTmage are classified into four categoriesincluding registration based on atlas, based on image segmentationmethod, based on learning method and based on deep learning method. In our research, we concluded that the method based on deep learning should bemore promoted in the future by compering the existing problems and future development of MR image predicting CT image method.

Cross-modal retrieval method for thyroid ultrasound image and text based on generative adversarial network / 生物医学工程学杂志

Ultrasonic examination is a common method in thyroid examination, and the results are mainly composed of thyroid ultrasound images and text reports. Implementation of cross modal retrieval method of images and text reports can provide great convenience for doctors and patients, but currently there is no retrieval method to correlate thyroid ultrasound images with text reports. This paper proposes a cross-modal method based on the deep learning and improved cross-modal generative adversarial network: ①the weight sharing constraints between the fully connection layers used to construct the public representation space in the original network are changed to cosine similarity constraints, so that the network can better learn the common representation of different modal data; ②the fully connection layer is added before the cross-modal discriminator to merge the full connection layer of image and text in the original network with weight sharing. Semantic regularization is realized on the basis of inheriting the advantages of the original network weight sharing. The experimental results show that the mean average precision of cross modal retrieval method for thyroid ultrasound image and text report in this paper can reach 0.508, which is significantly higher than the traditional cross-modal method, providing a new method for cross-modal retrieval of thyroid ultrasound image and text report.

Crossmodal interference between language and flavour / Interferencia intermodal entre lenguaje y sabor

Resumen El objetivo del presente trabajo es describir la interferencia del Stroop crossmodal en representaciones del sabor, visuales y auditivas. Es un diseño mixto con dos grupos seleccionados al azar. Los estímulos fueron palabras presentadas de forma visual (grupo 1) o auditiva (grupo 2). La congruencia de estímulo (congruentes, incongruentes y control) se definió como un factor intrasujetos. Se registraron los tiempos de reacción y el número de respuestas correctas. Los resultados mostraron una interferencia Stroop crossmodal menor en condiciones de congruencia. Mientras con estímulos incongruentes y de control los tiempos de reacción se incrementaron y las tasas de precisión disminuyeron en ambos grupos experimentales. Los datos de los dos grupos se compararon y se concluyó que cuando el distractor era escrito la interferencia fue mayor que cuando era hablado. Estos resultados se discuten en términos de la dificultad de representación lingüística visual y en relación con estudios anteriores.

Abstract This work describes crossmodal Stroop interference in flavoural, visual, and auditory representations. A mixed design was used with two randomized groups. As a between-subjects factor, words were presented in visual (group 1) or auditory (group 2) forms. Stimulus congruency (congruent, incongruent, and control) was defined as a within-subjects factor. Reaction times and the number of correct answers were recorded. The results showed a minor crossmodal Stroop interference in conditions of congruency. In contrast, with incongruent and control stimuli, reaction times increased and accuracy rates diminished in both experimental groups. Data from the two groups were compared, and it was concluded that the interference was greater when the distractor was written than when it was spoken. These results are discussed in terms of the difficulty of visual linguistic representation and in relation to previous studies.

Synesthesia and music perception / Sinestesia e percepção musical

The present review examined the cross-modal association of sensations and their relationship to musical perception. Initially, the study focuses on synesthesia, its definition, incidence, forms, and genetic and developmentalfactors. The theories of the neural basis of synesthesia were also addressed by comparing theories emphasizing the anatomical aspect against others reinforcing the importance of physiological processes. Secondly, cross-modal sensory associations, their role in perception, and relationship to synesthesia were analyzed. We propose the existence of a lower, unconscious degree of synesthesia in non-synesthetes. This latent synesthesia (without explicit sensory manifestations) would be functional, aiding the construction of abstract associations between different perceptual s. Musical meaningmight be constructed largely by synesthetic processes, where the sensory associations from sound activate memories, images, and emotions.

No presente trabalho de revisão, investigamos o cruzamento de sensações e sua relação com a percepção musical. Inicialmente, nos debruçamos sobre o estudos da sinestesia, sua definição, incidência, formas, fatores genéticos e de desenvolvimento. Abordamos ainda as teorias sobre as bases neurais da sinestesia, comparando teorias que enfatizam os aspectos anatômicos a outras que reforçam a importância de processos fisiológicos. Numa segunda seção tratamos dos cruzamentos de modos sensoriais, seu papel na percepção e sua relação com a sinestesia. Cogitamos haver um grau desinestesia mais baixo e inconsciente em não sinestetas, esta sinestesia latente (sem manifestações sensoriais explícitas) seria funcional, auxiliando na construção de associações abstratas entre campos perceptivos diferentes. A significação musical talvez seja construída, em grande parte, por processos sinestésicos, em que as associações sensoriais a partir do sonoro ativam memórias, imagens e emoções.

Modulation of spatial attention to visual targets by emotional environmental sounds

Previous research has shown that visual spatial attention can be modulated by emotional prosody cues, but it is not known whether such crossmodal modulation of visual attention is associated with the engagement or disengagement of attentional resources. To test this, we employed a modified spatial cueing task, where participants indicated whether a visual target appeared either on the left or the right, after hearing a spatially non-predictive peripheral sound. Prior studies using prosody cues have found that modulation of visual attention by emotional auditory cues was lateralized, but this may have been due to the speech content of the stimuli; here instead we used non-speech environmental sounds. The sound was either emotional (pleasant, unpleasant) or neutral, and was presented either on the same side as the visual target ('valid' trial) or on the opposite side ('invalid' trial). For the cue validity index (RT to invalid cue minus RT to valid cue), we found differences between emotional and neutral cues, but only for visual targets presented in the right hemifield; here the cue validity index was lower for unpleasant compared to neutral and pleasant cues. Absolute RTs for targets on the right were faster for invalid trials following unpleasant cues, compared to pleasant and neutral cues, indicating that the reduced cue validity effect was due to faster disengagement from unpleasant auditory cues. Further, our results show that the laterality effect is related to the emotional nature of the cues, rather than the speech content of the stimuli...