BlinkLinMulT: Transformer-Based Eye Blink Detection.

This work presents BlinkLinMulT, a transformer-based framework for eye blink detection. While most existing approaches rely on frame-wise eye state classification, recent advancements in transformer-based sequence models have not been explored in the blink detection literature. Our approach effectively combines low- and high-level feature sequences with linear complexity cross-modal attention mechanisms and addresses challenges such as lighting changes and a wide range of head poses. Our work is the first to leverage the transformer architecture for blink presence detection and eye state recognition while successfully implementing an efficient fusion of input features. In our experiments, we utilized several publicly available benchmark datasets (CEW, ZJU, MRL Eye, RT-BENE, EyeBlink8, Researcher's Night, and TalkingFace) to extensively show the state-of-the-art performance and generalization capability of our trained model. We hope the proposed method can serve as a new baseline for further research.

Cross-Viewpoint Semantic Mapping: Integrating Human and Robot Perspectives for Improved 3D Semantic Reconstruction.

Allocentric semantic 3D maps are highly useful for a variety of human-machine interaction related tasks since egocentric viewpoints can be derived by the machine for the human partner. Class labels and map interpretations, however, may differ or could be missing for the participants due to the different perspectives. Particularly, when considering the viewpoint of a small robot, which significantly differs from the viewpoint of a human. In order to overcome this issue, and to establish common ground, we extend an existing real-time 3D semantic reconstruction pipeline with semantic matching across human and robot viewpoints. We use deep recognition networks, which usually perform well from higher (i.e., human) viewpoints but are inferior from lower viewpoints, such as that of a small robot. We propose several approaches for acquiring semantic labels for images taken from unusual perspectives. We start with a partial 3D semantic reconstruction from the human perspective that we transfer and adapt to the small robot's perspective using superpixel segmentation and the geometry of the surroundings. The quality of the reconstruction is evaluated in the Habitat simulator and a real environment using a robot car with an RGBD camera. We show that the proposed approach provides high-quality semantic segmentation from the robot's perspective, with accuracy comparable to the original one. In addition, we exploit the gained information and improve the recognition performance of the deep network for the lower viewpoints and show that the small robot alone is capable of generating high-quality semantic maps for the human partner. The computations are close to real-time, so the approach enables interactive applications.

Design of a Machine Learning System to Predict the Thickness of a Melanoma Lesion in a Non-Invasive Way from Dermoscopic Images.

OBJECTIVES: Melanoma is the deadliest form of skin cancer, but it can be fully cured through early detection and treatment in 99% of cases. Our aim was to develop a non-invasive machine learning system that can predict the thickness of a melanoma lesion, which is a proxy for tumor progression, through dermoscopic images. This method can serve as a valuable tool in identifying urgent cases for treatment. METHODS: A modern convolutional neural network architecture (EfficientNet) was used to construct a model capable of classifying dermoscopic images of melanoma lesions into three distinct categories based on thickness. We incorporated techniques to reduce the impact of an imbalanced training dataset, enhanced the generalization capacity of the model through image augmentation, and utilized five-fold cross-validation to produce more reliable metrics. RESULTS: Our method achieved 71% balanced accuracy for three-way classification when trained on a small public dataset of 247 melanoma images. We also presented performance projections for larger training datasets. CONCLUSIONS: Our model represents a new state-of-the-art method for classifying melanoma thicknesses. Performance can be further optimized by expanding training datasets and utilizing model ensembles. We have shown that earlier claims of higher performance were mistaken due to data leakage during the evaluation process.

Peripheral gene interactions define interpretable clusters of core ASD genes in a network-based investigation of the omnigenic theory.

According to the recently proposed omnigenic theory, all expressed genes in a relevant tissue are contributing directly or indirectly to the manifestation of complex disorders such as autism. Thus, holistic approaches can be complementary in studying genetics of these complex disorders to focusing on a limited number of candidate genes. Gene interaction networks can be used for holistic studies of the omnigenic nature of autism. We used Louvain clustering on tissue-specific gene interaction networks and their subgraphs exclusively containing autism-related genes to study the effects of peripheral gene interactions. We observed that the autism gene clusters are significantly weaker connected to each other and the peripheral genes in non-neuronal tissues than in brain-related tissues. The biological functions of the brain clusters correlated well with previous findings on autism, such as synaptic signaling, regulation of DNA methylation, or regulation of lymphocyte activation, however, on the other tissues they did not enrich as significantly. Furthermore, ASD subjects with disruptive mutations in specific gene clusters show phenotypical differences compared to other disruptive variants carrying ASD individuals. Our results strengthen the omnigenic theory and can advance our understanding of the genetic background of autism.

Tracking Highly Similar Rat Instances under Heavy Occlusions: An Unsupervised Deep Generative Pipeline.

Identity tracking and instance segmentation are crucial in several areas of biological research. Behavior analysis of individuals in groups of similar animals is a task that emerges frequently in agriculture or pharmaceutical studies, among others. Automated annotation of many hours of surveillance videos can facilitate a large number of biological studies/experiments, which otherwise would not be feasible. Solutions based on machine learning generally perform well in tracking and instance segmentation; however, in the case of identical, unmarked instances (e.g., white rats or mice), even state-of-the-art approaches can frequently fail. We propose a pipeline of deep generative models for identity tracking and instance segmentation of highly similar instances, which, in contrast to most region-based approaches, exploits edge information and consequently helps to resolve ambiguity in heavily occluded cases. Our method is trained by synthetic data generation techniques, not requiring prior human annotation. We show that our approach greatly outperforms other state-of-the-art unsupervised methods in identity tracking and instance segmentation of unmarked rats in real-world laboratory video recordings.

The Autism Palette: Combinations of Impairments Explain the Heterogeneity in ASD.

Autism spectrum disorder (ASD) is a heterogeneous neuropsychiatric condition traditionally defined by core symptoms in social behavior, speech/communication, repetitive behavior, and restricted interests. Beyond the core symptoms, autism has strong association with other disorders such as intellectual disability (ID), epilepsy, schizophrenia among many others. This paper outlines a theory of ASD with capacity to connect heterogeneous "core" symptoms, medical and psychiatric comorbidities as well as other etiological theories of autism in a unifying cognitive framework rooted in neuroscience and genetics. Cognition is embedded into an ever-developing structure modified by experiences, including the outcomes of environment influencing behaviors. The key constraint of cognition is that the brain can handle only 7±2 relevant variables at a time, whereas sensory variables, i.e., the number of sensory neurons is orders of magnitude larger. As a result, (a) the extraction, (b) the encoding, and (c) the capability for the efficient cognitive manipulation of the relevant variables, and (d) the compensatory mechanisms that counteract computational delays of the distributed components are critical. We outline our theoretical model to describe a Cartesian Factor (CF) forming, autoencoder-like cognitive mechanism which breaks combinatorial explosion and is accelerated by internal reinforcing machineries and discuss the neural processes that support CF formation. Impairments in any of these aspects may disrupt learning, cognitive manipulation, decisions on interactions, and execution of decisions. We suggest that social interactions are the most susceptible to combinations of diverse small impairments and can be spoiled in many ways that pile up. Comorbidity is experienced, if any of the many potential impairments is relatively strong. We consider component spoiling impairments as the basic colors of autism, whereas the combinations of individual impairments make the palette of autism. We put forth arguments on the possibility of dissociating the different main elements of the impairments that can appear together. For example, impairments of generalization (domain general learning) and impairments of dealing with many variable problems, such as social situations may appear independently and may mutually enhance their impacts. We also consider mechanisms that may lead to protection.

Role of oligo(malic acid) on the formation of unilamellar vesicles.

Stable unilamellar dipalmitoylphosphatidylcholine vesicles were produced by using oligo(malic acid) and cholesterol. Detailed physico-chemical characterization prove that by using oligo(malic acid) the substitution of PEGylated lipids for sterically stabilization comes possible. The polymer molecules cover the outer surface of spherical-shaped vesicles, and an asymmetrical composition occurs in the two leaflets of the phospholipid bilayer. The oligo(malic-acid) and cholesterol are enriched in the outer side assuring the stabilization of vesicles. Cholesterol plays an important role in the self-assembly of components as it makes the entering of oligomers possible deep into the polar head-region of lipids. The presence of oligo(malic acid) molecules does not induce degradation by hydrolysis of lipid molecules but the vesicle system turns into a sensitive form giving a possibility for pH sensitive targeting. Preliminary investigation on the investigated oligo(malic acid)-stabilized vesicles do not show any toxic effect promising their applicability in the field of liposomal drug delivery.

Correction: Strategic predictors of performance in a divided attention task.

[This corrects the article DOI: 10.1371/journal.pone.0195131.].

Strategic predictors of performance in a divided attention task.

In this study we investigate the strategies of subjects in a complex divided attention task. We conducted a series of experiments with ten participants and evaluated their performance. After an extensive analysis, we identified four strategic measures that justify the achievement of the participants, by highlighting the individual differences and predicting performance in a regression analysis using generalized estimating equations. Selecting the more urgent task and user action between multiple simultaneous possibilities form two of the strategic decisions, respectively. The third one refers to choosing a response within the same task when the opportunity is present. The fourth and most important measure of strategy involves thinking ahead and executing an action before a situation would become critical. This latter one has the effect of reducing later cognitive load or timing constraints and it is shown to explain almost as much variance in performance as the other three, more straightforward predictors together. In addition to determining these strategic predictors, we also show how manipulating task difficulty induces a shift in strategy, thus impairing human performance in the rehearsed task. The results of this study indicate that considerable differences in the divided attention ability of normal subjects can be identified early and with simple measurements. The importance of describing and analyzing strategies is also emphasized, which can substantially influence performance in complex tasks and may serve training needs.

Semi-Supervised Learning of Cartesian Factors: A Top-Down Model of the Entorhinal Hippocampal Complex.

The existence of place cells (PCs), grid cells (GCs), border cells (BCs), and head direction cells (HCs) as well as the dependencies between them have been enigmatic. We make an effort to explain their nature by introducing the concept of Cartesian Factors. These factors have specific properties: (i) they assume and complement each other, like direction and position and (ii) they have localized discrete representations with predictive attractors enabling implicit metric-like computations. In our model, HCs make the distributed and local representation of direction. Predictive attractor dynamics on that network forms the Cartesian Factor "direction." We embed these HCs and idiothetic visual information into a semi-supervised sparse autoencoding comparator structure that compresses its inputs and learns PCs, the distributed local and direction independent (allothetic) representation of the Cartesian Factor of global space. We use a supervised, information compressing predictive algorithm and form direction sensitive (oriented) GCs from the learned PCs by means of an attractor-like algorithm. Since the algorithm can continue the grid structure beyond the region of the PCs, i.e., beyond its learning domain, thus the GCs and the PCs together form our metric-like Cartesian Factors of space. We also stipulate that the same algorithm can produce BCs. Our algorithm applies (a) a bag representation that models the "what system" and (b) magnitude ordered place cell activities that model either the integrate-and-fire mechanism, or theta phase precession, or both. We relate the components of the algorithm to the entorhinal-hippocampal complex and to its working. The algorithm requires both spatial and lifetime sparsification that may gain support from the two-stage memory formation of this complex.

MDR-1 and MRP-1 activity in peripheral blood leukocytes of rheumatoid arthritis patients.

BACKGROUND: Rheumatoid Arthritis is a chronic disease leading to decreased quality of life with a rather variable response rate to Disease Modifying Anti Rheumatic Drugs. Methotrexate (MTX) is the gold standard therapy in Rheumatoid Arthritis. The Multidrug resistance Related Protein and Multi Drug Resistance protein 1, also called P-glycoprotein-170 transporters can alter the intracellular concentration of different drugs. Methotrexate is an MRP1 substrate and thus the functional activity of MRP1 might have a clinical impact on the efficiency of the Methotrexate-therapy in Rheumatoid Arthritis. METHODS: We have compared the functional Multidrug Activity Factors (MAF) of the MDR1 and MRP1 transporters of Peripheral Blood Leukocytes of 59 Rheumatoid Arthritis patients with various response rate to MTX-therapy (MTX-responder, MTX-resistant and MTX-intolerant RA-groups) and 47 non-RA controls in six different leukocyte subpopulations (neutrophil leukocytes, monocytes, lymphocytes, CD4+, CD8+ and CD19+ cells). There was a decreased MAF of RA patients compared to non- Rheumatoid Arthritis patients and healthy controls in the leukocyte subpopulations. There was a significant difference between the MAF values of the MTX-responder and MTX intolerant groups. But we have not found significant differences between the MAF values of the MTX-responder and MTX-resistant Rheumatoid Arthritis -groups. RESULTS: Our results suggest that MDR1 and MRP1 functional activity does not seem to affect the response rate to MTX-therapy of Rheumatoid Arthritis-patients, but it might be useful in predicting MTX-side effects. We have demonstrated the decreased functional MDR-activity on almost 60 Rheumatoid Arthritis patients, which can be interpreted as a sign of the immune-suppressive effect of the MTX-treatment.

Decreased functional activity of multidrug resistance protein in primary colorectal cancer.

BACKGROUND: The ATP-Binding Cassette (ABC)-transporter MultiDrug Resistance Protein 1 (MDR1) and Multidrug Resistance Related Protein 1 (MRP1) are expressed on the surface of enterocytes, which has led to the belief that these high capacity transporters are responsible for modulating chemosensitvity of colorectal cancer. Several immunohistochemistry and reverse transcription polymerase chain reaction (RT-PCR) studies have provided controversial results in regards to the expression levels of these two ABC-transporters in colorectal cancer. Our study was designed to determine the yet uninvestigated functional activity of MDR1 and MRP1 transporters in normal human enterocytes compared to colorectal cancer cells from surgical biopsies. METHODS: 100 colorectal cancer and 28 adjacent healthy mucosa samples were obtained by intraoperative surgical sampling. Activity of MDR1 and MRP1 of viable epithelial and cancer cells were determined separately with the modified calcein-assay for multidrug resistance activity and sufficient data of 73 cancer and 11 healthy mucosa was analyzed statistically. RESULTS: Significantly decreased mean MDR1 activity was found in primary colorectal cancer samples compared to normal mucosa, while mean MRP1 activity showed no significant change. Functional activity was not affected by gender, age, stage or grade and localization of the tumor. CONCLUSION: We found lower MDR activity in cancer cells versus adjacent, apparently, healthy control tissue, thus, contrary to general belief, MDR activity seems not to play a major role in primary drug resistance, but might rather explain preferential/selective activity of Irinotecan and/or Oxaliplatin. Still, this picture might be more complex since chemotherapy by itself might alter MDR activity, and furthermore, today limited data is available about MDR activity of cancer stem cells in colorectal cancers. VIRTUAL SLIDES: The virtual slide(s) for this article can be found here: http://www.diagnosticpathology.diagnomx.eu/vs/1675739129145824.

Effects of ursolic acid on the structural and morphological behaviours of dipalmitoyl lecithin vesicles.

Effects of ursolic acid on the structural and morphological characteristics of dipalmitoyl lecithin(DPPC)-water system was studied by using differential scanning calorimetry (DSC), small- and wide-angle X-ray scattering (SWAXS), freeze-fracture method combined with transmission electron-microscopy (FF-TEM) and infrared spectroscopy (FT-IR). The surface of the uncorrelated lipid system is rippled or grained and a huge number of small, presumably unilamellar vesicles are present if the UA/DPPC molar ratio is 0.1 mol/mol or higher. Besides the destroyed layer packing of regular multilamellar vesicles, non-bilayer (e.g. cubic or hexagonal) local structures are evidenced by SAXS and FF-TEM methods. The ability of UA to induce non-bilayer structures in hydrated DPPC system originates from the actual geometry form of associated lipid and UA molecules as concluded from the FT-IR measurements and theoretical calculations. Beside numerous beneficial e.g. chemopreventive and chemotherapeutic effect of ursolic acid against cancer, their impact to modify the lipid bilayers can be utilized in liposomal formulations.

Spatio-temporal Event Classification using Time-series Kernel based Structured Sparsity.

In many behavioral domains, such as facial expression and gesture, sparse structure is prevalent. This sparsity would be well suited for event detection but for one problem. Features typically are confounded by alignment error in space and time. As a consequence, high-dimensional representations such as SIFT and Gabor features have been favored despite their much greater computational cost and potential loss of information. We propose a Kernel Structured Sparsity (KSS) method that can handle both the temporal alignment problem and the structured sparse reconstruction within a common framework, and it can rely on simple features. We characterize spatio-temporal events as time-series of motion patterns and by utilizing time-series kernels we apply standard structured-sparse coding techniques to tackle this important problem. We evaluated the KSS method using both gesture and facial expression datasets that include spontaneous behavior and differ in degree of difficulty and type of ground truth coding. KSS outperformed both sparse and non-sparse methods that utilize complex image features and their temporal extensions. In the case of early facial event classification KSS had 10% higher accuracy as measured by F1 score over kernel SVM methods.

Efficient sparse coding in early sensory processing: lessons from signal recovery.

Sensory representations are not only sparse, but often overcomplete: coding units significantly outnumber the input units. For models of neural coding this overcompleteness poses a computational challenge for shaping the signal processing channels as well as for using the large and sparse representations in an efficient way. We argue that higher level overcompleteness becomes computationally tractable by imposing sparsity on synaptic activity and we also show that such structural sparsity can be facilitated by statistics based decomposition of the stimuli into typical and atypical parts prior to sparse coding. Typical parts represent large-scale correlations, thus they can be significantly compressed. Atypical parts, on the other hand, represent local features and are the subjects of actual sparse coding. When applied on natural images, our decomposition based sparse coding model can efficiently form overcomplete codes and both center-surround and oriented filters are obtained similar to those observed in the retina and the primary visual cortex, respectively. Therefore we hypothesize that the proposed computational architecture can be seen as a coherent functional model of the first stages of sensory coding in early vision.

Perception based method for the investigation of audiovisual integration of speech.

Speech comprehension is significantly improved by visual input on the speaker's mouth movements. Audiovisual integration underlying this phenomenon is often studied in EEG experiments in which the event related brain potential (ERP) elicited by a bimodal stimulus is compared to the sum of ERPs triggered by auditory and visual signals of the same source. However, this method leads to spurious results in time ranges when ERP components common to all these stimulus types are present. A method that aims to filter out such common early anticipatory potentials is data high-pass filtering. In the present study, first, we demonstrated that subtle changes in filter cut-off frequency lead to remarkably different results on the interaction effect so that no reliable conclusion on the spatial distribution of the interaction could be drawn. Second, we suggested a different approach for the investigation of ERP correlates of audiovisual integration: bimodal syllables modified by light temporal asynchrony were presented to subjects and ERPs correlating with the fused and unfused perceptions were compared. We found that components corresponding to both auditory N1 and P2 waves were smaller in case of the fused perception, supporting the view that N1 and P2 generator activities are suppressed during multimodal speech perception. The N1 effect showed a clearly right hemisphere dominance while the effect around the P2 peak was most pronounced on centroparietal electrodes and dominated over the left hemisphere.

Here and now: how time segments may become events in the hippocampus.

The hippocampal formation is believed to play a central role in memory functions related to the representation of events. Events are usually considered as temporally bounded processes, in contrast to the continuous nature of sensory signal flow they originate from. Events are then organized and stored according to behavioral relevance and are used to facilitate prediction of similar events. In this paper we are interested in the kind of representation of sensory signals that allows for detecting and/or predicting events. Based on new results on the identification problem of linear hidden processes, we propose a connectionist network with biologically sound parameter tuning that can represent causal relationships and define events. Interestingly, the wiring diagram of our architecture not only resembles the gross anatomy of the hippocampal formation (including the entorhinal cortex), but it also features similar spatial distribution functions of activity (localized and periodic, 'grid-like' patterns) as found in the different parts of the hippocampal formation. We shortly discuss how our model corresponds to different theories on the role of the hippocampal formation in forming episodic memories or supporting spatial navigation. We speculate that our approach may constitute a step toward a unified theory about the functional role of the hippocampus and the structure of memory representations.

Novel anti-cholesterol monoclonal immunoglobulin G antibodies as probes and potential modulators of membrane raft-dependent immune functions.

Natural autoantibodies against cholesterol are present in the sera of all healthy individuals; their function, production, and regulation, however, are still unclear. Here, we managed to produce two monoclonal anti-cholesterol antibodies (ACHAs) by immunizing mice with cholesterol-rich liposomes. The new ACHAs were specific to cholesterol and to some structurally closely related 3beta-hydroxyl sterols, and they reacted with human lipoproteins VLDL, LDL, and HDL. They bound, usually with low avidity, to live human or murine lymphocyte and monocyte-macrophage cell lines, which was enhanced substantially by a moderate papain digestion of the cell surface, removing some protruding extracellular protein domains. Cell-bound ACHAs strongly colocalized with markers of cholesterol-rich lipid rafts and caveolae at the cell surface and intracellularly with markers of the endoplasmic reticulum and Golgi complex. These data suggest that these IgG ACHAs may serve as probes of clustered cholesterol (e.g., different lipid rafts) in live cells and thus may also have immunomodulatory potential.

Learning Tetris using the noisy cross-entropy method.

The cross-entropy method is an efficient and general optimization algorithm. However, its applicability in reinforcement learning (RL) seems to be limited because it often converges to suboptimal policies. We apply noise for preventing early convergence of the cross-entropy method, using Tetris, a computer game, for demonstration. The resulting policy outperforms previous RL algorithms by almost two orders of magnitude.