Sampling inequalities affect generalization of neuroimaging-based diagnostic classifiers in psychiatry.

BACKGROUND: The development of machine learning models for aiding in the diagnosis of mental disorder is recognized as a significant breakthrough in the field of psychiatry. However, clinical practice of such models remains a challenge, with poor generalizability being a major limitation. METHODS: Here, we conducted a pre-registered meta-research assessment on neuroimaging-based models in the psychiatric literature, quantitatively examining global and regional sampling issues over recent decades, from a view that has been relatively underexplored. A total of 476 studies (n = 118,137) were included in the current assessment. Based on these findings, we built a comprehensive 5-star rating system to quantitatively evaluate the quality of existing machine learning models for psychiatric diagnoses. RESULTS: A global sampling inequality in these models was revealed quantitatively (sampling Gini coefficient (G) = 0.81, p < .01), varying across different countries (regions) (e.g., China, G = 0.47; the USA, G = 0.58; Germany, G = 0.78; the UK, G = 0.87). Furthermore, the severity of this sampling inequality was significantly predicted by national economic levels (ß = - 2.75, p < .001, R2adj = 0.40; r = - .84, 95% CI: - .41 to - .97), and was plausibly predictable for model performance, with higher sampling inequality for reporting higher classification accuracy. Further analyses showed that lack of independent testing (84.24% of models, 95% CI: 81.0-87.5%), improper cross-validation (51.68% of models, 95% CI: 47.2-56.2%), and poor technical transparency (87.8% of models, 95% CI: 84.9-90.8%)/availability (80.88% of models, 95% CI: 77.3-84.4%) are prevailing in current diagnostic classifiers despite improvements over time. Relating to these observations, model performances were found decreased in studies with independent cross-country sampling validations (all p < .001, BF10 > 15). In light of this, we proposed a purpose-built quantitative assessment checklist, which demonstrated that the overall ratings of these models increased by publication year but were negatively associated with model performance. CONCLUSIONS: Together, improving sampling economic equality and hence the quality of machine learning models may be a crucial facet to plausibly translating neuroimaging-based diagnostic classifiers into clinical practice.

Evaluation of Risk of Bias in Neuroimaging-Based Artificial Intelligence Models for Psychiatric Diagnosis: A Systematic Review.

Importance: Neuroimaging-based artificial intelligence (AI) diagnostic models have proliferated in psychiatry. However, their clinical applicability and reporting quality (ie, feasibility) for clinical practice have not been systematically evaluated. Objective: To systematically assess the risk of bias (ROB) and reporting quality of neuroimaging-based AI models for psychiatric diagnosis. Evidence Review: PubMed was searched for peer-reviewed, full-length articles published between January 1, 1990, and March 16, 2022. Studies aimed at developing or validating neuroimaging-based AI models for clinical diagnosis of psychiatric disorders were included. Reference lists were further searched for suitable original studies. Data extraction followed the CHARMS (Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modeling Studies) and PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analyses) guidelines. A closed-loop cross-sequential design was used for quality control. The PROBAST (Prediction Model Risk of Bias Assessment Tool) and modified CLEAR (Checklist for Evaluation of Image-Based Artificial Intelligence Reports) benchmarks were used to systematically evaluate ROB and reporting quality. Findings: A total of 517 studies presenting 555 AI models were included and evaluated. Of these models, 461 (83.1%; 95% CI, 80.0%-86.2%) were rated as having a high overall ROB based on the PROBAST. The ROB was particular high in the analysis domain, including inadequate sample size (398 of 555 models [71.7%; 95% CI, 68.0%-75.6%]), poor model performance examination (with 100% of models lacking calibration examination), and lack of handling data complexity (550 of 555 models [99.1%; 95% CI, 98.3%-99.9%]). None of the AI models was perceived to be applicable to clinical practices. Overall reporting completeness (ie, number of reported items/number of total items) for the AI models was 61.2% (95% CI, 60.6%-61.8%), and the completeness was poorest for the technical assessment domain with 39.9% (95% CI, 38.8%-41.1%). Conclusions and Relevance: This systematic review found that the clinical applicability and feasibility of neuroimaging-based AI models for psychiatric diagnosis were challenged by a high ROB and poor reporting quality. Particularly in the analysis domain, ROB in AI diagnostic models should be addressed before clinical application.

A Positive Association between Working Memory Capacity and Human Creativity: A Meta-Analytic Evidence.

Creativity serves as a fountain for social and scientific development. As one of the most crucial human capabilities, creativity has been believed to be supported by the core component of higher cognitive functions­working memory capacity (WMC). However, the evidence supporting the association between WMC and creativity remains contradictory. Here, we conducted a meta-analysis using random-effects models to investigate the linear association between WMC and creativity by pooling the individual effect size from the previous literature. Further, a subgroup analysis was performed to examine whether such association is specific for different WMC categories (i.e., verbal WMC, visual−spatial WMC and dual-task WMC). The main meta-analytic results showed a significantly positive association between WMC and creativity (r = .083, 95% CI: .050−.115, p < .001, n = 3104, k = 28). The subgroup analysis demonstrated consistent results by showing a significantly positive association between them, irrespective of WMC category. We also found that cultural environments could moderate this association, and we identified a strong correlation in participants from an Asian cultural context. In conclusion, this study provides the evidence to clarify the positive association between WMC and creativity, and implies that the Asian cultural context may boost such an association.

Revision and psychometric properties of the negative cognitive processing bias scale.

Negative cognitive processing bias (NCPB) is a cognitive trait that makes individuals more inclined to prioritize negative external stimuli (cues) when processing information. Cognitive biases have long been observed in mood and anxiety disorders, improving validation of tools to measure this phenomenon will aid us to determine whether there is a robust relationship between NCPB and major depressive disorder, anxiety disorders and other clinical disorders. Despite the development of an initial measure of this trait, that is, the negative cognitive processing bias questionnaire (NCPBQ), the lack of psychometric examinations and applications in large-scale samples hinders the determination of its reliability and validity and further limits our understanding of how to measure the NCPB traits of individuals accurately. To address these issues, the current study evaluated the psychometric properties of the NCPBQ in a large-scale sample (n = 6,069), which was divided into two subsamples (Subsample 1, n = 3,035, serving as the exploratory subsample, and Subsample 2, n = 3,034, serving as the validation subsample), and further revised it into a standardized scale, that is the negative cognitive processing bias scale (NCPBS), based on psychometric constructs. The results show that NCPBS possesses good construct reliability, internally consistent reliability, and test-retest reliability. Furthermore, by removing two original items from NCPBQ, NCPBS was found to have good criterion-related validity. In conclusion, the present study provides a reliable and valid scale for assessing negative cognitive processing bias of individuals.

Attitude and Performance for Online Learning during COVID-19 Pandemic: A Meta-Analytic Evidence.

The COVID-19 pandemic prominently hit almost all the aspects of our life, especially in routine education. For public health security, online learning has to be enforced to replace classroom learning. Thus, it is a priority to clarify how these changes impacted students. We built a random-effect model of a meta-analysis to pool individual effect sizes for published articles concerning the attitudes and performance towards online learning. Databases included Google Scholar, PubMed and (Chinese) CNKI repository. Further, a moderated analysis and meta-regression were further used to clarify potential heterogenous factors impacting this pooled effect. Forty published papers (n = 98,558) were screened that were eligible for formal analysis. Meta-analytic results demonstrated that 13.3% (95% CI: 10.0-17.5) of students possessed negative attitudes towards online learning during the COVID-19 pandemic. A total of 12.7% (95% CI: 9.6-16.8) students were found to report poor performance in online learning. Moderated analysis revealed poor performance in online learning in the early pandemic (p = 0.006). Results for the meta-regression analysis showed that negative attitudes could predict poor learning performance significantly (p = 0.026). In conclusion, online learning that is caused by COVID-19 pandemic may have brought about negative learning attitudes and poorer learning performance compared to classroom learning, especially in the early pandemic.

Engaging the open science framework in quantifying and tracing scientists' research credits.

Preface illustration. The "first-last-author-credit" hierarchy has long been dominated in the scientific incentive system despite intensive calling for contribution-based credits (author contribution statement). In the scientific communities, senior researchers would still make a decision to recommend one's promotion based on first and last positions in authorship rather than their contributions. Similarly, in the job market, institutions would acknowledge one's credit by positions in authorship in a study for faculty recruitment, while overlooking the author contribution statement at the end of studies. Thus, the current authorship system has brought on the risks underlying authorship disputes and race/gender inequalities in credit allocation heavily, especially for early career researchers and female scientists. In addition, this is one of the major barriers to extend teamwork and academic collaboration. On the contrary, scrambling for first and last positions leads to prominent credit inflation-that is to be observed-the number of co-first and co-corresponding authors has been increasing dramatically. Thus, we shall propose a new contributionship to acknowledge the author's credit for an open science and quantitative framework to tackle these issues. Credit: ZC and XRL.

The mucosal adjuvant effect of plant polysaccharides for induction of protective immunity against Helicobacter pylori infection.

Some plant polysaccharides (PPSs) had been used as the adjuvants for systemic vaccination. In this study, we investigated whether PPSs could exhibit adjuvant effect at the mucosa. Groups of mice were intranasally immunized with Epimedium Polysaccharide (EPS), Trollius chinensis polysaccharide (TCPS), Siberian solomonseal rhizome polysaccharide (SSRPS) and Astragalus polysaccharides (APS) together with ovalbumin (OVA). Significantly higher levels of OVA-specific IgG in serum and secretory IgA in saliva, vaginal wash and intestinal lavage fluid were induced after immunization with OVA plus one of the four PPSs compared to OVA alone. Antigen absorption and TLR2 (Toll-like receptor 2) activation may be related to their mucosal adjuvant effect. Of note, when APS used as an adjuvant, intranasally vaccination with recombination UreB (rUreB, Urease subunit B) conferred more robust protection against Helicobacter pylori (H. pylori). Immunized with rUreB in combination APS resulted in mixed specific Th1 and Th17 immune response, which may contribute to the inhibition of H. pylori colonization. Though specific Th2-dominant responses were elicited when the other three PPS intranasally immunized with rUreB, no significant difference in the protective effect were found between those groups and rUreb alone group. Taken together, the four PPSs may be promising candidates for mucosal adjuvant, and APS could enhance rUreB-specific protective immunity against H. pylori infection.