Gender differences in peer reviewed grant applications, awards, and amounts: a systematic review and meta-analysis.

BACKGROUND: Differential participation and success in grant applications may contribute to women's lesser representation in the sciences. This study's objective was to conduct a systematic review and meta-analysis to address the question of gender differences in grant award acceptance rates and reapplication award acceptance rates (potential bias in peer review outcomes) and other grant outcomes. METHODS: The review was registered on PROSPERO (CRD42021232153) and conducted in accordance with PRISMA 2020 standards. We searched Academic Search Complete, PubMed, and Web of Science for the timeframe 1 January 2005 to 31 December 2020, and forward and backward citations. Studies were included that reported data, by gender, on any of the following: grant applications or reapplications, awards, award amounts, award acceptance rates, or reapplication award acceptance rates. Studies that duplicated data reported in another study were excluded. Gender differences were investigated by meta-analyses and generalized linear mixed models. Doi plots and LFK indices were used to assess reporting bias. RESULTS: The searches identified 199 records, of which 13 were eligible. An additional 42 sources from forward and backward searches were eligible, for a total of 55 sources with data on one or more outcomes. The data from these studies ranged from 1975 to 2020: 49 sources were published papers and six were funders' reports (the latter were identified by forwards and backwards searches). Twenty-nine studies reported person-level data, 25 reported application-level data, and one study reported both: person-level data were used in analyses. Award acceptance rates were 1% higher for men, which was not significantly different from women (95% CI 3% more for men to 1% more for women, k = 36, n = 303,795 awards and 1,277,442 applications, I2 = 84%). Reapplication award acceptance rates were significantly higher for men (9%, 95% CI 18% to 1%, k = 7, n = 7319 applications and 3324 awards, I2 = 63%). Women received smaller award amounts (g = -2.28, 95% CI -4.92 to 0.36, k = 13, n = 212,935, I2 = 100%). CONCLUSIONS: The proportions of women that applied for grants, re-applied, accepted awards, and accepted awards after reapplication were less than the proportion of eligible women. However, the award acceptance rate was similar for women and men, implying no gender bias in this peer reviewed grant outcome. Women received smaller awards and fewer awards after re-applying, which may negatively affect continued scientific productivity. Greater transparency is needed to monitor and verify these data globally.

Depression trial results: A cross-sectional study of ClinicalTrials.gov.

Published clinical trials represent a subsample of the objective information needed to appraise treatments for depression. We characterize the extent of selective and delayed reporting in a systematic review (PROSPERO #CRD42020173606) of depression trial results registered on ClinicalTrials.gov. Inclusion criteria were studies registered on ClinicalTrials.gov with depression as the condition, had enrolled ages 18 and over, were completed between January 1, 2008 and May 1, 2019, and had posted results by February 1, 2022. Cox regression analyses of time to result posting from registration and from study completion included enrollment as a covariate. Among 442 protocols, median result posting occurred over two years after study completion and five years after registration. Among protocols with incomplete results, effect sizes (d or W) were calculated for 134 protocols. Median effect sizes for protocols with incomplete results were small (0.16, 95% CI 0.08, 0.21). For 28% of protocols, observed effects were contrary to the expected direction. Between-group effect size calculations were based on post-treatment data as pre-treatment data were inconsistently provided. Although drug and device trials in the U.S. are required to register on ClinicalTrials.gov, compliance is imperfect, and submissions are not peer reviewed. For depression treatment trials, long intervals between study completion and posting of results are common. Further, investigators often fail to report the results of statistical tests. Failure to post trial results in a timely manner and omission of statistical test reporting may lead to overestimates of treatment effects in systematic literature reviews.

Peer review: Risk and risk tolerance.

Peer review, commonly used in grant funding decisions, relies on scientists' ability to evaluate research proposals' quality. Such judgments are sometimes beyond reviewers' discriminatory power and could lead to a reliance on subjective biases, including preferences for lower risk, incremental projects. However, peer reviewers' risk tolerance has not been well studied. We conducted a cross-sectional experiment of peer reviewers' evaluations of mock primary reviewers' comments in which the level and sources of risks and weaknesses were manipulated. Here we show that proposal risks more strongly predicted reviewers' scores than proposal strengths based on mock proposal evaluations. Risk tolerance was not predictive of scores but reviewer scoring leniency was predictive of overall and criteria scores. The evaluation of risks dominates reviewers' evaluation of research proposals and is a source of inter-reviewer variability. These results suggest that reviewer scoring variability may be attributed to the interpretation of proposal risks, and could benefit from intervention to improve the reliability of reviews. Additionally, the valuation of risk drives proposal evaluations and may reduce the chances that risky, but highly impactful science, is supported.

Couples and COVID-19 vaccination: Frequency and reasons for discordance.

The purpose of this study was to determine the frequency of concordance and discordance in the COVID-19 vaccination status of intimate couples. Partners are well known to influence each other's health behaviors, but previous reports of couples' COVID-19 vaccination status were not found. Among 1305 respondents to an on-line survey who were living with their partners, couples' COVID-19 vaccination status was 15.63% discordant and 84.37% concordant. Different potentially modifiable reasons for being unvaccinated were emphasized by unvaccinated participants with vaccinated partners and by vaccinated participants with unvaccinated partners. This study provides an estimate of the proportion of couples with discordant COVID-19 vaccine status. The development of tailored, scalable interventions potentially involving the partner might increase the COVID-19 vaccine concordance rate.

Transparency of results reporting for depression treatment studies in ClinicalTrials.gov: a cross-sectional study.

OBJECTIVES: Depression affects an estimated 7% of the adult population at an estimated cost of over US$200 billion/year in the USA. Complete, transparent reporting of clinical trial data facilitates valid estimates of treatment efficacy. In the USA, ClinicalTrials.gov increases transparency through mandatory prospective trial registration and outcome reporting. We examined characteristics of the transparent reporting of depression treatment studies registered in ClinicalTrials.gov. DESIGN: Cross sectional. SETTING AND PARTICIPANTS: US-based studies identified in a search of ClinicalTrials.gov with depression as the condition, enrolling ages 18 and older, and completed between 1 January 2008 and 1 May 2019. INTERVENTIONS: All interventions were included. MAIN OUTCOMES AND MEASURES: The main outcome was whether any results were reported prior to 1 May 2020. Data were extracted regarding inclusion and exclusion criteria, publications related to the study and specification of hypotheses. RESULTS: 725 studies involving 156 634 patients met inclusion criteria. 416 (57.4%) of the studies posted some results. However, statistical test results were not included in 230 studies (55.3%). Most studies had data that could have been analysed and reported. Compared with studies without results, studies with any results were more likely to have hypotheses, include drug treatment conditions, and to have publications related to the study. CONCLUSIONS: Required study registration does not always result in transparent outcome reporting. Better compliance with mandated reporting and improved reporting standards would facilitate a more comprehensive representation of depression treatment research.

Grant Review Feedback: Appropriateness and Usefulness.

The primary goal of the peer review of research grant proposals is to evaluate their quality for the funding agency. An important secondary goal is to provide constructive feedback to applicants for their resubmissions. However, little is known about whether review feedback achieves this goal. In this paper, we present a multi-methods analysis of responses from grant applicants regarding their perceptions of the effectiveness and appropriateness of peer review feedback they received from grant submissions. Overall, 56-60% of applicants determined the feedback to be appropriate (fair, well-written, and well-informed), although their judgments were more favorable if their recent application was funded. Importantly, independent of funding success, women found the feedback better written than men, and more white applicants found the feedback to be fair than non-white applicants. Also, perceptions of a variety of biases were specifically reported in respondents' feedback. Less than 40% of applicants found the feedback to be very useful in informing their research and improving grantsmanship and future submissions. Further, negative perceptions of the appropriateness of review feedback were positively correlated with more negative perceptions of feedback usefulness. Importantly, respondents suggested that highly competitive funding pay-lines and poor inter-panel reliability limited the usefulness of review feedback. Overall, these results suggest that more effort is needed to ensure that appropriate and useful feedback is provided to all applicants, bolstering the equity of the review process and likely improving the quality of resubmitted proposals.

Efficacy and effectiveness studies of depression are not well-differentiated in the literature: a systematic review.

BACKGROUND: In the literature on the treatment of depression, efficacy and effectiveness research have different purposes and should apply different research methodologies. OBJECTIVE: The purpose of the study was to review characteristics of depression treatment studies identified using efficacy or effectiveness search terms. We considered subject inclusion and exclusion criteria; numbers of subjects enrolled and the proportion in the primary analyses; inclusion of a Consolidated Standards of Reporting Trials (CONSORT) flow diagram; use of random assignment; use of placebo control conditions; lengths of treatment and follow-up; primary outcome variable; trial registration; journal impact factor. STUDY SELECTION: Studies indexed as efficacy AND 'real-world' AND depression or effectiveness AND 'real-world' AND depression in PubMed up to 18 May 2019. FINDINGS: 27 studies met the inclusion criteria: 13 effectiveness studies, 6 efficacy studies and 8 studies indexed as both effectiveness and efficacy. Studies identified as effectiveness, efficacy, or both differed on three outcome measures: the inclusion criteria were lengthier for efficacy than for effectiveness studies; efficacy studies were more likely to have a placebo control condition than effectiveness studies; and the journal impact factor was lower for effectiveness studies than for studies from the efficacy search or studies identified by both searches. CONCLUSIONS: Efficacy and effectiveness research hypothetically use different methodologies, but the efficacy and effectiveness literatures in the treatment of depression were comparable for most of the coded characteristics. The lack of distinguishable characteristics suggests that variably applied terminology may hinder efforts to narrow the gap between research and practice. PROSPERO REGISTRATION NUMBER: #CRD42019136840.

Treatment for Borderline Personality Disorder and Secondary Effects on Somatization.

Borderline personality disorder (BPD) is often accompanied by other diagnoses. Some comorbidities have received a good deal of attention, but others, including somatization, have not. As effective treatments for BPD are identified, the secondary effects of these treatments on BPD comorbidities are of clinical relevance. Secondary data were used from a randomized clinical trial of dialectical behavior therapy (DBT) and community treatment by experts among 101 women with BPD in order to characterize change in somatization with treatment for BPD, and to examine emotional avoidance as a mediator. Somatization decreased significantly over time, which did not differ by treatment condition. DBT had an indirect effect on somatization through less emotional avoidance. These results suggest that the comorbid presence of significant somatization should be evaluated among those with BPD, especially in the context of emotional avoidance; treatments could be enhanced by addressing emotion regulation skills.

The terminology conflict on efficacy and effectiveness in healthcare.

Designers and architects created the rule 'form follows function (FFF)' for their own profession. Our paper demonstrates that this FFF rule applies equally well to the designers of clinical studies. Four examples present are as follows: disregarding this FFF rule causes an inconsistent terminology to differentiate between efficacy and effectiveness, inconsistent differentiation of efficacy and effectiveness interferes with the consistent interpretation of the results of clinical studies, inconsistent interpretation of clinical studies results in an unexpectedly variance of recommendations in clinical guidelines and the fusion of the FFF designer rule and of the demands of Cochrane and Bradford Hill ('can it work?', 'does it work?' and 'is it worth it?') avoids the terminology problem and its misleading consequences. This strategy is presented.

Grant reviewer perceptions of the quality, effectiveness, and influence of panel discussion.

BACKGROUND: Funding agencies have long used panel discussion in the peer review of research grant proposals as a way to utilize a set of expertise and perspectives in making funding decisions. Little research has examined the quality of panel discussions and how effectively they are facilitated. METHODS: Here, we present a mixed-method analysis of data from a survey of reviewers focused on their perceptions of the quality, effectiveness, and influence of panel discussion from their last peer review experience. RESULTS: Reviewers indicated that panel discussions were viewed favorably in terms of participation, clarifying differing opinions, informing unassigned reviewers, and chair facilitation. However, some reviewers mentioned issues with panel discussions, including an uneven focus, limited participation from unassigned reviewers, and short discussion times. Most reviewers felt the discussions affected the review outcome, helped in choosing the best science, and were generally fair and balanced. However, those who felt the discussion did not affect the outcome were also more likely to evaluate panel communication negatively, and several reviewers mentioned potential sources of bias related to the discussion. While respondents strongly acknowledged the importance of the chair in ensuring appropriate facilitation of the discussion to influence scoring and to limit the influence of potential sources of bias from the discussion on scoring, nearly a third of respondents did not find the chair of their most recent panel to have performed these roles effectively. CONCLUSIONS: It is likely that improving chair training in the management of discussion as well as creating review procedures that are informed by the science of leadership and team communication would improve review processes and proposal review reliability.

The Participation and Motivations of Grant Peer Reviewers: A Comprehensive Survey.

Scientific peer reviewers play an integral role in the grant selection process, yet very little has been reported on the levels of participation or the motivations of scientists to take part in peer review. The American Institute of Biological Sciences (AIBS) developed a comprehensive peer review survey that examined the motivations and levels of participation of grant reviewers. The survey was disseminated to 13,091 scientists in AIBS's proprietary database. Of the 874 respondents, 76% indicated they had reviewed grant applications in the last 3 years; however, the number of reviews was unevenly distributed across this sample. Higher review loads were associated with respondents who had submitted more grant proposals over this time period, some of whom were likely to be study section members for large funding agencies. The most prevalent reason to participate in a review was to give back to the scientific community (especially among frequent grant submitters) and the most common reason to decline an invitation to review was lack of time. Interestingly, few suggested that expectation from the funding agency was a motivation to review. Most felt that review participation positively influenced their careers through improving grantsmanship and exposure to new scientific ideas. Of those who reviewed, respondents reported dedicating 2-5% of their total annual work time to grant review and, based on their self-reported maximum review loads, it is estimated they are participating at 56-87% of their capacity, which may have important implications regarding the sustainability of the system. Overall, it is clear that participation in peer review is uneven and in some cases near capacity, and more needs to be done to create new motivations and incentives to increase the future pool of reviewers.

Negative life events and incident alcohol use disorders among ethnic minorities.

The association of negative life events (NLEs) with incident alcohol use disorders (AUDs) was examined among Blacks, Hispanics, and Whites in the second wave of the National Comorbidity Survey (NCS) among 3,679 participants without AUDs at the first wave. The number of past-year NLEs at NCS-2 was higher for Black than White participants, but the rates of incident AUDs did not differ by racial/ethnic group (14.2% among all participants). Past-year NLEs were associated with increased odds of incident AUDs for Whites and Hispanics but not Blacks. The implications of racial/ethnic differences in life events and AUDs are discussed.

Longitudinal outcomes associated with significant other responses to chronic fatigue and pain.

This study investigated significant others' behavior associated with fatigue, pain, and mental health outcomes among 68 individuals with chronic fatigue (43% also had fibromyalgia) over 18 months. More negative significant others' responses were associated with more pain, poorer physical and mental health, and more fatigue-related symptoms over time. More fibromyalgia tender points covaried with more solicitous significant others' responses over time. Better mental health covaried with more distracting significant others' responses over time. The results are discussed in terms of theoretical models of the role of perceived significant others' responses on patient outcomes and recommendations for future research.

The association of major life events with chronic fatigue.

OBJECTIVE: Major life events have been associated with the onset of unexplained chronic fatigue (CF) and with variability in illness course. The purpose of this study was to characterize the associations of major life events with illness status over time. METHOD: Sentinel measures of clinical status were collected four times over 18 months in a cohort of 99 persons with CF; participants also were interviewed regarding major life events, which were independently rated by observers. RESULTS: Over time, more major life events attributed to CF predicted a worse clinical course, suggesting that illness-related major life events rather than stressful life events in general are associated with worse clinical status for persons with CF. CONCLUSION: This study adds to the literature regarding specific types of life events' relationship to illness course, which has implications for the management of CF.

Moderators of Outcome in Problem-Solving Therapy for Depression in Primary Care.

OBJECTIVE: The purpose of this study was to conduct a secondary analysis of data from trials of problem-solving therapy in primary care (PST-PC) to examine differential outcomes by gender or race-ethnicity. METHODS: The participants were 352 patients with depression treated with PST-PC in multiple primary care sites in the United States. RESULTS: Women's depressive symptoms improved more over time than men's. Hopkins Symptom Checklist depression scale scores decreased significantly (t=-10.12, df=692, p<0.001) (estimate=-0.02, 95% confidence interval=-0.03, -0.02). Although patients from racial-ethnic minority groups had more depressive symptoms over time than patients from nonminority groups, there was no evidence of differential change by racial-ethnic group. CONCLUSIONS: These data add to the literature supporting the usefulness of PST-PC without evidence of differential effects for patients from racial-ethnic minority or nonminority groups.

Incident Chronic Spinal Pain and Depressive Disorders: Data From the National Comorbidity Survey.

This study examined pre-existing depression as a risk factor for the development of chronic spinal pain, and pre-existing chronic spinal pain as a risk factor for the development of depression. Data from the National Comorbidity Survey, a stratified sample of 5,001 participants evaluated in 1990 to 1992 (NCS-1) and again in 2000 to 2001 (NCS-2) were used to address these associations. Cox regression was used to estimate hazard ratios and time-to-incidence after NCS-1. Participants with antecedent acute or chronic depressive disorders at NCS-1 were more likely to develop chronic spinal pain in the ensuing 10 years compared with participants without depressive disorders. Those with antecedent chronic spinal pain at NCS-1 were more likely to develop dysthymic disorder than subjects without chronic spinal pain at NCS-1; however, antecedent chronic spinal pain was not associated the subsequent development of major depressive disorder. These results suggest that both pain and depression are associated with the development of the other condition. In particular, chronic depression is more strongly linked to chronic spinal pain than is acute depression. The results are discussed in terms of the need to assess the presence of both disorders given the presence of one. PERSPECTIVE: Chronic spinal pain and depressive disorders, especially chronic depression, increase the likelihood for the subsequent development of the other condition. The results underscore the need to routinely assess for the presence of both disorders given the presence of one to mitigate the effects of developing comorbid conditions.

Longitudinal associations of lymphocyte subsets with clinical outcomes in chronic fatigue syndrome.

BACKGROUND: Chronic fatigue syndrome (CFS) is characterized by prolonged fatigue and other physical and neurocognitive symptoms. Some studies suggest that CFS is accompanied by disruptions in the number and function of various lymphocytes. However, it is not clear which lymphocytes might influence CFS symptoms. PURPOSE: To determine if patient reported fatigue symptoms and physical functioning scores significantly changed across time with lymphocyte counts as evidence of a relation among chronic fatigue symptoms and the immune response. METHODS: The current longitudinal, naturalistic study assessed the cellular expression of three lymphocyte subtypes -- natural killer (NK) cells (CD3-CD16+ and CD3-CD56+) and naïve T cells (CD4+CD45RA+) -- to determine whether changes in lymphocytes at 4 time points across 18 months were associated with clinical outcomes, including CFS symptoms, physical functioning, and vitality, among patients with chronic fatigue.. Latent growth curve models were used to examine the longitudinal relationship between lymphocytes and clinical outcomes. RESULTS: Ninety-three patients with Fukuda-based CFS and seven with non-CFS fatigue provided study data. Results indicated that higher proportions of naïve T cells and lower proportions of NK cells were associated with worse physical functioning, whereas higher proportions of NK cells (CD3-CD16+) and lower proportions of naïve T cells were associated with fewer CFS symptoms. CONCLUSION: These findings suggest that lymphocytes are modestly related to clinical outcomes over time.

Psychologists in Academic Administration: A Call to Action and Service.

Academic psychologists' backgrounds may prepare them for many aspects of academic administration such as: understanding and working with people; prioritizing others' needs and institutional needs; and managing projects and budgets, e.g., for research grants or training programs. Contemporary academic health centers also may provide opportunities for psychologists to serve in academic health administration. This article encourages psychologists to consider preparing for and seeking administrative and higher-level leadership roles. Six psychologists serving diverse administrative roles-from vice chairs in medical school departments to presidents of universities with academic health centers-reflected on: their paths to administration; their preparation for administrative roles; and the commonalities and differences between the work and skills sets of psychologist health service providers and the work and skill sets required for higher level administrative and leadership roles.

Neurocognitive complaints and functional status among patients with chronic fatigue syndrome and fibromyalgia.

PURPOSE: The purpose of this study was to conduct a longitudinal examination of cognitive complaints and functional status in patients with chronic fatigue syndrome (CFS) alone and those who also had fibromyalgia (CFS/FM). METHODS: A total of 93 patients from a tertiary care fatigue clinic were evaluated on four occasions, each 6 months apart. Each evaluation included a tender point assessment, and self-reported functional status and cognitive complaints. RESULTS: Patients with CFS/FM reported significantly worse physical functioning, more bodily pain, and more cognitive difficulties (visuo-perceptual ability and verbal memory) than patients with CFS alone. Over time, bodily pain decreased only for participants with CFS alone. Verbal memory problems were associated with more bodily pain for both patient groups, whereas visuo-perceptual problems were associated with worse functional status for patients with CFS alone. CONCLUSIONS: This study adds to the literature on functional status, longitudinal course, and cognitive difficulties among patients with CFS and those with CFS and FM. The results suggest that patients with CFS/FM are more disabled, have more cognitive complaints, and improve more slowly over time than patients with CFS alone. Specific cognitive difficulties are related to worse functional status, which supports the addition of cognitive difficulties to the FM case criteria.