Progressive and degenerative journals: on the growth and appraisal of knowledge in scholarly publishing.

Despite continued attention, finding adequate criteria for distinguishing "good" from "bad" scholarly journals remains an elusive goal. In this essay, I propose a solution informed by the work of Imre Lakatos and his methodology of scientific research programmes (MSRP). I begin by reviewing several notable attempts at appraising journal quality - focusing primarily on the impact factor and development of journal blacklists and whitelists. In doing so, I note their limitations and link their overarching goals to those found within the philosophy of science. I argue that Lakatos's MSRP and specifically his classifications of "progressive" and "degenerative" research programmes can be analogized and repurposed for the evaluation of scholarly journals. I argue that this alternative framework resolves some of the limitations discussed above and offers a more considered evaluation of journal quality - one that helps account for the historical evolution of journal-level publication practices and attendant contributions to the growth (or stunting) of scholarly knowledge. By doing so, the seeming problem of journal demarcation is diminished. In the process I utilize two novel tools (the mistake index and scite index) to further illustrate and operationalize aspects of the MSRP.

Causal and Associational Language in Observational Health Research: A Systematic Evaluation.

We estimated the degree to which language used in the high-profile medical/public health/epidemiology literature implied causality using language linking exposures to outcomes and action recommendations; examined disconnects between language and recommendations; identified the most common linking phrases; and estimated how strongly linking phrases imply causality. We searched for and screened 1,170 articles from 18 high-profile journals (65 per journal) published from 2010-2019. Based on written framing and systematic guidance, 3 reviewers rated the degree of causality implied in abstracts and full text for exposure/outcome linking language and action recommendations. Reviewers rated the causal implication of exposure/outcome linking language as none (no causal implication) in 13.8%, weak in 34.2%, moderate in 33.2%, and strong in 18.7% of abstracts. The implied causality of action recommendations was higher than the implied causality of linking sentences for 44.5% or commensurate for 40.3% of articles. The most common linking word in abstracts was "associate" (45.7%). Reviewers' ratings of linking word roots were highly heterogeneous; over half of reviewers rated "association" as having at least some causal implication. This research undercuts the assumption that avoiding "causal" words leads to clarity of interpretation in medical research.

Assessing the impact of predatory journals on policy and guidance documents: a cross-sectional study protocol.

INTRODUCTION: Many predatory journals fail to follow best publication practices. Studies assessing the impact of predatory journals have focused on how these articles are cited in reputable academic journals. However, it is possible that research from predatory journals is cited beyond the academic literature in policy documents and guidelines. Given that research used to inform public policy or government guidelines has the potential for widespread impact, we will examine whether predatory journals have penetrated public policy. METHODS AND ANALYSIS: This is a descriptive study with no hypothesis testing. Policy documents that cite work from the known predatory publisher OMICS will be downloaded from the Overton database. Overton collects policy documents from over 1200 sources worldwide. Policy documents will be evaluated to determine how the predatory journal article is used. We will also extract epidemiological details of the policy documents, including: who funded their development, the discipline the work is relevant to and the name of the organisations producing the policy. The record of scholarly citations of the identified predatory articles will also be examined. Findings will be reported with descriptive statistics using counts and percentages. ETHICS AND DISSEMINATION: No ethical approval was required for this study since it does not involve human or animal research. Study findings will be discussed at workshops on journalology and predatory publishing and will be disseminated through preprint, peer-reviewed literature and conference presentations.

A credit-like rating system to determine the legitimacy of scientific journals and publishers.

The predatory nature of a journal is in constant debate because it depends on multiple factors, which keep evolving. The classification of a journal as being predatory, or not, is no longer exclusively associated with its open access status, by inclusion or exclusion on perceived reputable academic indexes and/or on whitelists or blacklists. Inclusion in the latter may itself be determined by a host of criteria, may be riddled with type I errors (e.g., erroneous inclusion of a truly predatory journal in a whitelist) and/or type II errors (e.g., erroneous exclusion of a truly valid scholarly journal in a whitelist). While extreme cases of predatory publishing behavior may be clear cut, with true predatory journals displaying ample predatory properties, journals in non-binary grey zones of predatory criteria are difficult to classify. They may have some legitimate properties, but also some illegitimate ones. In such cases, it might be too extreme to refer to such entities as "predatory". Simply referring to them as "potentially predatory" or "borderline predatory" also does little justice to discern a predatory entity from an unscholarly, low-quality, unprofessional, or exploitative one. Faced with the limitations caused by this gradient of predatory dimensionality, this paper introduces a novel credit-like rating system, based in part on well-known financial credit ratings companies used to assess investment risk and creditworthiness, to assess journal or publisher quality. Cognizant of the weaknesses and criticisms of these rating systems, we suggest their use as a new way to view the scholarly nature of a journal or publisher. When used as a tool to supplement, replace, or reinforce current sets of criteria used for whitelists and blacklists, this system may provide a fresh perspective to gain a better understanding of predatory publishing behavior. Our tool does not propose to offer a definitive solution to this problem.

Beyond the p-value: Bayesian Statistics and Causation.

Statistical paradigms limit the perspective and tools social work researchers use to study the world and answer questions impacting people and policy. Currently, quantitative social work researchers overwhelmingly rely on the frequentist paradigm of statistics. This paper discusses foundational differences between the frequentist and Bayesian statistical paradigms, describes basic concepts of Bayesian analysis, compares Bayesian and frequentist statistical analysis for a sample social work problem, and introduces two types of causal analyses built on Bayesian statistical thinking: counterfactual causality, and causality based on work by computer scientist Judea Pearl. Implications for social work research are discussed.