Argument mining as rapid screening tool of COVID-19 literature quality: Preliminary evidence.

Background: The COVID-19 pandemic prompted the scientific community to share timely evidence, also in the form of pre-printed papers, not peer reviewed yet. Purpose: To develop an artificial intelligence system for the analysis of the scientific literature by leveraging on recent developments in the field of Argument Mining. Methodology: Scientific quality criteria were borrowed from two selected Cochrane systematic reviews. Four independent reviewers gave a blind evaluation on a 1-5 scale to 40 papers for each review. These scores were matched with the automatic analysis performed by an AM system named MARGOT, which detected claims and supporting evidence for the cited papers. Outcomes were evaluated with inter-rater indices (Cohen's Kappa, Krippendorff's Alpha, s* statistics). Results: MARGOT performs differently on the two selected Cochrane reviews: the inter-rater indices show a fair-to-moderate agreement of the most relevant MARGOT metrics both with Cochrane and the skilled interval scores, with larger values for one of the two reviews. Discussion and conclusions: The noted discrepancy could rely on a limitation of the MARGOT system that can be improved; yet, the level of agreement between human reviewers also suggests a different complexity between the two reviews in debating controversial arguments. These preliminary results encourage to expand and deepen the investigation to other topics and a larger number of highly specialized reviewers, to reduce uncertainty in the evaluation process, thus supporting the retraining of AM systems.

What Kinds of Dogs Are Used in Clinical and Experimental Research?

BACKGROUND: Dogs are widely used in research to answer questions about canine or human conditions. For the latter, research dogs are often used as models, since they are physiologically more similar to humans than other species used in research and they share similar environmental conditions. From a veterinary perspective, research findings are widely based on academic research, and thus are generated under experimental conditions. In that regard, the question arises: do the dogs used for research adequately represent the dog population seen in veterinary practice? It may, for example, be assumed that Beagle dogs are often used as experimental animals. The objective of this study was to evaluate the signalment of dogs used in veterinary research. Furthermore, we aimed to assess other relevant criteria regarding the validity of clinical trials in the context of six different veterinary medicine specialties: cardiology, internal medicine, neurology, orthopaedics, reproduction, and surgery. METHODS: A literature search was conducted and 25 studies per specialty were randomly selected. The breed, sex, neuter status, median age, and median weight of the dogs used for clinical studies (n = 150) published between 2007 and 2019 were evaluated. RESULTS: In total, 596,542 dogs were used in the 150 trials. Breed information was given for 33,835 of these dogs (5.7%). Of the latter, 1.9% were Beagles. Nine clinical trials exclusively used Beagles. The most frequently used breeds were German Shepherds (7.3%), Labrador Retrievers (6.7%), and Golden Retrievers (4.7%). The major reporting deficits found were missing breed specification in 25.3% of the articles; missing information about the sex of the dogs in 16.2%; missing age and weight information in 22.7 and 32.7%, respectively; and missing neuter status in 38.7% of the clinical studies. The median sample size was 56 (Q1:29; Q3:365) dogs. CONCLUSIONS: The presented project revealed that Beagle dogs represent only a small proportion of dogs in veterinary research. Based on the evaluated publications, it seems that some relevant dog attributes differ between the specialties. The results, however, show deficits in the reporting of demographic data for the dogs. The need for an improvement in the documentation and/or reporting of animal signalment is obvious and should be addressed by authors, reviewers, and journal editors in the future.

Study design quality of research on dogs published in peer-reviewed journals.

BACKGROUND: In the past it has been criticised that only a low proportion of well-designed and well-reported studies in some medical specialities is available. The objective of this study was to systematically evaluate the quality of literature about canine medicine published in peer-reviewed journals in relation to six specific veterinary medicine specialities. METHODS: A literature search was conducted and 25 studies per speciality were selected. The quality of the articles (n = 150) published between 2007 and 2019 was evaluated with a validated checklist. RESULTS: In articles related to all specialities, deficits were found, such as not adequate number of animals in 60.0% of the studies. In 88.0%, information about housing and feeding of the dogs were not specified. In 69.4% of the prospective clinical studies, an ethical approval was reported, and written informed consent of the owners was obtained in 46.2%. CONCLUSIONS: The findings revealed extensive deficits in the design and reporting of studies in canine medicine. The demand for improvement is obvious and should be addressed by authors, reviewers and journal editors in the future. Our results underline that practitioners should critically appraise the quality of literature before implementing information into practice.

Quality evaluation of clinical trial literatures on the COVID-19 treatment with traditional Chinese medicine (TCM) / 药学实践杂志

Objective To evaluate the quality of the clinical trial literatures published on the treatment of COVID-19 with TCM, find out the shortcomings and put forward corresponding suggestions, in order to promote TCM against COVID-19. Methods "COVID-19", "New Coronavirus Pneumonia", "TCM", "Chinese Medicine", "Clinical Trial", "Lianhua Qingwen", "Huoxiang Zhengqi", "Jinhua Qinggan", and other keywords were used to search relevant literatures in CNKI and PubMed database. Among the all the screened relevant literatures on the treatment of COVID-19 with TCM, the literature quality was assessed according to evaluation criteria of clinical trial literatures. Results A total of 463 papers related to the treatment of COVID-19 with TCM were obtained. 440 papers on theoretical research on the network pharmacology mechanism of Chinese medicine treatment of COVID-19 were excluded. Among the 23 articles included in the quality analysis, there are 3 randomized controlled studies, 1 multi-center prospective randomized controlled trial, 2 disease case report trials, and 5 uncontrolled single arm studies, 5 controlled trials and 7 retrospective studies. In the period of more than half a year, although many clinical trial documents of TCM for the treatment of COVID-19 have been published, the clinical trial design reflected in most TCM clinical trials were not standardized. There are problems in "randomness and rationality", such as no control group, no randomization design, only case studies, no blinding method in controlling bias, and insufficient objectivity in the evaluation criteria of efficacy. All of these need to be improved. Conclusion The treatment of new coronavirus pneumonia with TCM still requires more and standardized clinical trial verifications and publications to generate strong evidence-based results, such as adding control groups, increasing sample size, and using blinding methods to increase the credibility of clinical trials.

[Literature quality analysis of RCTs regarding acupuncture for chronic neck pain].

The internationally-accepted Consolidated Standards for Reporting of Trials (CONSORT) and Standards for Reporting Interventions in Controlled Trials of Acupuncture (STRICTA) were applied to evaluate the literature quality of randomized controlled trials (RCTs) regarding acupuncture for chronic neck pain in past 10 years. The literature of RCTs regarding acupuncture for chronic neck pain was searched by computer; the English literature was searched in PubMed and EMbase, while the Chinese literature was searched in CNKI, Wanfang database, VIP database and China Biomedical Literature Database. The literature published from January 2008 to January 2018 was searched. As a result, 29 Chinese articles and 10 English articles were included. According to CONSORT, among Chinese articles, 28 articles (96.6%) described baseline data, 23 articles (79.3%) described randomization, 0 articles (0.0%) described allocation concealment, 3 articles (10.3%) described blind method; among English articles, 6 articles (60.0%) described baseline data, 8 articles (80.0%) described randomization, 8 articles (80.0%) described allocation concealment, and 7 articles (70.0%) described blind method. According to STRICTA, among Chinese articles, 8 articles (27.6%) described needle instrument selection, 18 articles (62.1%) described needle depth, 24 articles (82.8%) described needling sensation, and 0 articles (0.0%) described acupuncturist' qualifications; among English articles, 5 articles (50.0%) described needle instrument selection, 8 articles (80.0%) described needle depth, 3 articles (30.0%) described needling sensation, and 4 articles (40.0%) described acupuncturist' qualifications. In conclusion, the reporting of acupuncture details in Chinese literature is superior to that in English literature, while the reporting of trial design in English literature is slightly superior to that in Chinese literature. Moreover, both Chinese and English literature need to further improve clinical trial design to improve the reporting quality of clinical evidence based on CONSORT and STRICTA.

Literature quality analysis of RCTs regarding acupuncture for chronic neck pain / 中国针灸

The internationally-accepted Consolidated Standards for Reporting of Trials (CONSORT) and Standards for Reporting Interventions in Controlled Trials of Acupuncture (STRICTA) were applied to evaluate the literature quality of randomized controlled trials (RCTs) regarding acupuncture for chronic neck pain in past 10 years. The literature of RCTs regarding acupuncture for chronic neck pain was searched by computer; the English literature was searched in PubMed and EMbase, while the Chinese literature was searched in CNKI, Wanfang database, VIP database and China Biomedical Literature Database. The literature published from January 2008 to January 2018 was searched. As a result, 29 Chinese articles and 10 English articles were included. According to CONSORT, among Chinese articles, 28 articles (96.6%) described baseline data, 23 articles (79.3%) described randomization, 0 articles (0.0%) described allocation concealment, 3 articles (10.3%) described blind method; among English articles, 6 articles (60.0%) described baseline data, 8 articles (80.0%) described randomization, 8 articles (80.0%) described allocation concealment, and 7 articles (70.0%) described blind method. According to STRICTA, among Chinese articles, 8 articles (27.6%) described needle instrument selection, 18 articles (62.1%) described needle depth, 24 articles (82.8%) described needling sensation, and 0 articles (0.0%) described acupuncturist' qualifications; among English articles, 5 articles (50.0%) described needle instrument selection, 8 articles (80.0%) described needle depth, 3 articles (30.0%) described needling sensation, and 4 articles (40.0%) described acupuncturist' qualifications. In conclusion, the reporting of acupuncture details in Chinese literature is superior to that in English literature, while the reporting of trial design in English literature is slightly superior to that in Chinese literature. Moreover, both Chinese and English literature need to further improve clinical trial design to improve the reporting quality of clinical evidence based on CONSORT and STRICTA.

Quality Evaluation of Domestic Pharmacoeconomic Literatures Based on PEERs in Recent Years / 中国药房

OBJECTIVE： To investigate current status and quality of domestic pharmacoeconomic literatures， and to provide reference for the standardization of pharmacoeconomic research. METHODS： Retrieved from CNKI， Wanfang， VIP and other database， the pharmacoeconomic literatures published from Jan. 2017 to May 2018 were collected. The qualities of literatures were evaluated with Guidelines for Quality Evaluation of Pharmacoeconomics Evaluation Reports （“PEERs” for short）. RESULTS： Totally 160 domestic pharmacoeconomic research literatures were included. The results of PEERs evaluation showed of which the coincidence rate was 32.5％ （52/160）. The literatureswhich were in full compliance （the report had a certain reference value） accounted for 1.3％ （2/160）， which were in basic compliance （the report had certain reference value after being revised） accounted for 31.3％（50/160），which were in non-conformity （the report did not had reference value） accounted for 67.5％（108/160）. Domestic pharmacoeconomic researches were of high quality in terms of research object， evaluation method and content，research purpose，  research design and design type， etc.； but the researches were of low quality in terms of research angle， incremental cost/incremental output analysis， sensitivity analysis and other aspects， and there was no explanation or unclear elaboration. CONCLUSIONS： The quality of domestic pharmacoeconomic research literatures are uneven， and their research quality needs to be further improved. It is recommended to standardize the evaluation of pharmacoeconomics， making the evaluation of pharmacoeconomics more scientific and objective.

Systematic Evaluation and GRADE Rating of Fire Needling Treatment for Rheumatoid Arthritis / 上海针灸杂志

Objective To assess the quality of literature on randomized or semirandomized clinical controlled trials of fire needling treatment for rheumatoid arthritis.Method Acomputer search assisted by a manual search was conducted of CBM (1979—2015), CNKI (1979—2015), VIP (1989—2015),Wanfang digital periodical group(1998—2015), PubMed (1966—2015), EMbase (1980—2015) and Cochrane Library (Issue4, 2015).Randomizedor semir and omized clinical controlled trials were included.Two reviewers extracted data independently and checked them each other.The methodological quality of research design, diagnostic and inclusion/exclusion criteria, intervening measures for fire needling/control group,efficacy evaluation criteriaand observation time points, and adverse reaction reporting were assessed according toCochrane Reviewers' Handbook 4.2.8.Ameta-analysis was made of homogeneity studies.The GRADE approach ratedsystem outcomes.ResultA total of sevenr and omized controlled trialswere included.Four included studies made a comparison between fire needling andmedication (analgesic+ antirheumatic) groups andthere was no statistically significant difference in the efficacy rate [RR=1.22, 95%CI(0.97, 1.52)].GRADE ratingwas very low.Internationally accepted diagnostic criteria and general evaluation methodswere still not used universally.Conclusion The current clinical reports show that fire needling treatment has some effect on rheumatoid arthritis, but it still needs related evidence support.

Evaluation of randomized clinical trial papers published in medical sci-tech journals depending on 2010 CONSORT statement / 中华医学科研管理杂志

Objective To explore the quality of clinical trial papers published in medical SCI-tech journal.Methods A total of 1201 original research projects were published in 6 medical journals from Mar.2012 to Mar.2014 in China.Fifty-four of them were selected according to the key words randomizedcontrolled,and they were evaluated using 2010 CONSORT statement.Results In 54 articles,the section of background and abstract were described clearly and standardize.Fifty-one (94.44 ％)papers described intervention method in details.Thirty-five (64.81％) papers described the experimental design method.Thirty-one (57.41％) papers defined the primary and secondary outcome indicators,only 3 (5.56％)of them described distribution order,and the method of blindness.Four (7.41％) papers provides the information on sample size calculation.The using of flow chart accounted for 7.41％.Ten (18.52％) papers described the reason of drop out.Two (3.70％) papers evaluated the effect and accuracy,and the trial results were explained clearly and consider other evidences.Only 18 (33.33％) papers wrote the limitation of the studies.For other information of the studies,5 (9.26％) of them was registered somewhere,the information on funding support was on 20 (37.04％)papers.Conclusions The description on most RCT papers were clear and complete,but close attention should be paid to the method of blindness and sample size determine.

Evaluation on Acupuncture Treatment of Primary Insomnia / 针刺研究

Acupuncture treatment of insomnia has been reported for many years,but its validity remains controversial up to now.In the present review,the authors took "Acupuncture","Insomnia" as the key words to do a literature search from databases in both Chinese and English via computer retrieval system and also conducted a hand-made retrieval from journals of Chinese medicine as Chinese Acu-moxibustion,and collections of theses of the related academic conferences.Then,they made a systemic evaluation on the original articles from methodology,diagnosis,criteria for the enlisted and excluded patients,intervention methods of acupuncture and control groups,standards for assessing the curative efficacy,follow-up visit,etc.Of the retrieved 80 clinical articles,20 are RCT(randomized controlled trails) composed of 1907 cases,including 4 high-quality trails and 16 lower-quality trails.Regarding the assessment of the therapeutic effect,measuring scales are often adopted in overseas studies,while in domestic researches,self-drawn standards are frequently used.In conclusion,there have had no high-quality clinical trails about acupuncture treatment of primary insomnia in China at the present,and the related evaluating methods could not definitely confirm the efficacy of acupuncture in relieving insomnia.Therefore,a strict and scientific clinical trail scheme being in line with evidence-based medicine is urgently needed in the coming studies on acupuncture treatment of primary insomnia.

Evaluation of Quality of Literature of Clinical Studies on Acupuncture and Moxibustion Treatment of Hyperthyroidism / 中医杂志

Objective:To evaluate quality of clinical trials about acupuncture and moxibustion treatment of hyperthyrodism in China. Methods:Literature of clinical trials of acupuncture and moxibustion treatment of hyperthyroidism were attained by computer and manual retrieval,and randomization,control,selection of study objects,comparability between groups,amount of samples, description and control of source of samples,criteria for diagnosis,application of blind method,criteria for assessment of therapeutic effects,statistical method,adverse effects,follow-up,and others in the literature were analyzed and evaluated by the randomized controlled test criteria and the clinical assessing method in Cochrane handbook of international Cochrane cooperation net.Results:In 63 papers,clinical randomized controlled tests of acupuncture and moxibustion treatment of hyperthyroidism only accounted for 22.2%,but with increasing tend year by year in recent years.Conclusion:Though acupuncture and moxibustion have been widely applied in prevention and treatment of hyperthyroidism,it could not provide evidences of higher reliability for clinical treatment due to less clinical randomized controlled tests and lower quality,which severely hinder testing and verifying of clinical therapeutic effects of acupuncture and moxibustion.It is proposed that multiple central and randomized controlled test should be made,so as to search for feasible acupuncture and moxibustion methods with definite therapeutic effect for hyperthyroidism,and provide basis for further systematical evaluation of acupuncture and moxibustion treatment of hyperthyroidism.