Measuring scientific impact beyond academia: An assessment of existing impact metrics and proposed improvements.

How does scientific research affect the world around us? Being able to answer this question is of great importance in order to appropriately channel efforts and resources in science. The impact by scientists in academia is currently measured by citation based metrics such as h-index, i-index and citation counts. These academic metrics aim to represent the dissemination of knowledge among scientists rather than the impact of the research on the wider world. In this work we are interested in measuring scientific impact beyond academia, on the economy, society, health and legislation (comprehensive impact). Indeed scientists are asked to demonstrate evidence of such comprehensive impact by authoring case studies in the context of the Research Excellence Framework (REF). We first investigate the extent to which existing citation based metrics can be indicative of comprehensive impact. We have collected all recent REF impact case studies from 2014 and we have linked these to papers in citation networks that we constructed and derived from CiteSeerX, arXiv and PubMed Central using a number of text processing and information retrieval techniques. We have demonstrated that existing citation-based metrics for impact measurement do not correlate well with REF impact results. We also consider metrics of online attention surrounding scientific works, such as those provided by the Altmetric API. We argue that in order to be able to evaluate wider non-academic impact we need to mine information from a much wider set of resources, including social media posts, press releases, news articles and political debates stemming from academic work. We also provide our data as a free and reusable collection for further analysis, including the PubMed citation network and the correspondence between REF case studies, grant applications and the academic literature.

Portable abdominal CT: analysis of quality and clinical impact in more than 100 consecutive cases.

OBJECTIVE: Our aim was to determine portable abdominal CT image quality and clinical content in a consecutive series of scans obtained in ICUs at a single center. MATERIALS AND METHODS: All helical portable abdominal CT scans obtained between June 1999 and December 2000 were reviewed by two observers. Image quality was assessed for feature detection and overall quality compared with the patients' contemporaneous stationary CT scans. Hospital records were used to determine patient demographics, scanning indications, and clinical impact and to verify portable CT findings when possible. RESULTS: One hundred twenty-two helical portable CT scans (47 contrast-enhanced) and 41 contemporaneous stationary CT scans in 107 patients were included. IV contrast material improved portable CT scan quality, but quality scores for portable CT scans were consistently lower than those for stationary CT scans, both with and without contrast material. Thirty-three conditions suspected before scanning were supported by findings on portable CT scans, which detected evidence of infection in 18 and hemorrhage in 16 cases and motivated seven laparotomies and six percutaneous drainage procedures. Thirty-three portable CT scans (27%) contributed to a change in patient treatment. Results of surgery or autopsy confirmed portable CT findings in 12 of 17 cases. CONCLUSION: Although image quality is inferior to conventional stationary CT, portable abdominal CT provides important diagnostic information without requiring patient transport outside the ICU. Radiologists should avoid overconfident interpretation of portable CT scans.