How to strengthen a health research system: WHO's review, whose literature and who is providing leadership?

BACKGROUND: Health research is important for the achievement of the Sustainable Development Goals. However, there are many challenges facing health research, including securing sufficient funds, building capacity, producing research findings and using both local and global evidence, and avoiding waste. A WHO initiative addressed these challenges by developing a conceptual framework with four functions to guide the development of national health research systems. Despite some progress, more is needed before health research systems can meet their full potential of improving health systems. The WHO Regional Office for Europe commissioned an evidence synthesis of the systems-level literature. This Opinion piece considers its findings before reflecting on the vast additional literature available on the range of specific health research system functions related to the various challenges. Finally, it considers who should lead research system strengthening. MAIN TEXT: The evidence synthesis identifies two main approaches for strengthening national health research systems, namely implementing comprehensive and coherent strategies and participation in partnerships. The literature describing these approaches at the systems level also provides data on ways to strengthen each of the four functions of governance, securing financing, capacity-building, and production and use of research. Countries effectively implementing strategies include England, Ireland and Rwanda, whereas West Africa experienced effective partnerships. Recommended policy approaches for system strengthening are context specific. The vast literature on each function and the ever-growing evidence-base are illustrated by considering papers in just one key journal, Health Research Policy and Systems, and analysing the contribution of two national studies. A review of the functions of the Iranian system identifies over 200 relevant and mostly national records; an analysis of the creation of the English National Institute for Health Research describes the key leadership role played by the health department. Furthermore, WHO is playing leadership roles in helping coordinate partnerships within and across health research systems that have been attempting to tackle the COVID-19 crisis. CONCLUSIONS: The evidence synthesis provides a firm basis for decision-making by policy-makers and research leaders looking to strengthen national health research systems within their own national context. It identifies five crucial policy approaches - conducting situation analysis, sustaining a comprehensive strategy, engaging stakeholders, evaluating impacts on health systems, and partnership participation. The vast and ever-growing additional literature could provide further perspectives, including on crucial leadership roles for health ministries.

Envisioning and shaping translation of knowledge into action: A comparative case-study of stakeholder engagement in the development of a European tobacco control tool.

Stakeholder engagement in health policy research is often said to increase 'research impact', but the active role of stakeholders in creating impact remains underexplored. We explored how stakeholders shaped the translation of health policy research into action. Our comparative case-study tracked a European research project that aimed to transfer an existing tobacco control return on investment tool. That project also aimed to increase its impact by engaging with stakeholders in further developing the tool. We conducted semi-structured interviews, using an actor-scenario mapping approach. Actor-scenarios can be seen as relational descriptions of a future world. We mapped the scenarios by asking stakeholders to describe who and what would play a role in the tool's utilisation. Our results show that stakeholders envisioned disparate futures for the tool. Some scenarios were specific, whereas most were generic projections of abstract potential users and responsibilities. We show how stakeholders mobilised elements of context, such as legislative support and agricultural practice, that would affect the tool's use. We conclude that stakeholders shape knowledge translation processes by continuously putting forth explicit or implicit scenarios about the future. Mapping actor-scenarios may help in aligning knowledge production with utilisation. Insights into potential roles and responsibilities could be fed back in research projects with the aim of increasing the likelihood that the study results may be used.

Tracing the indirect societal impacts of biomedical research: development and piloting of a technique based on citations.

There is growing interest in assessing the societal impacts of research such as informing health policies and clinical practice, and contributing to improved health. Bibliometric approaches have long been used to assess knowledge outputs, but can they also help evaluate societal impacts? We aimed to see how far the societal impacts could be traced by identifying key research articles in the psychiatry/neuroscience area and exploring their societal impact through analysing several generations of citing papers. Informed by a literature review of citation categorisation, we developed a prototype template to qualitatively assess a reference's importance to the citing paper and tested it on 96 papers. We refined the template for a pilot study to assess the importance of citations, including self-cites, to four key research articles. We then similarly assessed citations to those citing papers for which the key article was Central i.e. it was very important to the message of the citing article. We applied a filter of three or more citation occasions in order to focus on the citing articles where the reference was most likely to be Central. We found the reference was Central for 4.4 % of citing research articles overall and ten times more frequently if the article contained three or more citation occasions. We created a citation stream of influence for each key paper across up to five generations of citations. We searched the Web of Science for citations to all Central papers and identified societal impacts, including international clinical guidelines citing papers across the generations.

Evaluation of the impact of National Breast Cancer Foundation-funded research.

OBJECTIVE: To evaluate the impact of the National Breast Cancer Foundation's (NBCF's) research investment. DESIGN AND PARTICIPANTS: Surveys based on the Payback Framework were sent to chief investigators involved in research funded by the NBCF during 1995-2012; a bibliometric analysis of NBCF-funded publications in 2006-2010 was conducted; and a purposive, stratified sample of case studies was obtained. MAIN OUTCOME MEASURES: Research impact on knowledge production, the research system, informing policy, product development and broader health and economic benefits. RESULTS: Of 242 surveys sent, 153 (63%) were returned. The average impact of journals in which NBCF publications appeared was double that of world publications. Seventy surveys (46%) reported career progression, and 185 higher degrees were obtained or expected, including 121 PhDs. One hundred and one grants (66%) produced tools that built capacity across the research system, and research teams leveraged an additional $1.40 in funding for every dollar invested. Fifteen applied grants and one basic grant impacted on policy. Ten basic and four applied grants led to the development of drugs, prognostic tools or diagnostic technologies. Twenty applied and two basic grants led to changes in practice and behaviour of health care staff, consumers and the public, with further impacts anticipated. Case studies provided illustrations of high impact. CONCLUSIONS: NBCF's strategy of investing in a mixed portfolio of research areas and mechanisms encouraged a broad range of impacts across all Payback categories. The impacts from basic research tended to focus on knowledge production and drug development; while applied research generated greater impacts within the other Payback categories. The funding of shared infrastructure stimulated impact across the research system.

Conducting retrospective impact analysis to inform a medical research charity's funding strategies: the case of Asthma UK.

BACKGROUND: Debate is intensifying about how to assess the full range of impacts from medical research. Complexity increases when assessing the diverse funding streams of funders such as Asthma UK, a charitable patient organisation supporting medical research to benefit people with asthma. This paper aims to describe the various impacts identified from a range of Asthma UK research, and explore how Asthma UK utilised the characteristics of successful funding approaches to inform future research strategies. METHODS: We adapted the Payback Framework, using it both in a survey and to help structure interviews, documentary analysis, and case studies. We sent surveys to 153 lead researchers of projects, plus 10 past research fellows, and also conducted 14 detailed case studies. These covered nine projects and two fellowships, in addition to the innovative case studies on the professorial chairs (funded since 1988) and the MRC-Asthma UK Centre in Allergic Mechanisms of Asthma (the 'Centre') which together facilitated a comprehensive analysis of the whole funding portfolio. We organised each case study to capture whatever academic and wider societal impacts (or payback) might have arisen given the diverse timescales, size of funding involved, and extent to which Asthma UK funding contributed to the impacts. RESULTS: Projects recorded an average of four peer-reviewed journal articles. Together the chairs reported over 500 papers. All streams of funding attracted follow-on funding. Each of the various categories of societal impacts arose from only a minority of individual projects and fellowships. Some of the research portfolio is influencing asthma-related clinical guidelines, and some contributing to product development. The latter includes potentially major breakthroughs in asthma therapies (in immunotherapy, and new inhaled drugs) trialled by university spin-out companies. Such research-informed guidelines and medicines can, in turn, contribute to health improvements. The role of the chairs and the pioneering collaborative Centre is shown as being particularly important. CONCLUSIONS: We systematically demonstrate that all types of Asthma UK's research funding assessed are making impacts at different levels, but the main societal impacts from projects and fellowships come from a minority of those funded. Asthma UK used the study's findings, especially in relation to the Centre, to inform research funding strategies to promote the achievement of impact.

Tracing the wider impacts of biomedical research: a literature search to develop a novel citation categorisation technique.

There is an increasing need both to understand the translation of biomedical research into improved healthcare and to assess the range of wider impacts from health research such as improved health policies, health practices and healthcare. Conducting such assessments is complex and new methods are being sought. Our new approach involves several steps. First, we developed a qualitative citation analysis technique to apply to biomedical research in order to assess the contribution that individual papers made to further research. Second, using this method, we then proposed to trace the citations to the original research through a series of generations of citing papers. Third, we aimed eventually to assess the wider impacts of the various generations. This article describes our comprehensive literature search to inform the new technique. We searched various databases, specific bibliometrics journals and the bibliographies of key papers. After excluding irrelevant papers we reviewed those remaining for either general or specific details that could inform development of our new technique. Various characteristics of citations were identified that had been found to predict their importance to the citing paper including the citation's location; number of citation occasions and whether the author(s) of the cited paper were named within the citing paper. We combined these objective characteristics with subjective approaches also identified from the literature search to develop a citation categorisation technique that would allow us to achieve the first of the steps above, i.e., being able routinely to assess the contribution that individual papers make to further research.

The role of the national general medical journal: surveys of which journals UK clinicians read to inform their clinical practice / Papel de las revistas nacionales de medicina general: encuesta sobre qué revistas leen los médicos del Reino Unido para informarse sobre la práctica clínica

Background: For biomedical research findings to contribute toward health gains they must reach clinicians. Academic journals have historically been considered important information sources. Birken and Parkin found seven journals to most consistently contain the best pediatric evidence and, of these seven, four were general medical journals. Methods: We surveyed clinicians in three UK medical specialties (psychiatry, surgery and pediatrics), asking which journals they read and which they considered important to inform their clinical practice. Results: The readership of general medical journals, in comparison to specialty and sub-specialty journals, is widespread across the three UK medical specialties, although the importance of general medical journals varies widely. The BMJ is the most prominent general medical journal in terms of readership and importance but a dominant specialty or sub-specialty journal was usually more important for most groups. The Lancet is less widely read and less important, although more academics than non-academics consider it important. Conclusions: Overall, key general medical journals play an important role. Journal availability and cost, particularly in relation to membership for UK clinicians, and the position of academics and non-academics have to be considered in any analysis. Three of the four general medical journals containing the best pediatric evidence were found to be widely read by UK pediatricians and two UK-based general medical journals, the BMJ and The Lancet, were also considered important in our survey. Further investigation of the reasons for the importance of a journal and studies that would allow international comparisons would provide greater input to the discussion (AU)

Fundamento: Para que los hallazgos de la investigación biomédica contribuyan a mejorar la salud, es preciso que lleguen a los médicos. Históricamente, las revistas académicas se han considerado fuentes de información importante. Birken y Parkin encontraron que siete revistas contienen con más regularidad la mejor evidencia pediátrica y, de ellas, cuatro son de medicina general. Métodos: Encuestamos a médicos de tres especialidades del Reino Unido (psiquiatría, cirugía y pediatría), formulándoles preguntas sobre qué revistas leen y qué consideran importante para informarse sobre la práctica clínica. Resultados: En el Reino Unido, en comparación con las revistas de especialidades y subespecialidades, la lectura de revistas de medicina general es difundida a través de las tres especialidades citadas, aunque la importancia de estas revistas varía ampliamente. El British Medical Journal es la más destacada por lo que respecta a lectura e importancia, pero, en general, para los tres grupos de médicos es más importante una revista de especialidad o subespecialidad predominante. La lectura de Lancet es menos difundida y se considera menos importante, aunque un mayor número de académicos que de no acadé- micos la consideran importante. Conclusiones: En conjunto, las revistas de medicina general clave desempeñan un papel relevante. En cualquier análisis, es preciso considerar la disponibilidad y el coste de la revista, en particular en relación con el hecho de ser miembro de la sociedad o asociación para médicos del Reino Unido. Se encontró que los pediatras de este país leen ampliamente tres de las cuatro revistas de medicina general que contienen la mejor evidencia pediátrica, y, en la encuesta, también se consideraron importantes dos revistas de medicina general del RU, BMJ y Lancet. Una investigación adicional de las razones de la importancia de una revista y estudios que permitieran comparaciones internacionales proporcionarían mayor información para una discusión (AU)

The role of the national general medical journal: surveys of which journals UK clinicians read to inform their clinical practice.

BACKGROUND: For biomedical research findings to contribute toward health gains they must reach clinicians. Academic journals have historically been considered important information sources. Birken and Parkin found seven journals to most consistently contain the best pediatric evidence and, of these seven, four were general medical journals. METHODS: We surveyed clinicians in three UK medical specialties (psychiatry, surgery and pediatrics), asking which journals they read and which they considered important to inform their clinical practice. RESULTS: The readership of general medical journals, in comparison to specialty and sub-specialty journals, is widespread across the three UK medical specialties, although the importance of general medical journals varies widely. The BMJ is the most prominent general medical journal in terms of readership and importance but a dominant specialty or sub-specialty journal was usually more important for most groups. The Lancet is less widely read and less important, although more academics than non-academics consider it important. CONCLUSIONS: Overall, key general medical journals play an important role. Journal availability and cost, particularly in relation to membership for UK clinicians, and the position of academics and non-academics have to be considered in any analysis. Three of the four general medical journals containing the best pediatric evidence were found to be widely read by UK pediatricians and two UK-based general medical journals, the BMJ and The Lancet, were also considered important in our survey. Further investigation of the reasons for the importance of a journal and studies that would allow international comparisons would provide greater input to the discussion.

The information sources and journals consulted or read by UK paediatricians to inform their clinical practice and those which they consider important: a questionnaire survey.

BACKGROUND: Implementation of health research findings is important for medicine to be evidence-based. Previous studies have found variation in the information sources thought to be of greatest importance to clinicians but publication in peer-reviewed journals is the traditional route for dissemination of research findings. There is debate about whether the impact made on clinicians should be considered as part of the evaluation of research outputs. We aimed to determine first which information sources are generally most consulted by paediatricians to inform their clinical practice, and which sources they considered most important, and second, how many and which peer-reviewed journals they read. METHODS: We inquired, by questionnaire survey, about the information sources and academic journals that UK medical paediatric specialists generally consulted, attended or read and considered important to their clinical practice. RESULTS: The same three information sources--professional meetings & conferences, peer-reviewed journals and medical colleagues--were, overall, the most consulted or attended and ranked the most important. No one information source was found to be of greatest importance to all groups of paediatricians. Journals were widely read by all groups, but the proportion ranking them first in importance as an information source ranged from 10% to 46%. The number of journals read varied between the groups, but Archives of Disease in Childhood and BMJ were the most read journals in all groups. Six out of the seven journals previously identified as containing best paediatric evidence are the most widely read overall by UK paediatricians, however, only the two most prominent are widely read by those based in the community. CONCLUSION: No one information source is dominant, therefore a variety of approaches to Continuing Professional Development and the dissemination of research findings to paediatricians should be used. Journals are an important information source. A small number of key ones can be identified and such analysis could provide valuable additional input into the evaluation of clinical research outputs.

The journals of importance to UK clinicians: a questionnaire survey of surgeons.

BACKGROUND: Peer-reviewed journals are seen as a major vehicle in the transmission of research findings to clinicians. Perspectives on the importance of individual journals vary and the use of impact factors to assess research is criticised. Other surveys of clinicians suggest a few key journals within a specialty, and sub-specialties, are widely read. Journals with high impact factors are not always widely read or perceived as important. In order to determine whether UK surgeons consider peer-reviewed journals to be important information sources and which journals they read and consider important to inform their clinical practice, we conducted a postal questionnaire survey and then compared the findings with those from a survey of US surgeons. METHODS: A questionnaire survey sent to 2,660 UK surgeons asked which information sources they considered to be important and which peer-reviewed journals they read, and perceived as important, to inform their clinical practice. Comparisons were made with numbers of UK NHS-funded surgery publications, journal impact factors and other similar surveys. RESULTS: Peer-reviewed journals were considered to be the second most important information source for UK surgeons. A mode of four journals read was found with academics reading more than non-academics. Two journals, the BMJ and the Annals of the Royal College of Surgeons of England, are prominent across all sub-specialties and others within sub-specialties. The British Journal of Surgery plays a key role within three sub-specialties. UK journals are generally preferred and readership patterns are influenced by membership journals. Some of the journals viewed by surgeons as being most important, for example the Annals of the Royal College of Surgeons of England, do not have high impact factors. CONCLUSION: Combining the findings from this study with comparable studies highlights the importance of national journals and of membership journals. Our study also illustrates the complexity of the link between the impact factors of journals and the importance of the journals to clinicians. This analysis potentially provides an additional basis on which to assess the role of different journals, and the published output from research.