Modelling clinical narrative as computable knowledge: The NICE computable implementation guidance project.

Introduction: Translating narrative clinical guidelines to computable knowledge is a long-standing challenge that has seen a diverse range of approaches. The UK National Institute for Health and Care Excellence (NICE) Content Advisory Board (CAB) aims ultimately to (1) guide clinical decision support and other software developers to increase traceability, fidelity and consistency in supporting clinical use of NICE recommendations, (2) guide local practice audit and intervention to reduce unwarranted variation, (3) provide feedback to NICE on how future recommendations should be developed. Objectives: The first phase of work was to explore a range of technical approaches to transition NICE toward the production of natively digital content. Methods: Following an initial 'collaborathon' in November 2022, the NICE Computable Implementation Guidance project (NCIG) was established. We held a series of workstream calls approximately fortnightly, focusing on (1) user stories and trigger events, (2) information model and definitions, (3) horizon-scanning and output format. A second collaborathon was held in March 2023 to consolidate progress across the workstreams and agree residual actions to complete. Results: While we initially focussed on technical implementation standards, we decided that an intermediate logical model was a more achievable first step in the journey from narrative to fully computable representation. NCIG adopted the WHO Digital Adaptation Kit (DAK) as a technology-agnostic method to model user scenarios, personae, processes and workflow, core data elements and decision-support logic. Further work will address indicators, such as prescribing compliance, and implementation in document templates for primary care patient record systems. Conclusions: The project has shown that the WHO DAK, with some modification, is a promising approach to build technology-neutral logical specifications of NICE recommendations. Implementation of concurrent computable modelling by multidisciplinary teams during guideline development poses methodological and cultural questions that are complex but tractable given suitable will and leadership.

Meta-research evaluating redundancy and use of systematic reviews when planning new studies in health research: a scoping review.

BACKGROUND: Several studies have documented the production of wasteful research, defined as research of no scientific importance and/or not meeting societal needs. We argue that this redundancy in research may to a large degree be due to the lack of a systematic evaluation of the best available evidence and/or of studies assessing societal needs. OBJECTIVES: The aim of this scoping review is to (A) identify meta-research studies evaluating if redundancy is present within biomedical research, and if so, assessing the prevalence of such redundancy, and (B) to identify meta-research studies evaluating if researchers had been trying to minimise or avoid redundancy. ELIGIBILITY CRITERIA: Meta-research studies (empirical studies) were eligible if they evaluated whether redundancy was present and to what degree; whether health researchers referred to all earlier similar studies when justifying and designing a new study and/or when placing new results in the context of earlier similar trials; and whether health researchers systematically and transparently considered end users' perspectives when justifying and designing a new study. SOURCES OF EVIDENCE: The initial overall search was conducted in MEDLINE, Embase via Ovid, CINAHL, Web of Science, Social Sciences Citation Index, Arts & Humanities Citation Index, and the Cochrane Methodology Register from inception to June 2015. A 2nd search included MEDLINE and Embase via Ovid and covered January 2015 to 26 May 2021. No publication date or language restrictions were applied. CHARTING METHODS: Charting methods included description of the included studies, bibliometric mapping, and presentation of possible research gaps in the identified meta-research. RESULTS: We identified 69 meta-research studies. Thirty-four (49%) of these evaluated the prevalence of redundancy and 42 (61%) studies evaluated the prevalence of a systematic and transparent use of earlier similar studies when justifying and designing new studies, and/or when placing new results in context, with seven (10%) studies addressing both aspects. Only one (1%) study assessed if the perspectives of end users had been used to inform the justification and design of a new study. Among the included meta-research studies evaluating whether redundancy was present, only two of nine health domains (medical areas) and only two of 10 research topics (different methodological types) were represented. Similarly, among the included meta-research studies evaluating whether researchers had been trying to minimise or avoid redundancy, only one of nine health domains and only one of 10 research topics were represented. CONCLUSIONS THAT RELATE TO THE REVIEW QUESTIONS AND OBJECTIVES: Even with 69 included meta-research studies, there was a lack of information for most health domains and research topics. However, as most included studies were evaluating across different domains, there is a clear indication of a high prevalence of redundancy and a low prevalence of trying to minimise or avoid redundancy. In addition, only one meta-research study evaluated whether the perspectives of end users were used to inform the justification and design of a new study. SYSTEMATIC REVIEW REGISTRATION: Protocol registered at Open Science Framework: https://osf.io/3rdua/ (15 June 2021).

A New Era of National Guideline Development in Saudi Arabia.

Saudi Arabia's ambitious Vision 2030 project was launched in 2016 as a strategy for economic development and national growth, with 11 Vision Realization Programs put in charge of its implementation. The backbone of its Transformation Program for the Health Sector has been the definition of a new Model of Care aiming to deliver 42 coordinated interventions across 6 Systems of Care, with the development of clinical guidelines identified as a key cross-cutting intervention to foster the use of national, evidence-based practices across KSA, reduce care variation, and promote accountable care. This article provides an overview of the history, progress to date, and future outlook of the recently initiated National Guidelines Center in Saudi Arabia, established in collaboration between the Health Holding Company and the Saudi Health Council represented by its National Center for Evidence-based Medicine. The lessons learnt from previous guideline initiatives are grouped under the Center's design principles of high quality, relevance, practical implementation, and sustainability. Aspects setting the project apart from previous endeavors have been its focus on extensive engagement with key stakeholders in the Saudi guideline ecosystem, the co-development of evidence-based recommendations with aligned key performance measures, and the implementation of guideline recommendations in the clinical workflow via integrated electronic order sets. Nine activity streams aim to enable the Center to take its place among the leading regional and global guideline developing organizations and to optimally support clinicians and patients, Saudi Arabia's health sector transformation, and the work of guideline communities worldwide.

Justification of research using systematic reviews continues to be inconsistent in clinical health science-A systematic review and meta-analysis of meta-research studies.

BACKGROUND: Redundancy is an unethical, unscientific, and costly challenge in clinical health research. There is a high risk of redundancy when existing evidence is not used to justify the research question when a new study is initiated. Therefore, the aim of this study was to synthesize meta-research studies evaluating if and how authors of clinical health research studies use systematic reviews when initiating a new study. METHODS: Seven electronic bibliographic databases were searched (final search June 2021). Meta-research studies assessing the use of systematic reviews when justifying new clinical health studies were included. Screening and data extraction were performed by two reviewers independently. The primary outcome was defined as the percentage of original studies within the included meta-research studies using systematic reviews of previous studies to justify a new study. Results were synthesized narratively and quantitatively using a random-effects meta-analysis. The protocol has been registered in Open Science Framework (https://osf.io/nw7ch/). RESULTS: Twenty-one meta-research studies were included, representing 3,621 original studies or protocols. Nineteen of the 21 studies were included in the meta-analysis. The included studies represented different disciplines and exhibited wide variability both in how the use of previous systematic reviews was assessed, and in how this was reported. The use of systematic reviews to justify new studies varied from 16% to 87%. The mean percentage of original studies using systematic reviews to justify their study was 42% (95% CI: 36% to 48%). CONCLUSION: Justification of new studies in clinical health research using systematic reviews is highly variable, and fewer than half of new clinical studies in health science were justified using a systematic review. Research redundancy is a challenge for clinical health researchers, as well as for funders, ethics committees, and journals.

Systematic reviews are rarely used to contextualise new results-a systematic review and meta-analysis of meta-research studies.

BACKGROUND: Results of new studies should be interpreted in the context of what is already known to compare results and build the state of the science. This systematic review and meta-analysis aimed to identify and synthesise results from meta-research studies examining if original studies within health use systematic reviews to place their results in the context of earlier, similar studies. METHODS: We searched MEDLINE (OVID), EMBASE (OVID), and the Cochrane Methodology Register for meta-research studies reporting the use of systematic reviews to place results of original clinical studies in the context of existing studies. The primary outcome was the percentage of original studies included in the meta-research studies using systematic reviews or meta-analyses placing new results in the context of existing studies. Two reviewers independently performed screening and data extraction. Data were synthesised using narrative synthesis and a random-effects meta-analysis was performed to estimate the mean proportion of original studies placing their results in the context of earlier studies. The protocol was registered in Open Science Framework. RESULTS: We included 15 meta-research studies, representing 1724 original studies. The mean percentage of original studies within these meta-research studies placing their results in the context of existing studies was 30.7% (95% CI [23.8%, 37.6%], I2=87.4%). Only one of the meta-research studies integrated results in a meta-analysis, while four integrated their results within a systematic review; the remaining cited or referred to a systematic review. The results of this systematic review are characterised by a high degree of heterogeneity and should be interpreted cautiously. CONCLUSION: Our systematic review demonstrates a low rate of and great variability in using systematic reviews to place new results in the context of existing studies. On average, one third of the original studies contextualised their results. Improvement is still needed in researchers' use of prior research systematically and transparently-also known as the use of an evidence-based research approach, to contribute to the accumulation of new evidence on which future studies should be based. SYSTEMATIC REVIEW REGISTRATION: Open Science registration number https://osf.io/8gkzu/.

Systematic reviews are rarely used to inform study design - a systematic review and meta-analysis.

OBJECTIVE: Our aim was to identify and synthesize the results from meta-research studies to determine whether and how authors of original studies in clinical health research use systematic reviews when designing new studies. STUDY DESIGN AND SETTING: For this systematic review, we searched MEDLINE (OVID), Embase (OVID) and the Cochrane Methodology Register. We included meta-research studies and primary outcome was the percentage of original studies using systematic reviews to design their study. Risk of bias was assessed using an ad hoc created list of ten items. The results are presented both as a narrative synthesis and a meta-analysis. RESULTS: Sixteen studies were included. The use of a systematic review to inform the design of new clinical studies varied between 0% and 73%, with a mean percentage of 17%. The number of components of the design in which information from previous systematic reviews was used varied from three to 11. CONCLUSION: Clinical health research is characterized by variability regarding the extent to which systematic reviews are used to guide the design. An evidence-based research (EBR) approach towards research design when new clinical health studies are designed is necessary to decrease potential research redundancy and increase end-user value.

How to improve the study design of clinical trials in internal medicine: recent advances in the evidence­based methodology.

Meta-research has highlighted that up to half of all clinical studies may be redundant and do not add any value. We suggest that such unnecessary studies will continue to be prepared and published unless researchers systematically and transparently identify and consider the existing evidence. This approach of identifying and utilizing the existing knowledge base before and after conducting a new trial is called Evidence-Based Research (EBR), defined as the use of prior research in a systematic and transparent way to inform a new study so that it is answering questions that matter in a valid, efficient, and accessible manner. This paper describes the issues that have led to the development of the EBR approach, suggests what researchers should do to avoid wasteful and unnecessary research, and outlines the benefits of conducting evidence-based research. Finally, we present the international EBR Network established to support the efforts to minimize waste in research and increase the value of clinical studies.

Evidence-Based Research Series-Paper 2 : Using an Evidence-Based Research approach before a new study is conducted to ensure value.

BACKGROUND AND OBJECTIVES: There is considerable actual and potential waste in research. The aim of this article is to describe how using an evidence-based research approach before conducting a study helps to ensure that the new study truly adds value. STUDY DESIGN AND SETTING: Evidence-based research is the use of prior research in a systematic and transparent way to inform a new study so that it is answering questions that matter in a valid, efficient, and accessible manner. In this second article of the evidence-based research series, we describe how to apply an evidence-based research approach before starting a new study. RESULTS: Before a new study is performed, researchers need to provide a solid justification for it using the available scientific knowledge as well as the perspectives of end users. The key method for both is to conduct a systematic review of earlier relevant studies. CONCLUSION: Describing the ideal process illuminates the challenges and opportunities offered through the suggested evidence-based research approach. A systematic and transparent approach is needed to provide justification for and to optimally design a relevant and necessary new study.

Evidence-Based Research Series-Paper 3: Using an Evidence-Based Research approach to place your results into context after the study is performed to ensure usefulness of the conclusion.

BACKGROUND AND OBJECTIVE: There is considerable actual and potential waste in research. Using evidence-based research (EBR) can ensure the value of a new study. The aim of this article, the third in a series, is to describe an EBR approach to putting research results into context. STUDY DESIGN AND SETTING: EBR is the use of prior research in a systematic and transparent way to inform a new study so that it is answering questions that matter in a valid, efficient, and accessible manner. In this third and final article of a series, we describe how to use the context of existing evidence to reach and present a trustworthy and useful conclusion when reporting results from a new clinical study. RESULTS: We describe a method, the EBR approach, that by using a systematic and transparent consideration of earlier similar studies when interpreting and presenting results from a new original study will ensure usefulness of the conclusion. CONCLUSION: Using an EBR approach will improve the usefulness of a clinical study by providing the context to draw more valid conclusions and explicit information about new research needs.

Evidence-Based Research Series-Paper 1: What Evidence-Based Research is and why is it important?

OBJECTIVES: There is considerable actual and potential waste in research. Evidence-based research ensures worthwhile and valuable research. The aim of this series, which this article introduces, is to describe the evidence-based research approach. STUDY DESIGN AND SETTING: In this first article of a three-article series, we introduce the evidence-based research approach. Evidence-based research is the use of prior research in a systematic and transparent way to inform a new study so that it is answering questions that matter in a valid, efficient, and accessible manner. RESULTS: We describe evidence-based research and provide an overview of the approach of systematically and transparently using previous research before starting a new study to justify and design the new study (article #2 in series) and-on study completion-place its results in the context with what is already known (article #3 in series). CONCLUSION: This series introduces evidence-based research as an approach to minimize unnecessary and irrelevant clinical health research that is unscientific, wasteful, and unethical.

A framework for crafting clinical practice guidelines that are relevant to the care and management of people with multimorbidity.

Many patients of all ages have multiple conditions, yet clinicians often lack explicit guidance on how to approach clinical decision-making for such people. Most recommendations from clinical practice guidelines (CPGs) focus on the management of single diseases, and may be harmful or impractical for patients with multimorbidity. A major barrier to the development of guidance for people with multimorbidity stems from the fact that the evidence underlying CPGs derives from studies predominantly focused on the management of a single disease. In this paper, the investigators from the Improving Guidelines for Multimorbid Patients Study Group present consensus-based recommendations for guideline developers to make guidelines more useful for the care of people with multimorbidity. In an iterative process informed by review of key literature and experience, we drafted a list of issues and possible approaches for addressing important coexisting conditions in each step of the guideline development process, with a focus on considering relevant interactions between the conditions, their treatments and their outcomes. The recommended approaches address consideration of coexisting conditions at all major steps in CPG development, from nominating and scoping the topic, commissioning the work group, refining key questions, ranking importance of outcomes, conducting systematic reviews, assessing quality of evidence and applicability, summarizing benefits and harms, to formulating recommendations and grading their strength. The list of issues and recommendations was reviewed and refined iteratively by stakeholders. This framework acknowledges the challenges faced by CPG developers who must make complex judgments in the absence of high-quality or direct evidence. These recommendations require validation through implementation, evaluation and refinement.