Clinical Simulation in the Regulation of Software as a Medical Device: An eDelphi Study.

BACKGROUND: Accelerated digitalization in the health sector requires the development of appropriate evaluation methods to ensure that digital health technologies (DHTs) are safe and effective. Software as a medical device (SaMD) is a commonly used DHT by clinicians to provide care to patients. Traditional research methods for evaluating health care products, such as randomized clinical trials, may not be suitable for DHTs, such as SaMD. However, evidence to show their safety and efficacy is needed by regulators before they can be used in practice. Clinical simulation can be used by researchers to test SaMD in an agile and low-cost way; yet, there is limited research on criteria to assess the robustness of simulations and, subsequently, their relevance for a regulatory decision. OBJECTIVE: The objective of this study was to gain consensus on the criteria that should be used to assess clinical simulation from a regulatory perspective when it is used to generate evidence for SaMD. METHODS: An eDelphi study approach was chosen to develop a set of criteria to assess clinical simulation when used to evaluate SaMD. Participants were recruited through purposive and snowball sampling based on their experience and knowledge in relevant sectors. They were guided through an initial scoping questionnaire with key themes identified from the literature to obtain a comprehensive list of criteria. Participants voted upon these criteria in 2 Delphi rounds, with criteria being excluded if consensus was not met. Participants were invited to add qualitative comments during rounds and qualitative analysis was performed on the comments gathered during the first round. Consensus was predefined by 2 criteria: if <10% of the panelists deemed the criteria as "not important" or "not important at all" and >60% "important" or "very important." RESULTS: In total, 33 international experts in the digital health field, including academics, regulators, policy makers, and industry representatives, completed both Delphi rounds, and 43 criteria gained consensus from the participants. The research team grouped these criteria into 7 domains-background and context, overall study design, study population, delivery of the simulation, fidelity, software and artificial intelligence, and study analysis. These 7 domains were formulated into the simulation for regulation of SaMD framework. There were key areas of concern identified by participants regarding the framework criteria, such as the importance of how simulation fidelity is achieved and reported and the avoidance of bias throughout all stages. CONCLUSIONS: This study proposes the simulation for regulation of SaMD framework, developed through an eDelphi consensus process, to evaluate clinical simulation when used to assess SaMD. Future research should prioritize the development of safe and effective SaMD, while implementing and refining the framework criteria to adapt to new challenges.

Challenges and solutions to system-wide use of precision oncology as the standard of care paradigm.

The personalised oncology paradigm remains challenging to deliver despite technological advances in genomics-based identification of actionable variants combined with the increasing focus of drug development on these specific targets. To ensure we continue to build concerted momentum to improve outcomes across all cancer types, financial, technological and operational barriers need to be addressed. For example, complete integration and certification of the 'molecular tumour board' into 'standard of care' ensures a unified clinical decision pathway that both counteracts fragmentation and is the cornerstone of evidence-based delivery inside and outside of a research setting. Generally, integrated delivery has been restricted to specific (common) cancer types either within major cancer centres or small regional networks. Here, we focus on solutions in real-world integration of genomics, pathology, surgery, oncological treatments, data from clinical source systems and analysis of whole-body imaging as digital data that can facilitate cost-effectiveness analysis, clinical trial recruitment, and outcome assessment. This urgent imperative for cancer also extends across the early diagnosis and adjuvant treatment interventions, individualised cancer vaccines, immune cell therapies, personalised synthetic lethal therapeutics and cancer screening and prevention. Oncology care systems worldwide require proactive step-changes in solutions that include inter-operative digital working that can solve patient centred challenges to ensure inclusive, quality, sustainable, fair and cost-effective adoption and efficient delivery. Here we highlight workforce, technical, clinical, regulatory and economic challenges that prevent the implementation of precision oncology at scale, and offer a systematic roadmap of integrated solutions for standard of care based on minimal essential digital tools. These include unified decision support tools, quality control, data flows within an ethical and legal data framework, training and certification, monitoring and feedback. Bridging the technical, operational, regulatory and economic gaps demands the joint actions from public and industry stakeholders across national and global boundaries.

Evolution of the clinical simulation approach to assess digital health technologies.

Coronavirus 2019 (COVID-19) has catalysed digital transformation in the health space. However, it remains a challenge to generate timely and cost-effective evidence for digital health technologies (DHTs) to ensure their safety and efficacy. Traditional methods, such as randomised controlled trials (RCTs), are ill-suited for assessing DHTs for reasons of speed, agility, cost and context. Clinical simulation using high-fidelity synthetic patient cases is emerging as a promising yet underexplored method to evaluate DHTs. It offers several advantages, including conducting remote multi-site testing at low cost, inclusion of high-risk patient profiles that are usually excluded from RCTs and adaptability to different local clinical settings. This article shares some of the insights from studies using clinical simulation conducted at the Institute of Global Health Innovation (IGHI) at Imperial College London and describes the evolution of this approach as well as future opportunities.

Digital Health: Today's Solutions and Tomorrow's Impact.

Artificial intelligence (AI) is becoming an indispensable tool to augment decision making in different health care settings and by various members of the patient pathway, including the patient. AI provides the ability to optimize data to bring clinical decision support for clinicians and laboratorians and/or empower patients to actively participate in their own health care. Though there are many examples of AI in health care, the exact role of AI and digital health solutions is still taking shape. Although AI will not replace the clinician, those who do not adopt AI may in time, be left behind.

Adherence to guidelines-recommended diagnostic testing was associated with overall survival in patients with diffuse large B-cell lymphoma after rituximab-based treatment: an observational cohort study.

PURPOSE: This study assessed the impact of adherence to guidelines-recommended diagnostic testing on treatment selection and overall survival (OS) in patients with diffuse large B-cell lymphoma (DLBCL) initiated on rituximab-based first line of treatment (1-LOT). METHODS: This retrospective cohort study used a nationwide electronic health record-derived de-identified database, including diagnostic testing information on immunohistochemistry (IHC), fluorescence in situ hybridization (FISH) and karyotype analysis that were abstracted from pathology reports or clinical visit notes, where available. The study included patients above 18 years old who were diagnosed with DLBCL between January 2011 and December 2019 and initiated on rituximab-based 1-LOT. Patients were classified into 'non-adherence,' 'partial-adherence' and 'complete-adherence' groups according to the evidence/documentation of a confirmed known result for IHC and molecular profiling tests (FISH and karyotyping) on a selection of the markers prior to the initiation of 1-LOT. Logistic regression was used to evaluate associations of adherence to diagnostic testing with 1-LOT between R-CHOP and other rituximab-based regimens. Median OS after the start of rituximab-based 1-LOT was calculated using the Kaplan-Meier method. Multivariable-adjusted Cox proportional hazards regression was used to assess the risk of all-cause death after initiation of 1-LOT by the degrees of adherence to guidelines-recommended diagnostic testing. RESULTS: In total, 3730 patients with DLBCL who initiated on rituximab-based 1-LOT were included. No association was found between adherence to guidelines-recommended diagnostic testing and treatment selection of 1-LOT for R-CHOP versus other rituximab-based regimens. Patients with a higher degree of adherence to guidelines-recommended diagnostic testing survived longer (median OS at 5.1, 6.9 and 7.1 years for 'non-adherence,' 'partial-adherence' and 'complete-adherence' groups, respectively [log-rank p < 0.001]) and had a decreased mortality risk (multivariable-adjusted hazard ratio with 95% confidence intervals at 0.83 [0.70-0.99] for 'partial-adherence' and 0.77 [0.64-0.91] for 'complete-adherence' groups, respectively). CONCLUSION: Patients' adherence to guidelines-recommended diagnostic testing were associated with better survival benefit, reinforcing the need for adoption of diagnostic testing guidelines in routine clinical care.

Development and validation of ACTE-MTB: A tool to systematically assess the maturity of molecular tumor boards.

Molecular tumor boards (MTBs) require specialized activities to leverage genomic data for therapeutic decision-making. Currently, there are no defined standards for implementing, executing, and tracking the impact of MTBs. This study describes the development and validation of ACTE-MTB, a tool to evaluate the maturity of an organization's MTB to identify specific areas that would benefit from process improvements and standardization. The ACTE-MTB maturity assessment tool is composed of 3 elements: 1) The ACTE-MTB maturity model; 2) a 59-question survey on MTB processes and challenges; and 3) a 5-level MTB maturity scoring algorithm. This tool was developed to measure MTB maturity in the categories of Access, Consultation, Technology, and Evidence (ACTE) and was tested on 20 MTBs spanning the United States, Europe, and Asia-Pacific regions. Validity testing revealed that the average maturity score was 3.3 out of 5 (+/- 0.1; range 2.0-4.3) with MTBs in academic institutions showing significantly higher overall maturity levels than in non-academic institutions (3.7 +/- 0.2 vs. 3.1 +/- 0.2; P = .018). While maturity scores for academic institutions were higher for Consultation, Technology, and Evidence domains, the maturity score for the Access domain did not significantly differ between the two groups, highlighting a disconnect between MTB operations and the downstream impact on ability to access testing and/or therapies. To our knowledge, ACTE-MTB is the first tool of its kind to enable structured, maturity assessment of MTBs in a universally-applicable manner. In the process of establishing construct validity of this tool, opportunities for further investigation and improvements were identified that address the key functional areas of MTBs that would likely benefit from standardization and best practice recommendations. We believe a unified approach to assessment of MTB maturity will help to identify areas for improvement at both the organizational and system level.

Evaluation of a clinical decision support tool for matching cancer patients to clinical trials using simulation-based research.

There is a growing need for alternative methodologies to evaluate digital health solutions in a short timeframe and at relatively low cost. Simulation-based research (SBR) methods have been proposed as an alternative methodology for evaluating digital health solutions; however, few studies have described the applicability of SBR methods to evaluate such solutions. This study used SBR to evaluate the feasibility and user experience of a clinical decision support (CDS) tool used for matching cancer patients to clinical trials. Twenty-five clinicians and research staff were recruited to match 10 synthetic patient cases to clinical trials using both the CDS tool and publicly available online trial databases. Participants were significantly more likely to report having sufficient time (p = 0.020) and to require less mental effort (p = 0.001) to complete trial matching with the CDS tool. Participants required less time for trial matching using the CDS tool, but the difference was not significant (p = 0.093). Most participants reported that they had sufficient guidance to participate in the simulations (96%). This study demonstrates the use of SBR methods is a feasible approach to evaluate digital health solutions and to collect valuable user feedback without the need for implementation in clinical practice. Further research is required to demonstrate the feasibility of using SBR to conduct remote evaluations of digital health solutions.

Development of a "meta-model" to address missing data, predict patient-specific cancer survival and provide a foundation for clinical decision support.

OBJECTIVE: Like most real-world data, electronic health record (EHR)-derived data from oncology patients typically exhibits wide interpatient variability in terms of available data elements. This interpatient variability leads to missing data and can present critical challenges in developing and implementing predictive models to underlie clinical decision support for patient-specific oncology care. Here, we sought to develop a novel ensemble approach to addressing missing data that we term the "meta-model" and apply the meta-model to patient-specific cancer prognosis. MATERIALS AND METHODS: Using real-world data, we developed a suite of individual random survival forest models to predict survival in patients with advanced lung cancer, colorectal cancer, and breast cancer. Individual models varied by the predictor data used. We combined models for each cancer type into a meta-model that predicted survival for each patient using a weighted mean of the individual models for which the patient had all requisite predictors. RESULTS: The meta-model significantly outperformed many of the individual models and performed similarly to the best performing individual models. Comparisons of the meta-model to a more traditional imputation-based method of addressing missing data supported the meta-model's utility. CONCLUSIONS: We developed a novel machine learning-based strategy to underlie clinical decision support and predict survival in cancer patients, despite missing data. The meta-model may more generally provide a tool for addressing missing data across a variety of clinical prediction problems. Moreover, the meta-model may address other challenges in clinical predictive modeling including model extensibility and integration of predictive algorithms trained across different institutions and datasets.

Challenges for the evaluation of digital health solutions-A call for innovative evidence generation approaches.

The field of digital health, and its meaning, has evolved rapidly over the last 20 years. For this article we followed the most recent definition provided by FDA in 2020. Emerging solutions offers tremendous potential to positively transform the healthcare sector. Despite the growing number of applications, however, the evolution of methodologies to perform timely, cost-effective and robust evaluations have not kept pace. It remains an industry-wide challenge to provide credible evidence, therefore, hindering wider adoption. Conventional methodologies, such as clinical trials, have seldom been applied and more pragmatic approaches are needed. In response, several academic centers such as researchers from the Institute of Global Health Innovation at Imperial College London have initiated a digital health clinical simulation test bed to explore new approaches for evidence gathering relevant to solution type and maturity. The aim of this article is to: (1) Review current research approaches and discuss their limitations; (2) Discuss challenges faced by different stakeholders in undertaking evaluations; and (3) Call for new approaches to facilitate the safe and responsible growth of the digital health sector.

Digital Tumor Board Solutions Have Significant Impact on Case Preparation.

PURPOSE: Multidisciplinary tumor boards (TBs) are the gold standard for decision-making in cancer care. Variability in preparation, conduction, and impact is widely reported. The benefit of digital technologies to support TBs is unknown. This study evaluated the impact of the NAVIFY Tumor Board solution (NTB) on TB preparation time across multiple user groups in 4 cancer categories: breast, GI, head and neck (ie, ear, nose, and throat, or ENT), and hematopathology. METHODS: This prospective study evaluated TB preparation time in multiple phases pre- and post-NTB implementation at an academic health care center. TB preparation times were recorded for multiple weeks using a digital time tracker. RESULTS: Preparation times for 59 breast, 61 GI, 36 ENT, and 71 hematopathology cancer TBs comparing a pre-NTB phase to 3 phases of NTB implementation were evaluated between February 2018 and July 2019. NTB resulted in significant reductions in overall preparation time (30%) across 3 TBs pre-NTB compared with the final post-NTB implementation phase. In the breast TB, NTB reduced overall preparation time by 28%, with a 76% decrease in standard deviation (SD). In the GI TB, a 23% reduction in average preparation time was observed for all users, with a 48% decrease in SD. In the ENT TB, a 33% reduction in average preparation time was observed for all users, with a 73% decrease in SD. The hematopathology TB, which was the cocreation partner and initial adopter of the solution, showed variable results. CONCLUSION: This study showed a significant impact of a digital solution on time preparation for TBs across multiple users and different TBs, reflecting the generalizability of the NTB. Adoption of such a solution could improve the efficiency of TBs and have a direct economic impact on hospitals.

A clinician's perspective on co-developing and co-implementing a digital tumor board solution.

Healthcare has entered the information age. This will deliver huge opportunities for healthcare providers to deliver more individualized treatments for patients, and as such improve outcomes. Nowhere is the prospect greater than in cancer care. Healthcare providers now need to manage the challenge of how to best capture, interpret and exploit insights from real-world clinical data. A significant aspect of cancer care is the challenge of preparing and conducting tumor boards. Currently, data are distributed across multiple systems and cannot be easily aggregated or integrated. In recognition that no suitable solution existed, the University of Missouri School of Medicine, in partnership with Roche, have co-developed and co-implemented a digital tumor board solution. This article describes the development process and the enablers and barriers for adoption from a clinician's perspective. In addition, it reflects on some of the key factors for success and some of the future opportunities.

Do International Health Partnerships contribute to reverse innovation? a mixed methods study of THET-supported partnerships in the UK.

BACKGROUND: International health partnerships (IHPs) are changing, with an increased emphasis on mutual accountability and joint agenda setting for both the high- and the low- or middle-income country (LMIC) partners. There is now an important focus on the bi-directionality of learning however for the UK partners, this typically focuses on learning at the individual level, through personal and professional development. We sought to evaluate whether this learning also takes the shape of 'Reverse Innovation' -when an idea conceived in a low-income country is subsequently adopted in a higher-income country. METHODS: This mixed methods study used an initial scoping survey of all the UK-leads of the Tropical Health Education Trust (THET)-supported International Health Partnerships (n = 114) to ascertain the extent to which the IHPs are or have been vehicles for Reverse Innovation. The survey formed the sampling frame for further deep-dive interviews to focus on volunteers' experiences and attitudes to learning from LMICs. Interviews of IHP leads (n = 12) were audio-recorded and transcribed verbatim. Survey data was analysed descriptively. Interview transcripts were coded thematically, using an inductive approach. RESULTS: Survey response rate was 27% (n = 34). The majority (70%) strongly agreed that supporting LMIC partners best described the mission of the partnership but only 13% of respondents strongly agreed that learning about new innovations and models was a primary mission of their partnership. Although more than half of respondents reported having observed innovative practice in the LMIC, only one IHP respondent indicated that this has led to Reverse Innovation. Interviews with a sample of survey respondents revealed themes primarily around how learning is conceptualised, but also a central power imbalance between the UK and LMIC partners. Paternalistic notions of knowledge could be traced to partnership power dynamics and latent attitudes to LMICs. CONCLUSIONS: Given the global flow of innovation, if High-income countries (HICs) are to benefit from LMIC practices, it is paramount to keep an open mind about where such learning can come from. Making the potential for learning more explicit and facilitating innovation dissemination upon return will ultimately underpin the success of adoption.

African healthcare innovation: An untapped resource?.

Healthcare systems across the world are in need of innovations that can achieve more with less, for more people. African nations have always been good partners for high-income institutions as they develop and test new ideas. However, they are now at the forefront of developing novel approaches to healthcare, grounded in community-centered approaches, but which also capitalize on the potential of digital healthcare. African healthcare leaders should be bold in realizing their own potential, and leaders from high-income countries must be open to innovations emerging from non-traditional sources.

Aetiology of acquired 'drop foot' deformity in Malawian children: a case series of 50 patients.

The objective of our study was to investigate the clinical presentation of patients with acquired foot drop in Malawi; the association of intramuscular injections and quinine administration; and the association between private sector clinics. A consecutive case series of 50 paediatric patients with foot drop was identified that represented 5% of all outpatient appointments during the study period. Ninety percent of all patients had received a gluteal intramuscular injection of quinine, of which 58% were treated in private clinics. This study presents the first description of the aetiology of acquired foot drop in Malawi and suggests that gluteal intramuscular injection of quinine is the leading cause.

The use of ionising radiation in trauma and orthopaedic theatres.

Ionising radiation is used extensively in trauma and orthopaedic perioperative practice for operative planning and intraoperative decision making. Radiation poses risks to patients and theatre staff, and it is important to always justify, optimise and limit the exposure. This is facilitated by a thorough understanding and awareness of radiation, as well as a sound knowledge of the precautions required for using this technology to minimise the risks of radiation.

Page kidney--a review of the literature.

Page kidney was first described in animal experiments in 1939. In the 1950s and 1960s the human counterpart became evident. In this review we examine the modest literature on this rare but important renal/urological complication, summarize the clinical features, and discuss the best approach to diagnosis and management.