What impact has the Centre of Research Excellence in Digestive Health made in the field of gastrointestinal health in Australia and internationally? Study protocol for impact evaluation using the FAIT framework.

INTRODUCTION: The need for public research funding to be more accountable and demonstrate impact beyond typical academic outputs is increasing. This is particularly challenging and the science behind this form of research is in its infancy when applied to collaborative research funding such as that provided by the Australian National Health and Medical Research Council to the Centre for Research Excellence in Digestive Health (CRE-DH). METHODS AND ANALYSIS: In this paper, we describe the protocol for applying the Framework to Assess the Impact from Translational health research to the CRE-DH. The study design involves a five-stage sequential mixed-method approach. In phase I, we developed an impact programme logic model to map the pathway to impact and establish key domains of benefit such as knowledge advancement, capacity building, clinical implementation, policy and legislation, community and economic impacts. In phase 2, we have identified and selected appropriate, measurable and timely impact indicators for each of these domains and established a data plan to capture the necessary data. Phase 3 will develop a model for cost-consequence analysis and identification of relevant data for microcosting and valuation of consequences. In phase 4, we will determine selected case studies to include in the narrative whereas phase 5 involves collation, data analysis and completion of the reporting of impact.We expect this impact evaluation to comprehensively describe the contribution of the CRE-DH for intentional activity over the CRE-DH lifespan and beyond to improve outcomes for people suffering with chronic and debilitating digestive disorders. ETHICS AND DISSEMINATION: This impact evaluation study has been registered with the Hunter New England Human Research Ethics Committee as project 2024/PID00336 and ethics application 2024/ETH00290. Results of this study will be disseminated via medical conferences, peer-reviewed publications, policy submissions, direct communication with relevant stakeholders, media and social media channels such as X (formely Twitter).

Impact assessment of the medical practice assisting (MPA) program in general practice in the hunter New England and central coast regions of Australia.

BACKGROUND: A regional Australian Primary Health Network (PHN) has been subsidising administrative staff from local general practices to undertake the Medical Practice Assisting (MPA) course as part of its MPA Program. The MPA Program aimed to upskill administrative staff to undertake clinical tasks and fill in for busy or absent Practice Nurses (PNs), freeing up PNs to increase revenue-generating activity, avoiding casual replacement staff wages, and increasing patient throughput. An impact assessment was undertaken to evaluate the impact and estimate the economic costs of the MPA program to the PHN, general practices, and students to inform future uptake of the intervention. METHODS: The Framework to Assess the Impact of Translational Health Research (FAIT) was utilised. Originally designed to assess the impact of health research, this was its first application to a health services project. FAIT combines three validated methods of impact assessment-Payback, economic analysis and narratives underpinned by a program logic model. Quantified metrics describe the impacts of the program within various "domains of benefit", the economic model costs the intervention and monetises potential consequences, and the narrative tells the story of the MPA Program and the difference it has made. Data were collected via online surveys from general practitioners (GPs), PNs, practice managers; MPA graduates and PHN staff were interviewed by phone and on Zoom. RESULTS: FAIT was effective in evidencing the impacts and economic viability of the MPA Program. GPs and PNs reported greater work satisfaction, PNs reported less stress and reduced workloads and MPA graduates reported higher job satisfaction and greater confidence performing a range of clinical skills. MPA Program economic costs for general practices during candidature, and 12 month post-graduation was estimated at $69,756. With effective re-integration planning, this investment was recoverable within 12 months through increased revenue for practices. Graduates paid appropriately for their new skills also recouped their investment within 24 months. CONCLUSION: Utilisation of MPA graduates varied substantially between practices and COVID-19 impacted on their utilisation. More strategic reintegration of the MPA graduate back into the practice to most effectively utilise their new skillset could optimise potential benefits realised by participating practices.

Network evaluation of an innovation platform in continuous quality improvement in Australian Indigenous primary healthcare.

BACKGROUND: From 2014 to 2019, the Centre for Research Excellence in Integrated Quality Improvement (CRE-IQI) was evaluated as an innovation platform focusing on continuous quality improvement in Indigenous Australian primary healthcare. Although social network analysis (SNA) is a recognized method for evaluating the functioning, collaboration and effectiveness of innovation platforms, applied research is limited. This study applies SNA to evaluate the CRE-IQI's functioning as an innovation platform. METHODS: Two surveys (2017, 2019) were conducted using social survey and network methods. Survey items covered respondent characteristics, their perceptions of the CRE-IQI's performance, and its impact and sociometric relationships. Members' relationship information was captured for the CRE-IQI at three time points, namely start (retrospectively), midpoint and final year, on three network types (knew, shared information, collaborated). SNA software was used to compute standard network metrics including diameter, density and centrality, and to develop visualizations. Survey and network results were addressed in a workshop held by members to develop improvement strategies. RESULTS: The response rate was 80% in 2017 and 65% in 2019 (n = 49 and 47, respectively). Between 2017 and 2019, respondents' mean ratings of the CRE-IQI's functioning and achievements in meeting its goals were sustained. They perceived the CRE-IQI as multidisciplinary, having effective management and governance, and incorporating Indigenous research leadership, representation and ways of working. Respondents recognized high levels of trust amongst members, rated "good communication and coordination with participants" highly, and "facilitating collaboration" as the CRE's most strongly recognized achievement. In collaboration and information-sharing networks, average path length remained low in 2017 and 2019, indicating good small-world network properties for relaying information. On average, respondents shared information and collaborated with more CRE members in 2017 than 2019. However, in both 2017 and 2019 there were new collaborations and information-sharing outside of direct collaborations. CRE-IQI outcomes included: evidence generation; knowledge transfer and skills development in quality improvement; research capacity-building, career development; mentoring; grant support; development of new projects; health service support; and policy impact. CONCLUSIONS: This study shows the utility of network analysis in evaluating the functioning, and collaboration, at the individual, organizational and health system levels, of an innovation platform, and adds to our understanding of factors enabling successful innovation platforms.

What was the impact of a participatory research project in Australian Indigenous primary healthcare services? Applying a comprehensive framework for assessing translational health research to Lessons for the Best.

OBJECTIVES: To (1) apply the Framework to Assess the Impact from Translational health research (FAIT) to Lessons from the Best to Better the Rest (LFTB), (2) report on impacts from LFTB and (3) assess the feasibility and outcomes from a retrospective application of FAIT. SETTING: Three Indigenous primary healthcare (PHC) centres in the Northern Territory, Australia; project coordinating centre distributed between Townsville, Darwin and Cairns and the broader LFTB learning community across Australia. PARTICIPANTS: LFTB research team and one representative from each PHC centre. PRIMARY AND SECONDARY OUTCOME MEASURES: Impact reported as (1) quantitative metrics within domains of benefit using a modified Payback Framework, (2) a cost-consequence analysis given a return on investment was not appropriate and (3) a narrative incorporating qualitative evidence of impact. Data were gathered through in-depth stakeholder interviews and a review of project documentation, outputs and relevant websites. RESULTS: LFTB contributed to knowledge advancement in Indigenous PHC service delivery; enhanced existing capacity of health centre staff, researchers and health service users; enhanced supportive networks for quality improvement; and used a strengths-based approach highly valued by health centres. LFTB also leveraged between $A1.4 and $A1.6 million for the subsequent Leveraging Effective Ambulatory Practice (LEAP) Project to apply LFTB learnings to resource development and creation of a learning community to empower striving PHC centres. CONCLUSION: Retrospective application of FAIT to LFTB, although not ideal, was feasible. Prospective application would have allowed Indigenous community perspectives to be included. Greater appreciation of the full benefit of LFTB including a measure of return on investment will be possible when LEAP is complete. Future assessments of impact need to account for the limitations of fully capturing impact when intermediate/final impacts have not yet been realised and captured.