AI Quality Standards in Health Care: Rapid Umbrella Review.

BACKGROUND: In recent years, there has been an upwelling of artificial intelligence (AI) studies in the health care literature. During this period, there has been an increasing number of proposed standards to evaluate the quality of health care AI studies. OBJECTIVE: This rapid umbrella review examines the use of AI quality standards in a sample of health care AI systematic review articles published over a 36-month period. METHODS: We used a modified version of the Joanna Briggs Institute umbrella review method. Our rapid approach was informed by the practical guide by Tricco and colleagues for conducting rapid reviews. Our search was focused on the MEDLINE database supplemented with Google Scholar. The inclusion criteria were English-language systematic reviews regardless of review type, with mention of AI and health in the abstract, published during a 36-month period. For the synthesis, we summarized the AI quality standards used and issues noted in these reviews drawing on a set of published health care AI standards, harmonized the terms used, and offered guidance to improve the quality of future health care AI studies. RESULTS: We selected 33 review articles published between 2020 and 2022 in our synthesis. The reviews covered a wide range of objectives, topics, settings, designs, and results. Over 60 AI approaches across different domains were identified with varying levels of detail spanning different AI life cycle stages, making comparisons difficult. Health care AI quality standards were applied in only 39% (13/33) of the reviews and in 14% (25/178) of the original studies from the reviews examined, mostly to appraise their methodological or reporting quality. Only a handful mentioned the transparency, explainability, trustworthiness, ethics, and privacy aspects. A total of 23 AI quality standard-related issues were identified in the reviews. There was a recognized need to standardize the planning, conduct, and reporting of health care AI studies and address their broader societal, ethical, and regulatory implications. CONCLUSIONS: Despite the growing number of AI standards to assess the quality of health care AI studies, they are seldom applied in practice. With increasing desire to adopt AI in different health topics, domains, and settings, practitioners and researchers must stay abreast of and adapt to the evolving landscape of health care AI quality standards and apply these standards to improve the quality of their AI studies.

Exploring the Impact of In Basket Metrics on the Adoption of a New Electronic Health Record System Among Specialists in a Tertiary Hospital in Alberta: Descriptive Study.

BACKGROUND: Health care organizations implement electronic health record (EHR) systems with the expectation of improved patient care and enhanced provider performance. However, while these technologies hold the potential to create improved care and system efficiencies, they can also lead to unintended negative consequences, such as patient safety issues, communication problems, and provider burnout. OBJECTIVE: This study aims to document metrics related to the In Basket communication hub (time in In Basket per day, time in In Basket per appointment, In Basket messages received per day, and turnaround time) of the EHR system implemented by Alberta Health Services, the province-wide health delivery system called Connect Care (Epic Systems). The objective was to identify how a newly implemented EHR system was used, the timing of its use, and the duration of use specifically related to In Basket activities. METHODS: A descriptive study was conducted. Due to the diversity of specialties, the providers were grouped into medical and surgical based on previous similar studies. The participants were further subgrouped based on their self-reported clinical full-time equivalent (FTE ) measure. This resulted in 3 subgroups for analysis: medical FTE <0.5, medical FTE >0.5, and surgical (all of whom reported FTE >0.5). The analysis was limited to outpatient clinical interactions and explicitly excluded inpatient activities. RESULTS: A total of 72 participants from 19 different specialties enrolled in this study. The providers had, on average, 8.31 appointments per day during the reporting periods. The providers received, on average, 21.93 messages per day, and they spent 7.61 minutes on average in the time in In Basket per day metric and 1.84 minutes on average in the time in In Basket per appointment metric. The time for the providers to mark messages as done (turnaround time) was on average 11.45 days during the reporting period. Although the surgical group had, on average, approximately twice as many appointments per scheduled day, they spent considerably less connected time (based on almost all time metrics) than the medical group. However, the surgical group took much longer than the medical group to mark messages as done (turnaround time). CONCLUSIONS: We observed a range of patterns with no consistent direction. There does not seem to be evidence of a "learning curve," which would have shown a consistent reduction in time spent on the system over time due to familiarity and experience. While this study does not show how the included metrics could be used as predictors of providers' satisfaction or feelings of burnout, the use trends could be used to start discussions about future Canadian studies needed in this area.

Framework for policymaking on self-management of health by older adults using technologies.

BACKGROUND: During the coronavirus disease 2019 (COVID-19) pandemic, the use of information and communication technologies (ICTs) to support care management exponentially increased. Governments around the world adapted existing programs to meet the needs of patients. The reactivity of governments, however, led to changes that were inequitable, undermining groups such as older adults living with chronic diseases and disability. Policies that align with recent developments in ICTs can promote better health outcomes and innovation in care management. A framework for policymaking presents potential for overcoming barriers and gaps that exist in current policies. OBJECTIVE: The goal of this study was to examine how well a provisional framework for policymaking represented the interactions between various components of government policymaking on older adults' self-management of chronic disease and disability using ICTs. METHODS: Through an online survey, the study engaged policymakers from various ministries of the government of Ontario in the evaluation and revision of the framework. The data were analyzed using simple statistics and by interpreting written comments. RESULTS: Nine participants from three ministries in the government of Ontario responded to the questionnaire. Overall, participants described the framework as useful and identified areas for improvement and further clarification. A revised version of the framework is presented. CONCLUSIONS: Through the revision exercise, our study confirmed the relevance and usefulness for a policymaking framework on the self-management of disease and disability of older adults' using ICTs. Further inquiries should examine the application of the framework to jurisdictions other than Ontario considering the dissociated nature of Canadian provincial healthcare systems.

When rules turn into tools: An activity theory-based perspective on implementation processes and unintended consequences.

The idea that actions of people, organizations or governments may lead to Unintended Consequences (UICs) is not new. In health, UICs have been reported as a result of various interventions including quality improvement initiatives, health information technology implementation, and knowledge translation, especially those involving translation of broad policies (evidence-based medicine and patient-centred care) or system level improvement into actionable items or tools. While some unintended consequences cannot be anticipated, others may be predictable. In this article, we present a model based on cultural historical activity theory, which may help policy-makers, health leaders, and researchers better anticipate UICs resulting from implementation of new programs or technologies and take action to address them or mitigate their risk of occurrence. We support this model using examples of UICs of implementing family centred care principles, electronic health records, and computerized templates for quality improvement in chronic disease management.

Shifting Mindsets: The Impact of a Patient Portal on Functioning and Recovery in a Mental Health Setting.

OBJECTIVE: This study aims to understand whether higher use of a patient portal can have an impact on mental health functioning and recovery. METHOD: A mixed methods approach was used for this study. In 2019-2021, patients with mental health diagnoses at outpatient clinics in an academic centre were invited to complete World Health Organization Disability Assessment Scale 12 (WHODAS-12) and Mental Health Recovery Measure surveys at baseline, 3 months, and 6 months after signing up for the portal. At the 3-month time point, patients were invited to a semistructured interview with a member of the team to contextualize the findings obtained from the surveys. Analytics data was also collected from the platform to understand usage patterns on the portal. RESULTS: Overall, 113 participants were included in the analysis. There was no significant change in mental health functioning and recovery scores over the 6-month period. However, suboptimal usage was observed as 46% of participants did not complete any tasks within the portal. Thirty-five participants had low use of the portal (1-9 interactions) and 18 participants had high usage (10+ interactions). There were also no differences in mental health functioning and recovery scores between low and high users of the portal. Qualitative interviews highlighted many opportunities where the portal can support overall functioning and mental health recovery. CONCLUSIONS: Collectively, this study suggests that higher use of a portal had no impact, either positive or negative, on mental health outcomes. While it may offer convenience and improved patient satisfaction, adequate support is needed to fully enable these opportunities for patient care. As the type of interaction with the portal was not specifically addressed, future work should focus on looking at ways to support patient engagement and portal usage throughout their care journey.

Telehealth as a Component of One Health: a Position Paper.

INTRODUCTION: One Health (OH) refers to the integration of human, animal, and ecosystem health within one framework in the context of zoonoses, antimicrobial resistance and stewardship, and food security. Telehealth refers to distance delivery of healthcare. A systems approach is central to both One Health and telehealth, and telehealth can be a core component of One Health. Here we explain how telehealth might be integrated into One Health. METHODS: We have considered antimicrobial resistance (AMR) as a use case where both One Health and telehealth can be used for coordination among the farming sector, the veterinary services, and human health providers to mitigate the risk of AMR. We conducted a narrative review of the literature to develop a position on the inter-relationships between telehealth and One Health. We have summarised how telehealth can be incorporated within One Health. RESULTS: Clinicians have used telehealth to address antimicrobial resistance, zoonoses, food borne infection, improvement of food security and antimicrobial stewardship. We identified little existing evidence in support of the usage of telehealth within a One Health paradigm, although in isolation, both are useful for the same purpose, i.e., mitigation of the significant public health risks posed by zoonoses, food borne infections, and antimicrobial resistance. CONCLUSIONS: It is possible to integrate telehealth within a One Health framework to develop effective inter-sectoral communication essential for the mitigation and addressing of zoonoses, food security, food borne infection containment and antimicrobial stewardship. More research is needed to substantiate and investigate this model of healthcare.

One Health: Insights from Organizational & Social, Technology Assessment and Human Factors Perspectives.

OBJECTIVES: To offer diverse but complementary perspectives on how biomedical and health informatics can be informed by and help to achieve the vision of One Health. METHODS: Overview of key considerations and critical discussion of common themes, barriers and opportunities, based on collaborative review by International Medical Informatics Association (IMIA) working group members active in related fields. RESULTS: Health and care systems are complex sociotechnical systems that need explicit design and implementation strategies to align with the goals of One Health. The evidence-based health informatics paradigm and associated frameworks for evaluation of digital health technologies need to broaden their scope to take full account of the One Health approach. Informatics has specific contributions to make to One Health, for example by improved user experience reducing energy consumption and effective app design enhancing medication adherence. CONCLUSIONS: One Health is inherently intertwined with ergonomic, sociotechnical and evaluation perspectives in biomedical and health informatics. Health is a planetary issue that requires interdisciplinary collaborative action. The theories and principles of biomedical and health informatics offer many opportunities to transform digital health technology to better serve the One Health agenda.

General Practitioners' Perspectives About Remote Dermatology Care During the COVID-19 Pandemic in the Netherlands: Questionnaire-Based Study.

BACKGROUND: The COVID-19 pandemic affected the delivery of primary care and stimulated the use of digital health solutions such as remote digital dermatology care. In the Netherlands, remote store-and-forward dermatology care was already integrated into Dutch general practice before the COVID-19 pandemic. However, it is unclear how general practitioners (GPs) experienced this existing digital dermatology care during the pandemic period. OBJECTIVE: We investigated GPs' perspectives about facilitators and barriers related to store-and-forward digital dermatology care during the COVID-19 pandemic in the Netherlands, using a sociotechnical approach. METHODS: In December 2021, a web-based questionnaire was distributed via email to approximately 3257 GPs who could perform a digital dermatology consultation and who had started a digital consultation (not necessarily dermatology) in the previous 2 years. The questionnaire consisted of general background questions, questions from a previously validated telemedicine service user satisfaction questionnaire, and newly added questions related to the pandemic and use of the digital dermatology service in general practice. The open-ended and free-text responses were analyzed for facilitators and barriers using content analysis, guided by an 8-dimensional sociotechnical model. RESULTS: In total, 71 GPs completed the entire questionnaire, and 66 (93%) questionnaires were included in the data analysis. During the questionnaire distribution period, another national lockdown, social distancing, and stay-at-home mandates were announced; thus, GPs may have had increased workload and limited time to complete the questionnaire. Of the 66 responding GPs, 36 (55%) were female, 25 (38%) were aged 35-44 years, 33 (50%) were weekly platform users, 34 (52%) were working with the telemedicine organization for >5 years, 42 (64%) reported that they used the store-and-forward platform as often during as before the pandemic, 61 (92%) would use the platform again, 53 (80%) would recommend the platform to a colleague, and 10 (15%) used digital dermatology home consultation. Although GPs were generally satisfied with the digital dermatology service, platform, and telemedicine organization, they also experienced crucial barriers to the use of the service during the pandemic. These barriers were GPs' and patients' limited digital photography skills, costs and the lack of appropriate equipment, human-computer interface and interoperability issues on the telemedicine platform, and different use procedures of the digital dermatology service. CONCLUSIONS: Although remote dermatology care was already integrated into Dutch GP practice before the pandemic, which may have facilitated the positive responses of GPs about the use of the service, barriers impeded the full potential of its use during the pandemic. Training is needed to improve the use of equipment and quality of (dermoscopy) images taken by GPs and to inform GPs in which circumstances they can or cannot use digital dermatology. Furthermore, the dermatology platform should be improved to also guide patients in taking photographs with sufficient quality.

The Role of Human and Organizational Factors in the Pursuit of One Digital Health.

OBJECTIVE: This paper surveys a subset of the 2022 human and organizational factor (HOF) literature to provide guidance on building a One Digital Health ecosystem. METHODS: We searched a subset of journals in PubMed/Medline for studies with "human factors" or "organization" in the title or abstract. Papers published in 2022 were eligible for inclusion in the survey. Selected papers were categorized into structural and behavioural aspects to understand digital health enabled interactions across micro, meso, and macro systems. RESULTS: Our survey of the 2022 HOF literature showed that while we continue to make meaningful progress at digital health enabled interactions across systems levels, there are still challenges that must be overcome. For example, we must continue to grow the breadth of HOF research beyond individual users and systems to assist with the scale up of digital health systems across and beyond organizations. We summarize the findings by providing five HOF considerations to help build a One Digital Health ecosystem. CONCLUSION: One Digital Health challenges us to improve coordination, communication, and collaboration between the health, environmental and veterinary sectors. Doing so requires us to develop both the structural and behavioural capacity of digital health systems at the organizational level and beyond so that we can develop more robust and integrated systems across health, environmental and veterinary sectors. The HOF community has much to offer and must play a leading role in designing a One Digital Health ecosystem.

Policymaker perspectives on self-management of disease and disabilities using information and communication technologies.

BACKGROUND: Policies that support health self-management are malleable and highly dependent on various factors that influence governments. Within a world that is shifting toward digitalization due to pressures such as the COVID-19 pandemic and labor shortages, policymaking on older adults' self-management of chronic diseases and disability using information and communication technologies (ICTs) needs to be better understood. Using the province of Ontario, in Canada, as a case study, the research question was What is the environment that policymakers must navigate through in development and implementation of policies related to older adults' self-management of disease and disability using information and communication technologies (ICTs)? METHODS: This study used a qualitative approach where public servants from 4 ministries within the government of Ontario were invited to participate in a 1-h, one-on-one, semi-structured interview. The audio-recorded interviews were based on an adapted model of the policy triangle, where the researcher asked questions about the influences from the different sources identified in the model. The interviews were later transcribed and analyzed using a deductive-inductive coding approach. RESULTS: Ten participants across 4 different Ministries participated in the interviews. Participants shared insights on various aspects of context, process and actors that help shape the current content of policies. The analysis revealed that policies, in the form of programs, services, legislation and regulations, are the result of collaborations and dialogue between different actors and get developed and implemented via a set of complex government processes. In addition, policy actions come from a plethora of sectors which all get influenced by several predictable and unpredictable external pressures. CONCLUSIONS: The environment for policymaking in the government of Ontario regarding older adults' self-management of disease and disability using ICTs is one that is mostly reactive to external pressures, while organized within a set of complex processes and multi-sectoral collaborations. The present research helped us to understand the complexity of policymaking on the topic and highlights the need for increased foresight and proactive policymaking, regardless of which governments are in-place.

Automation vs. Innovation: Unexplored Strategies to Improve Virtual Care.

While there is a global desire to increase digital health capacity, digital health should transform health services delivery rather than simply automate - or worse - replicate existing practices. Failing to capitalize on this transformative potential misses an opportunity to engage patients and other users to provide a more person-centered experience. However, digital transformation done recklessly can disrupt workflow, alienate users, and jeopardize patient safety, as we have observed with implementation of many digital health tools. This paper uses a telemedicine example to provide insight into how digital health innovation can be a meaningful enabler of health system transformation. Examining different ways to leverage digital health technologies is crucial to best capitalize on their potential.

Differential Perceptions of What Constitutes a Medical Error Associated with Electronic Medical Records.

Perceptions of errors associated with healthcare information technology (HIT) often depend on the context and position of the viewer. HIT vendors posit very different causes of errors than clinicians, implementation teams, or IT staff. Even within the same hospital, members of departments and services often implicate other departments. Organizations may attribute errors to external care partners that refer patients, such as nursing homes or outside clinics. Also, the various clinical roles within an organization (e.g., physicians, nurses, pharmacists) can conceptualize errors and their root causes differently. Overarching all these perceptual factors, the definitions, mechanisms, and incidence of HIT-related errors are remarkably conflictual. There is neither a universal standard for defining or counting these errors. This paper attempts to enumerate and clarify the issues related to differential perceptions of medical errors associated with HIT. It then suggests solutions.

Identifying Failure Modes in Telemedicine: An Instructional Needs Assessment.

Technology failures in telehealth are common, and clinicians need the skills to diagnose and manage them at the point of care. However, there are issues beyond technology failures mediating the effective use of telehealth. We must teach best-practice procedures for conducting telemedicine visits and include in instructional simulations commonly encountered failure modes so students can build their skills. To this end, we recruited medical students to conduct a Healthcare Failure Modes and Effects Analysis (HFMEA) to predict failures in telemedicine, their potential causes, and the consequences to develop and teach prevention strategies. Sixteen students observed telehealth appointments independently. Based on their observations, we identified four categories of failures in telemedicine: technical issues, patient safety, communication, and social and structural determinants. We proposed a normalized workflow that included management and prevention strategies. Our findings can inform the creation of new curricula.

Evolution of public health policy on healthcare self-management: the case of Ontario, Canada.

BACKGROUND: As people live longer, they are at increased risk for chronic diseases and disability. Self-management is a strategy to improve health outcomes and quality of life of those who engage in it. This study sought to gain a better understanding of the factors, including digital technology, that affect public health policy on self-management through an analysis of government policy in the most populous and multicultural province in Canada: Ontario. The overarching question guiding the study was: What factors have influenced the development of healthcare self-management policies over time? METHODS: Archival research methods, combining document review and evaluation, were used to collect data from policy documents published in Ontario. The documents were analyzed using the READ approach, evaluated using a data extraction table, and synthesized into themes using the model for health policy analysis. RESULTS: Between January 1, 1985, and May 5, 2022, 72 policy documents on self-management of health were retrieved from databases, archives, and grey literature. Their contents largely focussed on self-management of general chronic conditions, while 47% (n = 18/72) mention diabetes, and 3% (n = 2/72) focussed solely on older adults. Digital technologies were mentioned and were viewed as tools to support self-management in the context of healthcare delivery and enhancing healthcare infrastructure (i.e., telehealth or software in healthcare settings). The actors involved in the policy document creation included mostly Ontario government agencies and departments, and sometimes expert organizations, community groups and engaged stakeholders. The results suggest that several factors including pressures on the healthcare system, hybrid top-down and bottom-up policymaking, and political context have influenced the nature and implementation timing of self-management policy in Ontario. CONCLUSIONS: The policy documents on self-management of health reveal a positive evolution of the content discussed over time. The changes were shaped by an evolving context, both from a health and political perspective, within a dynamic system of interactions between actors. This research helps understand the factors that have shaped changes and suggests that a critical evidence-based approach on public health policy is needed in understanding processes involved in the development of healthcare self-management policies from the perspective of a democratic governing system.

Assessing Telemedicine Competencies: Developing and Validating Learner Measures for Simulation-Based Telemedicine Training.

In 2021, the Association of American Medical Colleges published Telehealth Competencies Across the Learning Continuum, a roadmap for designing telemedicine curricula and evaluating learners. While this document advances educators' shared understanding of telemedicine's core content and performance expectations, it does not include turn-key-ready evaluation instruments. At the University of Oklahoma School of Community Medicine, we developed a year-long telemedicine curriculum for third-year medical and second-year physician assistant students. We used the AAMC framework to create program objectives and instructional simulations. We designed and piloted an assessment rubric for eight AAMC competencies to accompany the simulations. In this monograph, we describe the rubric development, scores for students participating in simulations, and results comparing inter-rater reliability between faculty and standardized patient evaluators. Our preliminary work suggests that our rubric provides a practical method for evaluating learners by faculty during telemedicine simulations. We also identified opportunities for additional reliability and validity testing.

Telehealth as a Means of Enabling Health Equity.

OBJECTIVE: The goal of this paper is to provide a consensus review on telehealth delivery prior to and during the COVID-19 pandemic to develop a set of recommendations for designing telehealth services and tools that contribute to system resilience and equitable health. METHODS: The IMIA-Telehealth Working Group (WG) members conducted a two-step approach to understand the role of telehealth in enabling global health equity. We first conducted a consensus review on the topic followed by a modified Delphi process to respond to four questions related to the role telehealth can play in developing a resilient and equitable health system. RESULTS: Fifteen WG members from eight countries participated in the Delphi process to share their views. The experts agreed that while telehealth services before and during COVID-19 pandemic have enhanced the delivery of and access to healthcare services, they were also concerned that global telehealth delivery has not been equal for everyone. The group came to a consensus that health system concepts including technology, financing, access to medical supplies and equipment, and governance capacity can all impact the delivery of telehealth services. CONCLUSION: Telehealth played a significant role in delivering healthcare services during the pandemic. However, telehealth delivery has also led to unintended consequences (UICs) including inequity issues and an increase in the digital divide. Telehealth practitioners, professionals and system designers therefore need to purposely design for equity as part of achieving broader health system goals.

Towards Equitable and Resilient Digital Primary Care Systems: An International Comparison and Insight for Moving Forward.

OBJECTIVE: While the COVID-19 pandemic provided a global stimulus for digital health capacity, its development has often been inequitable, short-term in planning, and lacking in health system coherence. Inclusive digital health and the development of resilient health systems are broad outcomes that require a systematic approach to achieving them. This paper from the IMIA Primary Care Informatics Working Group (WG) provides necessary first steps for the design of a digital primary care system that can support system equity and resilience. METHODS: We report on digital capability and growth in maturity in four key areas: (1) Vaccination/Prevention, (2) Disease management, (3) Surveillance, and (4) Pandemic preparedness for Australia, Canada, and the United Kingdom (data from England). Our comparison looks at seasonal influenza management prior to COVID-19 (2019-20) compared to COVID-19 (winter 2020 onwards). RESULTS: All three countries showed growth in digital maturity from the 2019-20 management of influenza to the 2020-21 year and the management of the COVID-19 pandemic. However, the degree of progress was sporadic and uneven and has led to issues of system inequity across populations. CONCLUSION: The opportunity to use the lessons learned from COVID-19 should not be wasted. A digital health infrastructure is not enough on its own to drive health system transformation and to achieve desired outcomes such as system equity and resilience. We must define specific measures to track the growth of digital maturity, including standardized and fit-for-context data that is shared accurately across the health and socioeconomic sectors.

Exploring human factors in the operating room: scoping review of training offerings for healthcare professionals.

BACKGROUND: Human factors (HF) integration can improve patient safety in the operating room (OR), but the depth of current knowledge remains unknown. This study aimed to explore the content of HF training for the operative environment. METHODS: We searched six bibliographic databases for studies describing HF interventions for the OR. Skills taught were classified using the Chartered Institute of Ergonomics and Human Factors (CIEHF) framework, consisting of 67 knowledge areas belonging to five categories: psychology; people and systems; methods and tools; anatomy and physiology; and work environment. RESULTS: Of 1851 results, 28 studies were included, representing 27 unique interventions. HF training was mostly delivered to interdisciplinary groups (n = 19; 70 per cent) of surgeons (n = 16; 59 per cent), nurses (n = 15; 56 per cent), and postgraduate surgical trainees (n = 11; 41 per cent). Interactive methods (multimedia, simulation) were used for teaching in all studies. Of the CIEHF knowledge areas, all 27 interventions taught 'behaviours and attitudes' (psychology) and 'team work' (people and systems). Other skills included 'communication' (n = 25; 93 per cent), 'situation awareness' (n = 23; 85 per cent), and 'leadership' (n = 20; 74 per cent). Anatomy and physiology were taught by one intervention, while none taught knowledge areas under work environment. CONCLUSION: Expanding HF education requires a broader inclusion of the entirety of sociotechnical factors such as contributions of the work environment, technology, and broader organizational culture on OR safety to a wider range of stakeholders.

A framework for role allocation in education, research and leadership services in Canadian academic divisions of general surgery: a modified Delphi consensus.

BACKGROUND: Moving toward a funding standard similar to that for clinical services for roles essential to the functioning of education, research and leadership services within divisions of general surgery is necessary to strengthen divisional resilience. We aimed to identify roles and underlying tasks in these services central to sustainable functioning of Canadian academic divisions of general surgery. METHODS: Between June 2018 and October 2020, we used a 4-step modified Delphi method (online survey, face-to-face nominal group technique [n = 12], semistructured telephone interview [n = 8] and nominal group technique [n = 12]) to achieve national consensus from an expert panel of all 17 heads of academic divisions of general surgery in Canada on the roles and accompanying tasks essential to education, research and leadership services within an academic division of general surgery. We used 70% agreement to determine consensus. RESULTS: The expert panel agreed that a framework for role allocation in education, research and leadership services was relevant and necessary. Consensus was reached for 7 roles within the educational service, 3 roles within the research service and 5 roles within the leadership service. CONCLUSION: Our framework represents a national consensus that defines role standards for education, research and leadership services in Canadian academic divisions of general surgery. The framework can help divisions build resiliency, and enable sustained and deliberate advances in these services.