Testing the reliability of accident analysis methods: a comparison of AcciMap, STAMP-CAST and AcciNet.

Accident analysis methods are used to model the multifactorial cause of adverse incidents. Methods such as AcciMap, STAMP-CAST and recently AcciNet, are systemic approaches that support the identification of safety interventions across sociotechnical system levels. Despite their growing popularity, little is known about how reliable systems-based methods are when used to describe, model and classify contributory factors and relationships. Here, we conducted an intra-rater and inter-rater reliability assessment of AcciMap, STAMP-CAST and AcciNet using the Signal Detection Theory (SDT) paradigm. A total of 180 hours' worth of analyses across 360 comparisons were performed by 30 expert analysts. Findings revealed that all three methods produced a weak to moderate positive correlation coefficient, however the inter-rater reliability of STAMP-CAST was significantly higher compared to AcciMap and AcciNet. No statistically significant or practically meaningful differences were found between methods in the overall intra-rater reliability analyses. Implications and future research directions are discussed.

Practitioners who undertake accident analysis within their organisations should consider the use of STAMP-CAST due to the significantly higher inter-rater reliability findings obtained in this study compared to AcciMap and AcciNet, particularly if they tend to work alone and/or part of relatively small teams.

Findings, progress, and lessons learned during the first 3 years of a student-led interprofessional health clinic in regional Australia.

We describe the establishment and operation of a student-led interprofessional chronic disease prevention and management clinic in regional Australia. Our aim was twofold. First, to report on service delivery, student placement, and health outcome data; and second, to discuss key lessons learned during the first 3½ years of clinic operations. Between July 2019 and December 2022, 146 (79.3%) clinic participants completed the 4-month program and participated in an average of 48.4 occasions of service (total 7,060). The clinic supported 1,060 clinical placement weeks across 147 health students. There was a significant improvement across health measures reported at program completion, with the largest changes observed for the 6-min walk test and preference-adjusted quality of life. Nine key challenges and lessons were identified that affected operations and service delivery, which should be of interest to healthcare teams considering establishing an interprofessional student-led clinic.

Understanding complexity in a safety critical setting: A systems approach to medication administration.

'Medication errors' are a significant concern and are associated with a higher incidence of adverse events and unintentional patient harm than any other aspect of healthcare. While much research has focused on adverse medication errors, limited studies have specifically examined 'normal' medication delivery performance and the interactions between tasks, agents, and information within the medication administration system. This article describes a study that applied the Event Analysis of Systemic Teamwork (EAST) model to study the hospital medication administration system to identify opportunities to optimise performance and patient safety. Key findings of this study demonstrate that this is a highly complex system, comprising many social agents and a relatively closely linked series of tasks and information. However, most of the workload relies on a small proportion of healthcare professionals. Significantly, the patient has a minimal role in the medication administration system during their hospital stay. The research has shown that this approach enables mapping networks and their interdependencies to optimise the system as a whole rather than its parts in isolation.

Impact of the COVID-19 pandemic on student supervision and education in health care settings: A state-wide survey of health care workers.

OBJECTIVE: To investigate student supervisor experiences of supervising students on clinical placements since the onset of the COVID-19 pandemic. BACKGROUND: Studies on the impact of COVID-19 on student clinical placements have focused largely on student reports and have been specific to individual professions or topic areas. There is a need to investigate student supervisor experiences. This study was conducted in Queensland (Australia) in four regional and rural public health services and four corresponding primary health networks. METHODS: The anonymous, mixed methods online survey, consisting of 35 questions, was administered to student supervisors from allied health, medicine, nursing and midwifery between May and August 2021. Numerical data were analysed descriptively using chi-square tests. Free-text comments were analysed using content analysis. RESULTS: Complete datasets were available for 167 respondents. Overall trends indicated perceived significant disruptions to student learning and support, plus mental health and well-being concerns for both students and supervisors. Extensive mask wearing was noted to be a barrier to building rapport, learning and teaching. Some positive impacts of the pandemic on student learning were also noted. CONCLUSIONS: This study has highlighted the perceived impact of the pandemic on supervisors' mental health, and on the mental health, learning and work readiness of students. This study provides evidence of the pandemic impacts on student clinical placements from a supervisor point of view. Findings can assist in future-proofing clinical education and ensuring that students continue to receive learning experiences of benefit to them, meeting curriculum requirements, in the event of another pandemic.

Towards a unified model of accident causation: refining and validating the systems thinking safety tenets.

The systems thinking tenets were developed based on a synthesis of contemporary accident causation theory, models and approaches and encapsulate 15 features of complex systems that interact to create both safety and adverse events. Whilst initial testing provided supportive evidence, the tenets have not yet been subject to formal validation. This article presents the findings from a three-round Delphi study undertaken to refine and validate the tenets and assess their suitability for inclusion in a unified model of accident causation. Participants with expertise in accident causation and systems thinking provided feedback on the tenets and associated definitions until an acceptable level of consensus was achieved. The results reduced the original 15 tenets to 14 and 10 were identified as important to include in unified model of accident causation. The refined systems thinking tenets are presented along with future research directions designed to facilitate their use in safety practice.Practitioner summary: This article presents a refined and validated set of systems thinking tenets which describe features of complex systems that interact to create adverse events. The tenets can be used by practitioners to proactively identify safety leading indicators and contributory factors during adverse event analysis.

Using systems thinking-based risk assessment methods to assess hazardous manual tasks: a comparison of Net-HARMS, EAST-BL, FRAM and STPA.

Formal risk assessment is a component of safety management relating to hazardous manual tasks (HMT). Systems thinking approaches are currently gaining interest for supporting safety management. Existing HMT risk assessment methods have been found to be limited in their ability to identify risks across the whole work system; however, systems thinking-based risk assessment (STBRA) methods were not designed for the HMT context and have not been tested in this area. The aim of this study was to compare the performance of four state-of-the-art STBRA methods: Net-HARMS, EAST-BL, FRAM and STPA to determine which would be most useful for identifying HMT risks. Each method was independently applied by one of four analysts to assess the risks associated with a hypothetical HMT system. The outcomes were assessed for alignment with a benchmark analysis. Using signal detection theory (SDT), overall STPA was found to be the best performing method having the highest hit rate, second lowest false alarm rate and highest Matthews Correlation Coefficient of the four methods.Practitioner summary: A comparison of four systems thinking risk assessment methods found that STPA had the highest level of agreement with the benchmark analysis and is the most suitable for practitioners to use to identify the risks associated with HMT systems.

The need for a complex systems approach in rural health research.

On a global scale, many major rural health issues have persisted for decades despite the introduction of new health interventions and public health policies. Although research efforts have generated valuable new knowledge about the aetiology of health, disease and health inequities in rural communities, rural health systems remain to be some of the most deprived and challenged in both the developing and developed world. While the reasons for this are many, a significant factor contributing to the current state of play is the pressing need for methodological innovation and relevant scientific approaches that have the capacity to support the translation of novel solutions into 'real world' rural contexts. Fortunately, complex systems approaches, which have seen an increase in popularity in the wider public health literature, could provide answers to some of the most resilient rural health problems in recent times. The purpose of this article is to promote the value and utility of a complex systems approach in rural health research. We explain the benefits of a complex systems approach and provide a background to the complexity sciences, including the main characteristics of complex systems. Two popular computational methods are described. The next step for rural health research involves exploring how a complex systems approach can help with the identification and evaluation of new and existing solutions to policy-resistant rural health issues. This includes generating awareness around the analytical trade-offs that occur between the use of traditional scientific methods and complex systems approaches.

Testing the reliability and validity of risk assessment methods in Human Factors and Ergonomics.

There is growing interest in the use of systems-based risk assessment methods in Human Factors and Ergonomics (HFE). The purpose of this study was to test the intra-rater reliability and criterion-referenced concurrent validity of three systems-based risk assessment approaches: (i) the Systems-Theoretic Process Analysis (STPA) method; (ii) the Event Analysis of Systemic Teamwork Broken Links (EAST-BL) method; and, (iii) the Network Hazard Analysis and Risk Management System (Net-HARMS) method. Reliability and validity measures were obtained using the Signal Detection Theory (SDT) paradigm. Whilst STPA identified the highest number of risks, the findings indicate a weak to moderate level of reliability and validity for STPA, EAST-BL and Net-HARMS. There were no statistically significant differences between the methods across analyses. The results suggest that there is merit to the continued use of systems-based risk assessment methods following a series of methodological extensions that aim to enhance the reliability and validity of future applications. Practitioner summary The three risk assessment methods produced weak to moderate levels of stability and accuracy regarding their capability to predict risks. There is a pressing need to further test the reliability and validity of safety methods in Human Factors and Ergonomics.

Human Factors and Ergonomics and the management of existential threats: A work domain analysis of a COVID-19 return from lockdown restrictions system.

Following strict "lockdown" restrictions designed to control the spread of the COVID-19 virus, many jurisdictions are now engaged in a process of easing restrictions in an attempt to stimulate economic and social activity while continuing to suppress virus transmission. This is challenging and complex, and in several regions, new outbreaks have emerged. We argue that systems Human Factors and Ergonomics methods can assist in understanding and optimizing the return from lockdown. To demonstrate, we used work domain analysis to develop an abstraction hierarchy model of a generic "return from lockdown restrictions" system. The model was assessed to identify (a) issues preventing a successful return from lockdown; and (b) leverage points that could be exploited to optimize future processes. The findings show that the aim of continuing to suppress virus transmission conflicts with the aims of returning to pre-virus economic and social activity levels. As a result, many functions act against each other, ensuring that the system cannot optimally achieve all three of its primary aims. Potential leverage points include modifying the goals and rules of the system and enhancing communications and feedback. Specifically, it is argued that moderating economic aims and modifying how social and community activities are undertaken will result in longer term suppression of the virus.

The impact of power on health care team performance and patient safety: a review of the literature.

Communication failure within health care teams is a major cause of patient harm across health care settings. Factors which contribute to communication failure include actual or perceived 'power'. Whilst a great deal of ergonomics research has focussed on teamwork in health care, the role of power in relation to measurable patient safety and performance outcomes remains relatively unknown. This article presents the findings from a review of the literature on power within multidisciplinary health care team settings. Following a systematic literature search, nineteen studies were evaluated in terms of research design, methods and analyses across the included studies. The main impacts resulting from power imbalances include negative effects on team collaboration, decision-making, communication and overall performance. Wider patient safety research, and more specifically the ergonomics discipline, is encouraged to address the complex interplay between power and teamwork in the health care sector.Practitioner Statement: We conducted a review of studies focussed on the influence of power on teamwork in health care. The findings show that power can have negative impacts on collaboration, decision-making, communication, and team performance. We conclude that power represents an important area for ergonomics, both in health care and other settings.Abbreviations: CRM: crew resource management; TEM: threat and error management; SNA: social network analysis; EAST: event analysis of systemic teamwork.

Complexity theory in accident causation: using AcciMap to identify the systems thinking tenets in 11 catastrophes.

The quest to explain and understand the cause of accidents is both ever-present and ongoing amongst the safety science community. In an attempt to advance the theory and science of accident causation, researchers have recently formalised a set of '15 systems thinking tenets' that cover the conditions and characteristics of work systems that are believed to contribute to the cause of accidents. The purpose of this study was to attempt to identify the systems thinking tenets across a range of different systems and accidents using the Accident Mapping (AcciMap) method. The findings suggest that the tenets can be attributed to play a role in accident causation, however as a result of this process, the capability of AcciMap has been brought into question. Implications and directions for future research are described. Practitioner Summary: This study is an extension of previous work that suggested there was a need to test for the 'systems thinking tenets of accident causation' in a multi-incident dataset. We used AcciMap to evaluate whether it has the capability to support ongoing accident analysis activities in ergonomics research.

Systems thinking-based risk assessment methods applied to sports performance: A comparison of STPA, EAST-BL, and Net-HARMS in the context of elite women's road cycling.

There is increasing interest in applying systems Human Factors and Ergonomics (HFE) methods in sport. Risk assessment (RA) methods can be used identify risks which may impact the performance of individual athletes, teams, and overall sports systems; however, they have not yet been tested in sport. This study sets out to apply and compare three systems thinking-based RA methods in the context of elite sports performance and report on the frequency and types of the risks identified. The Systems-Theoretic Process Analysis (STPA) method, the Event Analysis of Systemic Teamwork Broken Links (EAST-BL) method, and the Networked Hazard Analysis and Risk Management System (Net-HARMS) method were applied to elite women's road cycling to identify all the credible risks that could degrade optimal team performance. The findings demonstrate that all three methods appear to provide useful results in a context other than safety, and that multiple risks threatening the performance of the cycling team were identified. Whilst the frequency and types of risks differed across the methods applied, there are additional theoretical, methodological, and practical implications to be considered prior to the selection and use of systems thinking-based RA approaches. Recommendations and directions for future HFE and sports science research are discussed.

A Systematic Review and Meta-analysis on Sodium Bicarbonate Administration and Equine Running Performance: Is it Time to Stop Horsing Around With Baking Soda?

Sodium bicarbonate administration in the hours prior to exercise has been used as a performance-enhancing substance in horses since the late 1980s. Although sodium bicarbonate administration to racehorses 24 hours before racing is a banned practice in most racing industries, whether or not it improves running performance in racehorses is currently unclear. The aim of this systematic review and meta-analysis was to establish whether or not acute sodium bicarbonate administration improves running performance in trained Standardbred and Thoroughbred horses. Seven randomized controlled trials, including eight experimental (exercise) trials and 74 horses, were included after a comprehensive search for relevant studies that met the inclusion criteria. Results indicated that sodium bicarbonate administration at 2.5-5 hours prior to a standardized treadmill test to exhaustion or simulated race (time-trial) does not influence running performance (number of horses, the overall effect [95% CI]: 32, -0.13 [-0.64 to 0.37] and 42, 0.01 [-0.42 to 0.44], respectively, both P > .05). The included studies demonstrated minimal heterogeneity (I2 = 0%-2%), low risks of bias according to the Cochrane Risk of Bias Tool and a lack of publication bias. On the basis of these findings, there is high-quality evidence to suggest that sodium bicarbonate administration does not improve running performance in trained Standardbred or Thoroughbred horses.

Out of control? Using STAMP to model the control and feedback mechanisms surrounding identity crime in darknet marketplaces.

Darknet marketplaces have emerged as a facilitator of identity crime and trading. This study aimed to (1) understand the entities and control and feedback mechanisms that influence identity crime prevention and occurrence on the darknet in the Australian system and to (2) comprehensively identify the implications of control failures across all system levels. The Systems-Theoretic Accident Model and Processes (STAMP) was used to develop an identity crime control structure in consultation with subject matter experts and then the Systems-Theoretic Process Analysis (STPA) was applied. The STPA identified 310 risk states, resulting from control failures and which were associated with the range of agencies, organisations, and individuals present across system levels. As darknet marketplaces rapidly evolve, alignment between these entities is necessary to enable agile system responses. STAMP and STPA have promise in understanding the potential for intervention across all system levels in preventing societal issues such as identity crime.

Methods matter: exploring the 'too much, too soon' theory, part 1: causal questions in sports injury research.

BACKGROUND: It is widely accepted that athletes sustain sports injury if they train 'too much, too soon'. However, not all athletes are built the same; some can tolerate more training than others. It is for this reason that prescribing the same training programme to all athletes to reduce injury risk is not optimal from a coaching perspective. Rather, athletes require individualised training plans. In acknowledgement of athlete diversity, it is therefore essential to ask the right causal research question in studies examining sports injury aetiology. PURPOSE: In this first part of a British Journal of Sports Medicine educational series, we present four different causal research questions related to the 'too much, too soon' theory and critically discuss their relevance to sports injury prevention. CONTENT: If it is true that there is no 'one size fits all' training programme, then we need to consider by how much training can vary depending on individual athlete characteristics. To provide an evidence-base for subgroup-specific recommendations, a stronger emphasis on the following questions is needed: (1) How much training is 'too much' before athletes with different characteristics sustain sports-related injury? and (2) Does the risk of sports injury differ among athletes with a certain characteristic (eg, high experience) compared with athletes with other characteristics (eg, low experience) depending on how much training they perform? CONCLUSION: We recommend that sports injury researchers aiming to examine the 'too much, too soon' theory should carefully consider how they, assisted by coaches, athletes and clinicians, pose their causal research question. In the light of the limitations of population-based prevention that intends to provide all athletes with the same advice, we argue that a stronger emphasis on research questions targeting subgroups of athletes is needed. In doing so, researchers may assist athletes, clinicians and coaches to understand what training advice/programme works best, for whom and under what circumstances.

Radical systems thinking and the future role of computational modelling in ergonomics: an exploration of agent-based modelling.

We are teetering on the precipice of the imminent Fourth Industrial Revolution. In this new age, systems are set to become more densely intraconnected and interconnected, and massive sociotechnical systems exhibiting unprecedented levels of complexity will increasingly take hold. At the dawning of this new age, the Ergonomics discipline must reflect on its preparedness for tackling problems in these novel systems. This paper engages in this reflection by putting forth a critical commentary on the implication of these changes on the discipline and discusses the utility of our current methods in this new paradigm. A resulting Radical Systems Thinking in Ergonomics Manifesto is put forward - a set of mandates to guide practitioners and researchers in the development of new methods capable of coping with these imminent challenges. From the manifesto are derived a series of capability requirements for future computational modelling approaches in Ergonomics. Practitioner summary: The goal of this paper was to inspire the Ergonomics community to pursue further applications involving computational modelling approaches such as Agent-Based Modelling. It presents a manifesto for the future of the discipline, and from this the capabilities that future computational modelling approaches need to possess. Abbreviations: 1IR: first industrial revolution; 2IR: second industrial revolution; 3IR: third industrial revolution; 4IR: fourth industrial revolution; ABM: agent based model; AI: artificial intelligence; AoF: allocation of function; CPA: cyber physical attack; CPS: cyber-physical system; CWA: cognitive work analysis; DDoS: distributed denial of service; EAST: event analysis of systemic teamwork; FRAM; functional resonance analysis method; HCI: human-computer interaction; HERA: human error and recovery assessment; HET: human error template; HMC: human-machine cooperation; IoT: internet of things; RSTEM: radical systems thinking in ergonomics manifesto; SAI: situated artificial intelligence; STAMP: systems theoretic accident model and processes; TRACEr: technique for the retrospective and predictive analysis of cognitive errors in air traffic control.

A Systems Approach to Performance Analysis in Women's Netball: Using Work Domain Analysis to Model Elite Netball Performance.

Netball is a newly professional women's sport, as such there has been little research conducted investigating performance analysis (PA) in elite netball. The aim of this study was to develop a model of the elite netball performance system to identify the complex relationships among key performance indicators. Eleven elite subject matter experts (SMEs) participated in workshops to produce a systems model of the netball match performance. The model was developed using the work domain analysis (WDA) method. A model of the netball match performance system was produced showing the interrelated objects, processes, functions, values, and purposes involved in elite level netball matches. The model identified the components of elite level netball performance and the interactions and relationships between them. The output of this research has identified novel PA measures including passing and possession measures, measures of cognitive performance, and measures related to physical activity. Netball is a complex sport, involving multiple dynamic and interrelated components. Consequently, there is an opportunity to develop holistic PA measures that focus on interacting components, as opposed to components in isolation.

Applying systems ergonomics methods in sport: A systematic review.

INTRODUCTION: As sports systems become increasingly more complex, competitive, and technology-centric, there is a greater need for systems ergonomics methods to consider the performance, health, and safety of athletes in context with the wider settings in which they operate. Therefore, the purpose of this systematic review was to identify and critically evaluate studies which have applied a systems ergonomics research approach in the context of sports performance and injury management. MATERIAL AND METHODS: Five databases (PubMed, Scopus, ScienceDirect, Web of Science, and SPORTDiscus) were searched for the dates 01 January 1990 to 01 August 2017, inclusive, for original peer-reviewed journal articles and conference papers. Reported analyses were underpinned by a recognised systems ergonomics method, and study aims were related to the optimisation of sports performance (e.g. communication, playing style, technique, tactics, or equipment), and/or the management of sports injury (i.e. identification, prevention, or treatment). RESULTS: A total of seven articles were identified. Two articles were focussed on understanding and optimising sports performance, whereas five examined sports injury management. The methods used were the Event Analysis of Systemic Teamwork, Cognitive Work Analysis (the Work Domain Analysis Abstraction Hierarchy), Rasmussen's Risk Management Framework, and the Systems Theoretic Accident Model and Processes method. The individual sport application was distance running, whereas the team sports contexts examined were cycling, football, Australian Football League, and rugby union. CONCLUSIONS: The included systems ergonomics applications were highly flexible, covering both amateur and elite sports contexts. The studies were rated as valuable, providing descriptions of injury controls and causation, the factors influencing injury management, the allocation of responsibilities for injury prevention, as well as the factors and their interactions underpinning sports performance. Implications and future directions for research are described.