Measuring Real-Time Team Cognition During Team Training.

OBJECTIVE: A method for detecting real-time changes in team cognition in the form of significant communication reorganizations is described. We demonstrate the method in the context of scenario-based simulation training. BACKGROUND: We present the dynamical view that individual- and team-level aspects of team cognition are temporally intertwined in a team's real-time response to challenging events. We suggest that this real-time response represents a fundamental team cognitive skill regarding the rapidity and appropriateness of the response, and methods and metrics are needed to track this skill. METHOD: Communication data from medical teams (Study 1) and submarine crews (Study 2) were analyzed for significant communication reorganization in response to training events. Mutual information between team members informed post hoc filtering to identify which team members contributed to reorganization. RESULTS: Significant communication reorganizations corresponding to challenging training events were detected for all teams. Less experienced teams tended to show delayed and sometimes ineffective responses that more experienced teams did not. Mutual information and post hoc filtering identified the individual-level inputs driving reorganization and potential mechanisms (e.g., leadership emergence, role restructuring) underlying reorganization. CONCLUSION: The ability of teams to rapidly and effectively reorganize coordination patterns as the situation demands is a team cognitive skill that can be measured and tracked. APPLICATION: Potential applications include team monitoring and assessment that would allow for visualization of a team's real-time response and provide individualized feedback based on team member's contributions to the team response.

Alternative Markers of Performance in Simulation: Where We Are and Where We Need To Go.

This article on alternative markers of performance in simulation is the product of a session held during the 2017 Academic Emergency Medicine Consensus Conference "Catalyzing System Change Through Health Care Simulation: Systems, Competency, and Outcomes." There is a dearth of research on the use of performance markers other than checklists, holistic ratings, and behaviorally anchored rating scales in the simulation environment. Through literature review, group discussion, and consultation with experts prior to the conference, the working group defined five topics for discussion: 1) establishing a working definition for alternative markers of performance, 2) defining goals for using alternative performance markers, 3) implications for measurement when using alternative markers, identifying practical concerns related to the use of alternative performance markers, and 5) identifying potential for alternative markers of performance to validate simulation scenarios. Five research propositions also emerged and are summarized.