Untangling connections between challenges in the adoption of learning analytics in higher education.

Potential benefits of learning analytics (LA) for improving students' performance, predicting students' success, and enhancing teaching and learning practice have increasingly been recognized in higher education. However, the adoption of LA in higher education institutions (HEIs) to date remains sporadic and predominantly small in scale due to several socio-technical challenges. To better understand why HEIs struggle to scale LA adoption, it is needed to untangle adoption challenges and their related factors. This paper presents the findings of a study that sought to investigate the associations of adoption factors with challenges HEIs face in the adoption of LA and how these associations are compared among HEIs at different scopes of adoption. The study was based on a series of semi-structured interviews with senior managers in HEIs. The interview data were thematically analysed to identify the main challenges in LA adoption. The connections between challenges and other factors related to LA adoption were analysed using epistemic network analysis (ENA). From senior managers' viewpoints, ethical issues of informed consent and resistance culture had the strongest links with challenges of learning analytic adoption in HEI; this was especially true for those institutions that had not adopted LA or who were in the initial phase of adoption (i.e., preparing for or partially implementing LA). By contrast, among HEIs that had fully adopted LA, the main challenges were found to be associated with centralized leadership, gaps in the analytic capabilities, external stakeholders, and evaluations of technology. Based on the results, we discuss implications for LA strategy that can be useful for institutions at various stages of LA adoption, from early stages of interest to the full adoption phase.

Are the World Health Organization's Patient Safety Learning Objectives Still Up-to-Date: A Group Concept Mapping Study.

BACKGROUND: The World Health Organization (WHO) Patient Safety Curriculum Guide defines learning objectives for patient safety. Current implementation in healthcare education is insufficient. Possible explanations may be obsolescence and/or a shift in needs. We investigated whether overarching topics and specific learning objectives of the WHO Patient Safety Curriculum Guide are still up-to-date, their attributed importance, and their perceived difficulty to achieve. METHODS: Experts on patient safety and medical education from 3 European countries were asked to suggest learning objectives concerning patient safety using group concept mapping. Following 3 successive steps, experts rated ideas by importance and difficulty to achieve. Correlation analyses investigated the relationship between those. Overarching topics of the learning goals (clusters) were identified with multivariate analysis. RESULTS: A total of 119 statements about intended learning objectives on patient safety were generated, of which 86 remained for sorting and rating. Based on multivariate analyses, 10 overarching topics (clusters) emerged. Both the learning objectives and the overarching topics showed high correspondence with the WHO Patient Safety Curriculum Guide. Strong correlations emerged between importance and difficulty ratings for learning objectives and overarching topics. CONCLUSIONS: The WHO Patient Safety Curriculum Guide's learning goals are still relevant and up-to-date. Remarkably, learning objectives categorized as highly important are also perceived as difficult to achieve. In summary, the insufficient implementation in medical curricula cannot be attributed to the content of the learning goals. The future focus should be on how the WHO learning goals can be implemented in existing curricular courses.

Mobile Sensing with Smart Wearables of the Physical Context of Distance Learning Students to Consider Its Effects on Learning.

Research shows that various contextual factors can have an impact on learning. Some of these factors can originate from the physical learning environment (PLE) in this regard. When learning from home, learners have to organize their PLE by themselves. This paper is concerned with identifying, measuring, and collecting factors from the PLE that may affect learning using mobile sensing. More specifically, this paper first investigates which factors from the PLE can affect distance learning. The results identify nine types of factors from the PLE associated with cognitive, physiological, and affective effects on learning. Subsequently, this paper examines which instruments can be used to measure the investigated factors. The results highlight several methods involving smart wearables (SWs) to measure these factors from PLEs successfully. Third, this paper explores how software infrastructure can be designed to measure, collect, and process the identified multimodal data from and about the PLE by utilizing mobile sensing. The design and implementation of the Edutex software infrastructure described in this paper will enable learning analytics stakeholders to use data from and about the learners' physical contexts. Edutex achieves this by utilizing sensor data from smartphones and smartwatches, in addition to response data from experience samples and questionnaires from learners' smartwatches. Finally, this paper evaluates to what extent the developed infrastructure can provide relevant information about the learning context in a field study with 10 participants. The evaluation demonstrates how the software infrastructure can contextualize multimodal sensor data, such as lighting, ambient noise, and location, with user responses in a reliable, efficient, and protected manner.

Are We There Yet? - A Systematic Literature Review on Chatbots in Education.

Chatbots are a promising technology with the potential to enhance workplaces and everyday life. In terms of scalability and accessibility, they also offer unique possibilities as communication and information tools for digital learning. In this paper, we present a systematic literature review investigating the areas of education where chatbots have already been applied, explore the pedagogical roles of chatbots, the use of chatbots for mentoring purposes, and their potential to personalize education. We conducted a preliminary analysis of 2,678 publications to perform this literature review, which allowed us to identify 74 relevant publications for chatbots' application in education. Through this, we address five research questions that, together, allow us to explore the current state-of-the-art of this educational technology. We conclude our systematic review by pointing to three main research challenges: 1) Aligning chatbot evaluations with implementation objectives, 2) Exploring the potential of chatbots for mentoring students, and 3) Exploring and leveraging adaptation capabilities of chatbots. For all three challenges, we discuss opportunities for future research.

Towards Automatic Collaboration Analytics for Group Speech Data Using Learning Analytics.

Collaboration is an important 21st Century skill. Co-located (or face-to-face) collaboration (CC) analytics gained momentum with the advent of sensor technology. Most of these works have used the audio modality to detect the quality of CC. The CC quality can be detected from simple indicators of collaboration such as total speaking time or complex indicators like synchrony in the rise and fall of the average pitch. Most studies in the past focused on "how group members talk" (i.e., spectral, temporal features of audio like pitch) and not "what they talk". The "what" of the conversations is more overt contrary to the "how" of the conversations. Very few studies studied "what" group members talk about, and these studies were lab based showing a representative overview of specific words as topic clusters instead of analysing the richness of the content of the conversations by understanding the linkage between these words. To overcome this, we made a starting step in this technical paper based on field trials to prototype a tool to move towards automatic collaboration analytics. We designed a technical setup to collect, process and visualize audio data automatically. The data collection took place while a board game was played among the university staff with pre-assigned roles to create awareness of the connection between learning analytics and learning design. We not only did a word-level analysis of the conversations, but also analysed the richness of these conversations by visualizing the strength of the linkage between these words and phrases interactively. In this visualization, we used a network graph to visualize turn taking exchange between different roles along with the word-level and phrase-level analysis. We also used centrality measures to understand the network graph further based on how much words have hold over the network of words and how influential are certain words. Finally, we found that this approach had certain limitations in terms of automation in speaker diarization (i.e., who spoke when) and text data pre-processing. Therefore, we concluded that even though the technical setup was partially automated, it is a way forward to understand the richness of the conversations between different roles and makes a significant step towards automatic collaboration analytics.

Epistemic Network Analyses of Economics Students' Graph Understanding: An Eye-Tracking Study.

Learning to solve graph tasks is one of the key prerequisites of acquiring domain-specific knowledge in most study domains. Analyses of graph understanding often use eye-tracking and focus on analyzing how much time students spend gazing at particular areas of a graph-Areas of Interest (AOIs). To gain a deeper insight into students' task-solving process, we argue that the gaze shifts between students' fixations on different AOIs (so-termed transitions) also need to be included in holistic analyses of graph understanding that consider the importance of transitions for the task-solving process. Thus, we introduced Epistemic Network Analysis (ENA) as a novel approach to analyze eye-tracking data of 23 university students who solved eight multiple-choice graph tasks in physics and economics. ENA is a method for quantifying, visualizing, and interpreting network data allowing a weighted analysis of the gaze patterns of both correct and incorrect graph task solvers considering the interrelations between fixations and transitions. After an analysis of the differences in the number of fixations and the number of single transitions between correct and incorrect solvers, we conducted an ENA for each task. We demonstrate that an isolated analysis of fixations and transitions provides only a limited insight into graph solving behavior. In contrast, ENA identifies differences between the gaze patterns of students who solved the graph tasks correctly and incorrectly across the multiple graph tasks. For instance, incorrect solvers shifted their gaze from the graph to the x-axis and from the question to the graph comparatively more often than correct solvers. The results indicate that incorrect solvers often have problems transferring textual information into graphical information and rely more on partly irrelevant parts of a graph. Finally, we discuss how the findings can be used to design experimental studies and for innovative instructional procedures in higher education.

Beyond Reality-Extending a Presentation Trainer with an Immersive VR Module.

The development of multimodal sensor-based applications designed to support learners with the improvement of their skills is expensive since most of these applications are tailor-made and built from scratch. In this paper, we show how the Presentation Trainer (PT), a multimodal sensor-based application designed to support the development of public speaking skills, can be modularly extended with a Virtual Reality real-time feedback module (VR module), which makes usage of the PT more immersive and comprehensive. The described study consists of a formative evaluation and has two main objectives. Firstly, a technical objective is concerned with the feasibility of extending the PT with an immersive VR Module. Secondly, a user experience objective focuses on the level of satisfaction of interacting with the VR extended PT. To study these objectives, we conducted user tests with 20 participants. Results from our test show the feasibility of modularly extending existing multimodal sensor-based applications, and in terms of learning and user experience, results indicate a positive attitude of the participants towards using the application (PT+VR module).

Dancing Salsa with Machines-Filling the Gap of Dancing Learning Solutions.

Dancing is an activity that positively enhances the mood of people that consists of feeling the music and expressing it in rhythmic movements with the body. Learning how to dance can be challenging because it requires proper coordination and understanding of rhythm and beat. In this paper, we present the first implementation of the Dancing Coach (DC), a generic system designed to support the practice of dancing steps, which in its current state supports the practice of basic salsa dancing steps. However, the DC has been designed to allow the addition of more dance styles. We also present the first user evaluation of the DC, which consists of user tests with 25 participants. Results from the user test show that participants stated they had learned the basic salsa dancing steps, to move to the beat and body coordination in a fun way. Results also point out some direction on how to improve the future versions of the DC.

Detecting Mistakes in CPR Training with Multimodal Data and Neural Networks.

This study investigated to what extent multimodal data can be used to detect mistakes during Cardiopulmonary Resuscitation (CPR) training. We complemented the Laerdal QCPR ResusciAnne manikin with the Multimodal Tutor for CPR, a multi-sensor system consisting of a Microsoft Kinect for tracking body position and a Myo armband for collecting electromyogram information. We collected multimodal data from 11 medical students, each of them performing two sessions of two-minute chest compressions (CCs). We gathered in total 5254 CCs that were all labelled according to five performance indicators, corresponding to common CPR training mistakes. Three out of five indicators, CC rate, CC depth and CC release, were assessed automatically by the ReusciAnne manikin. The remaining two, related to arms and body position, were annotated manually by the research team. We trained five neural networks for classifying each of the five indicators. The results of the experiment show that multimodal data can provide accurate mistake detection as compared to the ResusciAnne manikin baseline. We also show that the Multimodal Tutor for CPR can detect additional CPR training mistakes such as the correct use of arms and body weight. Thus far, these mistakes were identified only by human instructors. Finally, to investigate user feedback in the future implementations of the Multimodal Tutor for CPR, we conducted a questionnaire to collect valuable feedback aspects of CPR training.

[Patient handoffs in undergraduate medical education: A systematic analysis of training needs]. / Patientenübergaben in der Ausbildung? Eine systematische Analyse des Trainingsbedarfs unter Medizinstudierenden.

BACKGROUND: Insufficient handoffs lead to an increase in the risk of complications and malpractice, treatment delays, prolonged hospital stays, costs and patient complaints. The German Society for Anesthesiology and Intensive Care (DGAI) and the European Resuscitation Council (ERC) recommend the implementation of a communication procedure according to the SBAR concept. So far, there have been few curricular requirements in Germany regarding handoffs. METHODS: As part of the EU-funded PATIENT project an online-based cross-sectional needs analysis was conducted in three countries. In Aachen, 237 medical students answered 45 items concerning handoffs in three sections: A: skills (relevance and self-assessment), B: clinical experience (agreement), C: curricula content (presence and relevance). Data was recorded using a Likert scale (0-7). RESULTS: The students rated an adequate handoff performance as highly important (M=6.8; SD: ±0.6) and their own expertise as low (M=4.0; SD: ±1.3). A high training need was identified for writing discharge letters and performing accurate handoffs. The students were aware of the link between adequate handoffs and patient safety (M=6.5; SD: ±0.9). They considered standardized handoff procedures as an important curricular component (M=6.1; SD: ±1.1). From their point of view, the handling of medical errors is underrepresented in the curriculum (61.7 %). CONCLUSION: A need for handoff training was revealed, especially regarding transfers and discharges. Accordingly, learning objectives were determined and training modules developed and integrated into the curriculum in Aachen.

Designing Learning Outcomes for Handoff Teaching of Medical Students Using Group Concept Mapping: Findings From a Multicountry European Study.

PURPOSE: To develop, by consultation with an expert group, agreed learning outcomes for the teaching of handoff to medical students using group concept mapping. METHOD: In 2013, the authors used group concept mapping, a structured mixed-methods approach, applying both quantitative and qualitative measures to identify an expert group's common understanding about the learning outcomes for training medical students in handoff. Participants from four European countries generated and sorted ideas, then rated generated themes by importance and difficulty to achieve. The research team applied multidimensional scaling and hierarchical cluster analysis to analyze the themes. RESULTS: Of 127 experts invited, 45 contributed to the brainstorming session. Twenty-two of the 45 (48%) completed pruning, sorting, and rating phases. They identified 10 themes with which to select learning outcomes and operationally define them to form a basis for a curriculum on handoff training. The themes "Being able to perform handoff accurately" and "Demonstrate proficiency in handoff in workplace" were rated as most important. "Demonstrate proficiency in handoff in simulation" and "Engage with colleagues, patients, and carers" were rated most difficult to achieve. CONCLUSIONS: The study identified expert consensus for designing learning outcomes for handoff training for medical students. Those outcomes considered most important were among those considered most difficult to achieve. There is an urgent need to address the preparation of newly qualified doctors to be proficient in handoff at the point of graduation; otherwise, this is a latent error within health care systems. This is a first step in this process.

Convergence and translation: attitudes to inter-professional learning and teaching of creative problem-solving among medical and engineering students and staff.

BACKGROUND: Healthcare worldwide needs translation of basic ideas from engineering into the clinic. Consequently, there is increasing demand for graduates equipped with the knowledge and skills to apply interdisciplinary medicine/engineering approaches to the development of novel solutions for healthcare. The literature provides little guidance regarding barriers to, and facilitators of, effective interdisciplinary learning for engineering and medical students in a team-based project context. METHODS: A quantitative survey was distributed to engineering and medical students and staff in two universities, one in Ireland and one in Belgium, to chart knowledge and practice in interdisciplinary learning and teaching, and of the teaching of innovation. RESULTS: We report important differences for staff and students between the disciplines regarding attitudes towards, and perceptions of, the relevance of interdisciplinary learning opportunities, and the role of creativity and innovation. There was agreement across groups concerning preferred learning, instructional styles, and module content. Medical students showed greater resistance to the use of structured creativity tools and interdisciplinary teams. CONCLUSIONS: The results of this international survey will help to define the optimal learning conditions under which undergraduate engineering and medicine students can learn to consider the diverse factors which determine the success or failure of a healthcare engineering solution.

The Handover Toolbox: a knowledge exchange and training platform for improving patient care.

BACKGROUND: Safe and effective patient handovers remain a global organisational and training challenge. Limited evidence supports available handover training programmes. Customisable training is a promising approach to improve the quality and sustainability of handover training and outcomes. OBJECTIVE: We present a Handover Toolbox designed in the context of the European HANDOVER Project. The Toolbox aims to support physicians, nurses, individuals in health professions training, medical educators and handover experts by providing customised handover training tools for different clinical needs and contexts. METHODS: The Handover Toolbox uses the Technology Enhanced Learning Design Process (TEL-DP), which encompasses user requirements analysis; writing personas; group concept mapping; analysis of suitable software; plus, minus, interesting rating; and usability testing. TEL-DP is aligned with participatory design approaches and ensures development occurs in close collaboration with, and engagement of, key stakeholders. RESULTS: Application of TEL-DP confirmed that the ideal formats of handover training differs for practicing professionals versus individuals in health profession education programmes. Training experts from different countries differed in their views on the optimal content and delivery of training. Analysis of suitable software identified ready-to-use systems that provide required functionalities and can be further customised to users' needs. Interest rating and usability testing resulted in improved usability, navigation and uptake of the Handover Toolbox. CONCLUSIONS: The design of the Handover Toolbox was based on a carefully led stakeholder participatory design using the TEL-DP approach. The Toolbox supports a customisable learning approach that allows trainers to design training that addresses the specific information needs of the various target groups. We offer recommendations regarding the application of the Handover Toolbox to medical educators.

Mapping and assessing clinical handover training interventions.

BACKGROUND: The literature reveals a patchwork of knowledge about the effectiveness of handover and transfer of care-training interventions, their influence on handover practices and on patient outcomes. We identified a range of training interventions, defined their content, and then proposed practical measures for improving the training effectiveness of handover practices. METHODS: We applied the Group Concept Mapping approach to identify objectively the shared understanding of a group of experts about patient handover training interventions. We collected 105 declarative statements about handover training interventions from an exhaustive literature review, and from structured expert interviews. The statements were then given to 21 healthcare and training design specialists to sort the statements on similarity in meaning, and rate them on their importance and feasibility. RESULTS: We used multidimensional scaling and hierarchical cluster analysis to depict the following seven clusters related to various handover training issues: standardisation, communication, coordination of activities, clinical microsystem care, transfer and impact, training methods and workplace learning. CONCLUSIONS: Ideas on handover training interventions, grouped in thematic clusters, and prioritised on importance and feasibility creates a repository of approaches. This allows healthcare institutions to design and test concrete solutions for improving formal training and workplace learning related to handovers, and addressing informal social learning at the organisational level, with the aim of increasing impact on handover practice and patient outcomes. Measures need to be taken to assure a continuum of handover training interventions from formal training through workplace learning through less formal social learning, and to embed this training in the design of the clinical microsystem.