Immersive distance simulation: Exploring the educational impact of stereoscopic extended reality (XR) video in remote learning environments.

What was the educational challenge?There is a growing need for healthcare simulation options when local expertise or resources are not available. To connect instructors with remote learners, current options for distance simulation are typically limited to videoconferencing on desktop computers or mobile devices, which may not fully capture the complexity of clinical scenarios.What was the solution?Extended reality (XR) technology may provide a more immersive and realistic distance healthcare simulation experience compared to traditional videoconferencing options. Unlike computer- or phone-based video calls, stereoscopic video in XR provides a sense of depth that may increase spatial understanding and engagement in distance simulation.How was the solution implemented?We investigated the impact of XR for synchronous distance simulation compared to traditional desktop-based videoconferencing in Emergency Medicine (EM) resident training for an obstetrical emergency. A randomized controlled experiment was conducted with half of the residents using XR and half using computers to participate in the simulation.What lessons were learned that are relevant to a wider global audience?There was an unanticipated interaction between postgraduate year and condition such that performance in the XR condition was superior for first year residents, while this was reversed for more experienced residents. This indicates that the benefits of XR might be dependent on participant characteristics, such as learner level.What are the next steps?We plan to extend this research to clarify characteristics of learners and tasks that are important determinants of differences in outcomes between stereoscopic XR versus traditional videoconference displays.

Comparing Capabilities of Simulation Modalities for Training Combat Casualty Care: Perspectives of Combat Medics.

INTRODUCTION: Combat casualty care requires learning a complex set of skills to treat patients in challenging situations, including resource scarce environments, multiple casualty incidents, and care under fire. To train the skills needed to respond efficiently and appropriately to these diverse conditions, instructors employ a wide array of simulation modalities. Simulation modalities for medical training include manikins, task trainers, standardized patient actors (i.e., role players), computer or extended reality simulations (e.g., virtual reality, augmented reality), cadavers, and live tissue training. Simulation modalities differ from one another in multiple attributes (e.g., realism, availability). The purpose of this study was to compare capabilities across simulation modalities for combat casualty care from the perspective of experienced military medics. MATERIALS AND METHODS: To provide a more complete understanding of the relative merits and limitations of modalities, military combat medics (N = 33) were surveyed on the capabilities of simulation modalities during a 5-day technical experimentation event where they observed medical simulations from industry developers. The survey asked them to rate each of eleven modalities on each of seven attributes. To elicit additional context for the strengths, limitations, and unique considerations of using each modality, we also collected open-ended comments to provide further insight on when and how to use specific simulation modalities. RESULTS: Results showed differences among the simulation modalities by attribute. Cadavers, role play, moulage, and live tissue all received high ratings on two or more attributes. However, there was no modality that was rated uniformly superior to the others. Instead, modalities appear to have unique strengths and limitations depending on the training context and objectives. For example, cadavers were seen as highly realistic, but not very reusable. CONCLUSIONS: The study furthers our understanding of simulation modalities for medical training by providing insight from combat medics on the benefits, limitations, and considerations for implementing different modalities depending on the training context. These results may be helpful to instructors in selecting modalities for their programs.

Capturing the underlying distribution in meta-analysis: Credibility and tolerance intervals.

Tolerance intervals provide a bracket intended to contain a percentage (e.g., 80%) of a population distribution given sample estimates of the mean and variance. In random-effects meta-analysis, tolerance intervals should contain researcher-specified proportions of underlying population effect sizes. Using Monte Carlo simulation, we investigated the coverage for five relevant tolerance interval estimators: the Schmidt-Hunter credibility intervals, a prediction interval, two content tolerance intervals adapted to meta-analysis, and a bootstrap tolerance interval. None of the intervals contained the desired percentage of coverage at the nominal rates in all conditions. However, the prediction worked well unless the number of primary studies was small (<30), and one of the content tolerance intervals approached nominal levels with small numbers (<20) of primary studies. The bootstrap tolerance interval achieved near nominal coverage if there were sufficient numbers of primary studies (30+) and large enough sample sizes (N ≅ 70) in the included primary studies, although it slightly exceeded nominal coverage with large numbers of large-sample primary studies. Next, we showed the results of applying the intervals to real data using a set of previously published analyses and provided suggestions for practice. Tolerance intervals incorporate error of estimation into the construction of proper brackets for fractions of population true effects. In many contexts, such intervals approach the desired nominal levels of coverage.

Quantifying uncertainty in the meta-analytic lower bound estimate.

In meta-analyses, it is customary to compute a confidence interval for the overall mean effect (ρ̄ or δ̄), but not for the underlying standard deviation (τ) or the lower bound of the credibility value (90%CV), even though the latter entities are often as important to the interpretation as is the overall mean. We introduce 2 methods of computing confidence intervals for the lower bound (Lawless and bootstrap). We compare both methods using 3 lower bound estimators (Schmidt-Hunter, Schmidt-Hunter with k correction, and Morris/Hedges, labeled HOVr/HOVd) in 2 Monte Carlo studies (1 for correlations and 1 for standardized mean differences) and illustrate their application to published meta-analyses. For correlations, we found that HOVr bootstrap confidence intervals yielded coverage greater than 90% across a wide variety of conditions provided that there were at least 25 studies. For the standardized mean difference, all 3 methods produced acceptable results using the bootstrap for the lower bound confidence interval provided that there were at least 20 studies with an average sample size of at least 50. When the number of studies was small (k = 8 or 10), coverage was less than 90% and confidence intervals were very wide. Even with larger numbers of studies, if there was indirect range restriction coupled with a small underlying between-studies variance, the between-studies variance was poorly estimated and coverage of the lower bound suffered. We provide software to allow meta-analysts to compute bootstrap confidence intervals for the estimators described in the article. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Direct Aspiration versus Stent Retriever Thrombectomy for Acute Stroke: A Systematic Review and Meta-Analysis in 9127 Patients.

BACKGROUND: The two most common approaches to thrombectomy of emergent large vessel occlusion (direct aspiration and primary stent retriever thrombectomy) have been extensively studied; however, the detailed benefit and risk comparison is largely unknown. OBJECTIVE: To conduct a systematic review and meta-analysis to compare radiographic and clinical outcomes between the use of primary stent retrievers and direct aspiration in management of acute ischemic stroke. METHODS: PubMed database was searched for studies between September 1, 2012 and December 31, 2017 with acute ischemic stroke patients. RESULTS: We identified 64 studies with 6875 patients in the primary stent retriever group and 25 studies with 2252 patients in the aspiration group. Primary aspiration alone, without the need of rescue stent retriever devices within the aspiration cohort, was performed in 65% of 2252 patients. There was no difference in the distribution of emergent large vessel occlusion based on occlusion site, age, baseline National Institutes of Health Stroke Scale, or the use of intravenous tPA (P = .19, .051, .23, and .093, respectively). Successful recanalization rates, defined as thrombolysis in cerebral Infarction 2b/3, were significantly higher in the aspiration group than the primary stent retriever group (89% versus 80%, P < .0001). No significant difference in good clinical outcome, defined as modified Rankin scale 0-2 (aspiration 52% versus stent 48%, P = .13), symptomatic intracerebral hemorrhage (aspiration 5.6% versus stent 7.2%, P = .07), and mortality at 3 months (aspiration 15% versus stent 19%, P = .10). CONCLUSIONS: Both aspiration-first (including the subsequent use of stent retriever) and primary stent retriever thrombectomy approaches are equally effective in achieving good clinical outcomes. Our study suggests that direct aspiration with or without subsequent use of stent retriever is a safe and effective alternative to primary stent retriever in acute ischemic stroke.

Estimating the Reliability of Nontechnical Skills in Medical Teams.

OBJECTIVE: To assess the reliability of measurement of assessments of nontechnical skills in medical teams. BACKGROUND: There are several scales for assessing nontechnical skills in medical teams. In this critique, we describe the reliability of such assessments and discuss factors that may influence the obtained estimate of reliability for such scales. METHOD: A review of the literature yielded 28 reliability estimates, mostly intraclass correlations. We analyzed the methods of scale development and reliability estimation for factors that are likely to influence the resulting estimate, including aspects of the judges, the targets, what is observed, and the kind of reliability coefficient. RESULTS: Typical reliability was approximately 0.7, but estimates were quite varied in magnitude, from quite low (0.3) to quite high (0.9). Thus, highly reliable assessment is possible, but rarely achieved. Factors that are likely to influence the magnitude of reliability include the number of assessors, the participants' range in skill level, and the observational process. Most studies do not present enough information to make the estimates comparable from one study to the next. CONCLUSIONS: Although nontechnical skills can be measured reliably under some circumstances, it is not clear from the existing literature that one scale provides more reliable data than another. We recommend several practices that would help readers to make informed decisions about which scales are most useful for their purposes.

Cross-hairs: a scatterplot for meta-analysis in R.

We describe a meta-analytic scatterplot that indicates precision of points for two variables paired within studies; this is equivalent in form to a 'cross-hairs' plot used to portray specificity and sensitivity in diagnostic testing. At the user's discretion, the plot also displays boxplots for each of the X and Y variable distributions, means for each of the variables, and the correlation between the two. The cross-hairs may be suppressed for dense point clouds. The program is written in R, so it can be modified by the user and can serve as a companion to existing meta-analysis programs. Some of the program's novel uses are described and illustrated with (1) independent effect sizes, (2) dependent effect sizes, and (3) shrunken estimates. Copyright © 2016 John Wiley & Sons, Ltd.

Medical student quality-of-life in the clerkships: a scale validation study.

Many aspects of medical school are stressful for students. To empirically assess student reactions to clerkship programs, or to assess efforts to improve such programs, educators must measure the overall well-being of the students reliably and validly. The purpose of the study was to develop and validate a measure designed to achieve these goals. The authors developed a measure of quality of life for medical students by sampling (public domain) items tapping general happiness, fatigue, and anxiety. A quality-of-life scale was developed by factor analyzing responses to the items from students in two different clerkships from 2005 to 2008. Reliability was assessed using Cronbach's alpha. Validity was assessed by factor analysis, convergence with additional theoretically relevant scales, and sensitivity to change over time. The refined nine-item measure is a Likert scaled survey of quality-of-life items comprised of two domains: exhaustion and general happiness. The resulting scale demonstrated good reliability and factorial validity at two time points for each of the two samples. The quality-of-life measure also correlated with measures of depression and the amount of sleep reported during the clerkships. The quality-of-life measure appeared more sensitive to changes over time than did the depression measure. The measure is short and can be easily administered in a survey. The scale appears useful for program evaluation and more generally as an outcome variable in medical educational research.

Malleability of weight-biased attitudes and beliefs: a meta-analysis of weight bias reduction interventions.

Weight bias exists across many important life domains, necessitating interventions designed to reduce weight-biased attitudes and beliefs. Though the effectiveness of weight bias interventions has been questioned, to our knowledge no meta-analysis of these interventions has been conducted. This meta-analysis evaluated the impact of weight bias interventions on weight-biased attitudes and beliefs and explored potential moderators. Interventions were eligible if they used an adult sample and a validated measure of weight-biased attitudes, which resulted in the inclusion of 30 studies represented in 29 articles. A random effects approach using inverse weights resulted in a mean effect size estimate of g=-0.33 (lower scores indicate less weight bias) for both attitudes and beliefs. Intervention type, publication type, and population type were not significant moderators but demonstrated noteworthy trends. Results reveal a small, positive effect of weight bias interventions on weight-biased attitudes and beliefs and provide useful information for future interventions.

FLS skill acquisition: a comparison of blocked vs interleaved practice.

INTRODUCTION: Fundamentals of Laparoscopic Surgery (FLS) certification is required for general surgery. The recommended practice for learning FLS is to practice tasks one at a time until proficient (blocked practice). Learning theory suggests that interleaved practice, a method in which tasks are rotated rather than learned one at a time, may result in superior learning. METHOD: Residents were randomized into 1 of 2 groups: blocked practice or interleaved practice. We compared the performance of residents across groups over 20 trials of each of 4 FLS tasks (peg transfer, pattern cut, extracorporeal suture, and intracorporeal suture). Four weeks later, participants returned to the laboratory and completed 2 additional trials of each of the 4 tasks. RESULTS: Performance on each of the tasks improved with increased practice. The interleaved group showed significantly better performance on the peg transfer task; trends favoring the interleaved group resulted for the other tasks. Standardized mean differences in favor of the interleaved group were substantial both at the end of practice and at follow-up (with the exception of the pattern cut). CONCLUSION: Interleaved practice appears to have advantages over blocked practice in developing and retaining FLS skills. We encourage others to experiment with the method to confirm our findings.

Bayesian meta-analysis of coefficient alpha.

The current paper describes and illustrates a Bayesian approach to the meta-analysis of coefficient alpha. Alpha is the most commonly used estimate of the reliability or consistency (freedom from measurement error) for educational and psychological measures. The conventional approach to meta-analysis uses inverse variance weights to combine information from independent studies to provide an overall estimate. The Bayesian approach provides similar estimates to the conventional approach if a diffuse prior is used. However, the Bayesian approach also provides 'shrunken' local estimates of reliability in each context. The amount of shrinkage depends upon both the variability in the underlying populations and the sampling variance of the local estimates. Advantages of the approach are the estimation of individual studies adjusted for sampling error and the application of meta-analytic results to new local studies in which the local study 'borrows strength' from the meta-analysis. The ability to borrow strength for the new local studies is particularly useful in applied work in which the estimate of the local parameter is of primary interest. The approach is illustrated by the analysis of studies of the reliability of the General Ethnicity Questionnaire - Abridged, a measure of identification with the culture of one's heritage and the culture of one's host country. Copyright © 2013 John Wiley & Sons, Ltd.

Surgical interest and surgical match for third-year students: results of a prospective multivariate longitudinal cohort study.

BACKGROUND: Numerous factors have been linked to surgical career choice, including the quality of third-year surgical clerkship. The vast majority of studies also selectively evaluate one or only a few variables that link to surgical career choice, so relative impact cannot be assessed. This study simultaneously evaluates the majority of variables linked to surgical career choice in previous research so that the relative contributions of each of these variables with respect to surgical career choice can be determined. STUDY DESIGN: Surveys before, during, and after the third-year surgical clerkship included student demographics, background, and values (eg, importance of money, controllable lifestyle), and student reactions to the third-year surgical clerkship. The dependent variables in this study included interest in surgery at the beginning and end of the clerkship and matching into a surgical residency. RESULTS: Both univariate and multivariate analyses generally supported findings in the literature, but the strengths of these associations reported previously might not have been accurate. In this study, the surgical resident match odds ratio for students starting the clerkship already knowing they wanted to be a surgeon was 22.46; the next highest associations were 4.65 and 3.37, which corresponded to earlier exposure to a surgical specialty and earlier exposure to general surgery, respectively. Differences in career choice for general surgeons and surgical specialists were also explored. CONCLUSIONS: Although the experience of the clerkship is related to career choice, the largest impact of the clerkship is for those already interested in a surgical career. Interest in a surgical career largely develops before the third-year clerkship. Implications of the results for recruiting greater numbers of students into surgical careers are discussed.

Publication bias in psychological science: prevalence, methods for identifying and controlling, and implications for the use of meta-analyses.

The issue of publication bias in psychological science is one that has remained difficult to address despite decades of discussion and debate. The current article examines a sample of 91 recent meta-analyses published in American Psychological Association and Association for Psychological Science journals and the methods used in these analyses to identify and control for publication bias. Of the 91 studies analyzed, 64 (70%) made some effort to analyze publication bias, and 26 (41%) reported finding evidence of bias. Approaches to controlling publication bias were heterogeneous among studies. Of these studies, 57 (63%) attempted to find unpublished studies to control for publication bias. Nonetheless, those studies that included unpublished studies were just as likely to find evidence for publication bias as those that did not. Furthermore, authors of meta-analyses themselves were overrepresented in unpublished studies acquired, as compared with published studies, suggesting that searches for unpublished studies may increase rather than decrease some sources of bias. A subset of 48 meta-analyses for which study sample sizes and effect sizes were available was further analyzed with a conservative and newly developed tandem procedure of assessing publication bias. Results indicated that publication bias was worrisome in about 25% of meta-analyses. Meta-analyses that included unpublished studies were more likely to show bias than those that did not, likely due to selection bias in unpublished literature searches. Sources of publication bias and implications for the use of meta-analysis are discussed.

Psychometrics of Mayer-Salovey-Caruso Emotional Intelligence Test (MSCEIT) scores.

A sample of 183 medical students completed the Mayer-Salovey-Caruso Emotional Intelligence Test (MSCEIT V2.0). Scores on the test were examined for evidence of reliability and factorial validity. Although Cronbach's alpha for the total scores was adequate (.79), many of the scales had low internal consistency (scale alphas ranged from .34 to .77; median = .48). Previous factor analyses of the MSCEIT are critiqued and the rationale for the current analysis is presented. Both confirmatory and exploratory factor analyses of the MSCEIT item parcels are reported. Pictures and faces items formed separate factors rather than loading on a Perception factor. Emotional Management appeared as a factor, but items from Blends and Facilitation failed to load consistently on any factor, rendering factors for Emotional Understanding and Emotional Facilitation problematic.

A systematic review of the reliability of objective structured clinical examination scores.

CONTEXT: The objective structured clinical examination (OSCE) is comprised of a series of simulations used to assess the skill of medical practitioners in the diagnosis and treatment of patients. It is often used in high-stakes examinations and therefore it is important to assess its reliability and validity. METHODS: The published literature was searched (PsycINFO, PubMed) for OSCE reliability estimates (coefficient alpha and generalisability coefficients) computed either across stations or across items within stations. Coders independently recorded information about each study. A meta-analysis of the available literature was computed and sources of systematic variance in estimates were examined. RESULTS: A total of 188 alpha values from 39 studies were coded. The overall (summary) alpha across stations was 0.66 (95% confidence interval [CI] 0.62-0.70); the overall alpha within stations across items was 0.78 (95% CI 0.73-0.82). Better than average reliability was associated with a greater number of stations and a higher number of examiners per station. Interpersonal skills were evaluated less reliably across stations and more reliably within stations compared with clinical skills. CONCLUSIONS: Overall scores on the OSCE are often not very reliable. It is more difficult to reliably assess communication skills than clinical skills when considering both as general traits that should apply across situations. It is generally helpful to use two examiners and large numbers of stations, but some OSCEs appear more reliable than others for reasons that are not yet fully understood.

Appearance-related teasing, body dissatisfaction, and disordered eating: A meta-analysis.

A meta-analysis was conducted to determine the relationship between appearance and weight-based teasing and three outcome measures: body dissatisfaction, restrictive eating, and bulimic behaviors. Four meta-analyses were conducted. Fifty effect sizes (N=10,618) resulted in a moderate effect size of .39 for the relationship between weight teasing and body dissatisfaction; 24 effect sizes (N=7190) resulted in an effect size of .32 for the relationship between appearance teasing and body dissatisfaction; 20 effect sizes (N=4792) resulted in an effect size of .35 for the relationship between weight teasing and dietary restraint; and 22 effect sizes (N=5091) resulted in an effect size of .36 for the relationship between weight teasing and bulimic behaviors. Significant moderators that emerged were teasing measure type, publication type, study type, age group, and gender. The findings offer further support for the inclusion of strategies in body image and eating disorders' prevention and intervention programs that focus on handling negative, appearance-related commentary.

A Meta-Meta-Analysis: Empirical Review of Statistical Power, Type I Error Rates, Effect Sizes, and Model Selection of Meta-Analyses Published in Psychology.

This article uses meta-analyses published in Psychological Bulletin from 1995 to 2005 to describe meta-analyses in psychology, including examination of statistical power, Type I errors resulting from multiple comparisons, and model choice. Retrospective power estimates indicated that univariate categorical and continuous moderators, individual moderators in multivariate analyses, and tests of residual variability within individual levels of categorical moderators had the lowest and most concerning levels of power. Using methods of calculating power prospectively for significance tests in meta-analysis, we illustrate how power varies as a function of the number of effect sizes, the average sample size per effect size, effect size magnitude, and level of heterogeneity of effect sizes. In most meta-analyses many significance tests were conducted, resulting in a sizable estimated probability of a Type I error, particularly for tests of means within levels of a moderator, univariate categorical moderators, and residual variability within individual levels of a moderator. Across all surveyed studies, the median effect size and the median difference between two levels of study level moderators were smaller than Cohen's (1988) conventions for a medium effect size for a correlation or difference between two correlations. The median Birge's (1932) ratio was larger than the convention of medium heterogeneity proposed by Hedges and Pigott (2001) and indicates that the typical meta-analysis shows variability in underlying effects well beyond that expected by sampling error alone. Fixed-effects models were used with greater frequency than random-effects models; however, random-effects models were used with increased frequency over time. Results related to model selection of this study are carefully compared with those from Schmidt, Oh, and Hayes (2009), who independently designed and produced a study similar to the one reported here. Recommendations for conducting future meta-analyses in light of the findings are provided.

What have the difference scores not been telling us? A critique of the use of self--ideal discrepancy in the assessment of body image and evaluation of an alternative data-analytic framework.

Difference scores are often used as a means of assessing body image satisfaction using silhouette scales. Unfortunately, difference scores suffer from numerous potential methodological problems, including reduced reliability, ambiguity, confounded effects, untested constraints, and dimensional reduction. In this article, the methodological problems are outlined and an alternative framework is discussed. The alternative consists of a minimum of testing the constraints implied by the difference score model, and at most evaluating more exact body image hypotheses by incorporating nonlinear terms in a regression and testing features of the response surface. Two empirical examples are used to illustrate the utility of these methods. The first example uses both polynomial regression and response surface methods to examine eating disorder outcomes, whereas the second example uses polynomial regression to examine the outcomes related to muscle dysmorphia. Directions for future research related to assessment of body image are discussed.

Evaluation of an error-reduction training program for surgical residents.

PURPOSE: To reduce errors in surgery using a resident training program based on a taxonomy that highlights three kinds of errors: judgment, inattention to detail, and problem understanding. METHOD: The training program module at the University of South Florida incorporated a three-item situational judgment test, video training (which included a lecture and behavior modeling), and role-plays (in which residents participated and received feedback from faculty). Two kinds of outcome data were collected from 33 residents during 2006-2007: (1) behaviors during the training and (2) on-the-job surgical complication records 12 months before and 6 months after training. For the data collected during training, participants were assigned to a condition (19 video condition, 13 control condition); for the data collected on the job, an interrupted time series design was used. RESULTS: Data from 32 residents were analyzed (one resident's data were excluded). One of the situational judgment items improved significantly over time (d = 0.45); the other two did not (d = 0.36, 0.25). Surgical complications and errors decreased over the course of the study (the correlation between complications and time in months was r = -0.47, for errors and time, r = -0.55). Effects of video behavior modeling on specific errors measured during role-plays were not significant (effect sizes for binary outcomes were phi = -0.05 and phi = 0.01, and for continuous outcomes, d ranged from -0.02 to 0.34). CONCLUSIONS: The training seemed to reduce errors in surgery, but the training had little effect on the specific kinds of errors targeted during training.