A Framework of College Student Buy-in to Evidence-Based Teaching Practices in STEM: The Roles of Trust and Growth Mindset.

Evidence-based teaching practices (EBPs) foster college science, technology, engineering, and mathematics (STEM) students' engagement and performance, yet our knowledge of what contributes to the effectiveness of these practices is less established. We propose a framework that links four social-cognitive variables-students' trust in their instructors, growth mindset, buy-in to instructional practices, and course engagement-to long-standing desired student outcomes of academic performance and intent to persist in science. This framework was tested in classrooms identified as having a high level of EBP implementation with a multi-institutional sample of 2102 undergraduates taught by 14 faculty members. Results indicate that the buy-in framework is a valid representation of college students' learning experiences within EBP contexts overall as well as across underrepresented student groups. In comparison to students' level of growth mindset, students' trust in their instructors was more than twice as predictive of buy-in to how the course was being taught, suggesting that students' views of their instructors are more associated with thriving in a high-EBP course environment than their views of intelligence. This study contributes to the dialogue on transforming undergraduate STEM education by providing a validated student buy-in framework as a lens to understand how EBPs enhance student outcomes.

A Novel Application of the Modified Angoff Method to Rate Case Difficulty in Simulation-Based Research.

INTRODUCTION: Simulation use in research is often limited by controlling for scenario difficulty when using repeated measures. Our study assesses the feasibility of the Modified Angoff Method to reach expert consensus regarding difficulty of medical simulations. We compared scores with participant physiologic stress. METHODS: Emergency medicine physicians with expertise in simulation education were asked to review 8 scenarios and estimate the percentage of resident physicians who would perform all critical actions using the modified Angoff method. A standard deviation (SD) of less than 10% of estimated percentage correct signified consensus. Twenty-five residents then performed the 6 scenarios that met consensus and heart rate variability (HRV) was measured. RESULTS: During round 1, experts rated 4/8 scenarios within a 10% SD for postgraduate year 3 (PGY3) and 3/8 for PGY4 residents. In round 2, 6/8 simulation scenarios were within an SD of 10% points for both years. Intraclass correlation coefficient was 0.84 for PGY3 ratings and 0.89 for PGY4 ratings. A mixed effects analysis of variance showed no significant difference in HRV change from rest to simulation between teams or scenarios. Modified Angoff Score was not a predictor of HRV (multiple R2 = 0.0176). CONCLUSIONS: Modified Angoff ratings demonstrated consensus in quantifying the estimated percentage of participants who would complete all critical actions for most scenarios. Although participant HRV did decrease during the scenarios, we were unable to significantly correlate this with ratings. This modified Angoff method is a feasible approach to evaluate simulation difficulty for educational and research purposes and may decrease the time and resources necessary for scenario piloting.

Communicating Complex STEM Program Evaluation to Diverse Stakeholders.

Visual representations, such as pathway models, are increasingly being used to both communicate higher education science, technology, engineering, and mathematics (STEM) education program evaluation plans as well as accurately represent complex programs and the systems within which the educational programs reside. However, these representations can be overwhelming to audiences that are not familiar with the program's structure or engaged in the evaluation process. The goal of this methods essay is to help both evaluators and discipline-based education researchers improve communication about program evaluation with a variety of stakeholders. We propose a three-stage method for developing progressively less complex visualizations to build affordances that help make the program evaluation process and statements of program impact more meaningful to a wider range of audiences. The creation of less complex visualizations can facilitate understanding by allowing a stakeholder to more easily "see" the structure of the program and thereby may evoke a greater willingness to take action and make meaningful programmatic changes based on strategic evaluation planning. To aid readers, we describe how we modified the Systems Evaluation Protocol (SEP) to develop simplified visualizations when evaluating a long-standing college science faculty development program, the Summer Institutes on Scientific Teaching.

A comparison of technologically mediated and face-to-face help-seeking sources.

BACKGROUND: Current post-secondary school students have access to multiple help-seeking sources. As help-seeking behaviour relates to academic achievement, the provision of preferred help sources would be beneficial to students, instructors, and course designers. AIMS: This study examines whether students prefer and intend to utilize technologically mediated or face-to-face help-seeking sources. SAMPLE: Participants (n = 226) were recruited from two sections of an on campus, introductory, educational psychology class. METHODS: An online survey was distributed containing measures of help-seeking threat, adaptive help-seeking tendencies, avoidant help-seeking tendencies, and the intention to seek help from six sources. Correlations and an ANOVA were calculated to determine whether source preferences differed by self-reported course grade. RESULTS: Help-seeking threat was only negatively associated with sources of help that required face-to-face interaction. Despite the threat, students intended to use face-to-face help-seeking sources more than technologically mediated sources. Students intended to seek help the most before or after class, via email, or during class. Students intended to seek help the least through the discussion board and during online office hours. Higher performing students preferred face-to-face sources, particularly before or after class and during class, more than lower performing students. Lower performing students intended to use mediated sources especially the discussion board and online office hours more than the higher performing students. CONCLUSIONS: The results provide new insights into the help-seeking process and suggestions for instructors when allocating time and classroom resources. Additionally, the study illustrates the need for continual refinement of a help-seeking source classification system.