Environmental influences and individual characteristics that affect learner-centered teaching practices.

Research-based teaching practices can improve student learning outcomes in a variety of complex educational environments. The implementation of learner-centered teaching practices in STEM can both benefit from or be constrained by different factors related to individual instructors and the teaching environment. Additionally, we know little of how the instructional climate varies across institutions and how this climate affects teaching practices. Our study sought to examine the relative importance of environmental influences and individual characteristics on learner-centered teaching practices across institutions. We also assessed differences in our study population and departmental climate for 35 US higher education institutions across the country. We found that self-efficacy in teaching and professional development exert a strong influence on faculty teaching practices in biology. While departmental climate did not emerge as a significant predictor of teaching practices, there was consistently low support for teaching, and institution size was negatively correlated with leadership and evaluation of effective teaching. We also found that intensive professional development programs, such as the Faculty Institutes for Reforming Science Teaching IV program, may prepare instructors to teach learner-centered courses in different collegial teaching climates. Our results suggest that through cultivating self-efficacy and participating in iterative professional development, instructors can implement effective teaching practices in a variety of institutional environments.

Characterizing college science instruction: The Three-Dimensional Learning Observation Protocol.

The importance of improving STEM education is of perennial interest, and to this end, the education community needs ways to characterize transformation efforts. Three-dimensional learning (3DL) is one such approach to transformation, in which core ideas of the discipline, scientific practices, and crosscutting concepts are combined to support student development of disciplinary expertise. We have previously reported on an approach to the characterization of assessments, the Three-Dimensional Learning Assessment Protocol (3D-LAP), that can be used to identify whether assessments have the potential to engage students in 3DL. Here we present the development of a companion, the Three-Dimensional Learning Observation Protocol (3D-LOP), an observation protocol that can reliably distinguish between instruction that has potential for engagement with 3DL and instruction that does not. The 3D-LOP goes beyond other observation protocols, because it is intended not only to characterize the pedagogical approaches being used in the instructional environment, but also to identify whether students are being asked to engage with scientific practices, core ideas, and crosscutting concepts. We demonstrate herein that the 3D-LOP can be used reliably to code for the presence of 3DL; further, we present data that show the utility of the 3D-LOP in differentiating between instruction that has the potential to promote 3DL from instruction that does not. Our team plans to continue using this protocol to evaluate outcomes of instructional transformation projects. We also propose that the 3D-LOP can be used to support practitioners in developing curricular materials and selecting instructional strategies to promote engagement in three-dimensional instruction.

Early-career faculty practice learner-centered teaching up to 9 years after postdoctoral professional development.

The goal of STEM professional development for teaching is that participants continue to practice what they learn in the long term. However, we do not know if the outcomes are achieved and ultimately persist. We tracked postdoctoral participants from the Faculty Institutes for Reforming Science Teaching (FIRST) IV program into their current positions as early-career biology faculty. We assessed their teaching approaches, practices, and student perceptions of the learning environment over 6 to 9 years after finishing the program. Simultaneously, we evaluated paired faculty in the same departments. We found that professional development outcomes persisted over time and across a career transition. FIRST IV faculty maintained their learner-centered practices and were more learner-centered than their peers. Last, we found that teaching approaches were correlated with teaching practices in all faculty participants. These results provide evidence for the success of the FIRST IV program and the long-term persistence of professional development outcomes.

Evaluating the extent of a large-scale transformation in gateway science courses.

We evaluate the impact of an institutional effort to transform undergraduate science courses using an approach based on course assessments. The approach is guided by A Framework for K-12 Science Education and focuses on scientific and engineering practices, crosscutting concepts, and core ideas, together called three-dimensional learning. To evaluate the extent of change, we applied the Three-dimensional Learning Assessment Protocol to 4 years of chemistry, physics, and biology course exams. Changes in exams differed by discipline and even by course, apparently depending on an interplay between departmental culture, course organization, and perceived course ownership, demonstrating the complex nature of transformation in higher education. We conclude that while transformation must be supported at all organizational levels, ultimately, change is controlled by factors at the course and departmental levels.

A Problem-Sorting Task Detects Changes in Undergraduate Biological Expertise over a Single Semester.

Calls for undergraduate biology reform share similar goals: to produce people who can organize, use, connect, and communicate about biological knowledge. Achieving these goals requires students to gain disciplinary expertise. Experts organize, access, and apply disciplinary knowledge differently than novices, and expertise is measurable. By asking introductory biology students to sort biological problems, we investigated whether they changed how they organized and linked biological ideas over one semester of introductory biology. We administered the Biology Card Sorting Task to 751 students enrolled in their first or second introductory biology course focusing on either cellular-molecular or organismal-population topics, under structured or unstructured sorting conditions. Students used a combination of superficial, deep, and yet-uncharacterized ways of organizing and connecting biological knowledge. In some cases, this translated to more expert-like ways of organizing knowledge over a single semester, best predicted by whether students were enrolled in their first or second semester of biology and by the sorting condition completed. In addition to illuminating differences between novices and experts, our results show that card sorting is a robust way of detecting changes in novices' biological expertise-even in heterogeneous populations of novice biology students over the time span of a single semester.

Characterizing College Science Assessments: The Three-Dimensional Learning Assessment Protocol.

Many calls to improve science education in college and university settings have focused on improving instructor pedagogy. Meanwhile, science education at the K-12 level is undergoing significant changes as a result of the emphasis on scientific and engineering practices, crosscutting concepts, and disciplinary core ideas. This framework of "three-dimensional learning" is based on the literature about how people learn science and how we can help students put their knowledge to use. Recently, similar changes are underway in higher education by incorporating three-dimensional learning into college science courses. As these transformations move forward, it will become important to assess three-dimensional learning both to align assessments with the learning environment, and to assess the extent of the transformations. In this paper we introduce the Three-Dimensional Learning Assessment Protocol (3D-LAP), which is designed to characterize and support the development of assessment tasks in biology, chemistry, and physics that align with transformation efforts. We describe the development process used by our interdisciplinary team, discuss the validity and reliability of the protocol, and provide evidence that the protocol can distinguish between assessments that have the potential to elicit evidence of three-dimensional learning and those that do not.

Assessing faculty professional development in STEM higher education: Sustainability of outcomes.

We tested the effectiveness of Faculty Institutes for Reforming Science Teaching IV (FIRST), a professional development program for postdoctoral scholars, by conducting a study of program alumni. Faculty professional development programs are critical components of efforts to improve teaching and learning in the STEM (Science, Technology, Engineering, and Mathematics) disciplines, but reliable evidence of the sustained impacts of these programs is lacking. We used a paired design in which we matched a FIRST alumnus employed in a tenure-track position with a non-FIRST faculty member at the same institution. The members of a pair taught courses that were of similar size and level. To determine whether teaching practices of FIRST participants were more learner-centered than those of non-FIRST faculty, we compared faculty perceptions of their teaching strategies, perceptions of environmental factors that influence teaching, and actual teaching practice. Non-FIRST and FIRST faculty reported similar perceptions of their teaching strategies and teaching environment. FIRST faculty reported using active learning and interactive engagement in lecture sessions more frequently compared with non-FIRST faculty. Ratings from external reviewers also documented that FIRST faculty taught class sessions that were learner-centered, contrasting with the teacher-centered class sessions of most non-FIRST faculty. Despite marked differences in teaching practice, FIRST and non-FIRST participants used assessments that targeted lower-level cognitive skills. Our study demonstrated the effectiveness of the FIRST program and the empirical utility of comparison groups, where groups are well matched and controlled for contextual variables (for example, departments), for evaluating the effectiveness of professional development for subsequent teaching practices.

Functional mismatch in a bumble bee pollination mutualism under climate change.

Ecological partnerships, or mutualisms, are globally widespread, sustaining agriculture and biodiversity. Mutualisms evolve through the matching of functional traits between partners, such as tongue length of pollinators and flower tube depth of plants. Long-tongued pollinators specialize on flowers with deep corolla tubes, whereas shorter-tongued pollinators generalize across tube lengths. Losses of functional guilds because of shifts in global climate may disrupt mutualisms and threaten partner species. We found that in two alpine bumble bee species, decreases in tongue length have evolved over 40 years. Co-occurring flowers have not become shallower, nor are small-flowered plants more prolific. We argue that declining floral resources because of warmer summers have favored generalist foraging, leading to a mismatch between shorter-tongued bees and the longer-tubed plants they once pollinated.

Breaking the cycle: future faculty begin teaching with learner-centered strategies after professional development.

The availability of reliable evidence for teaching practices after professional development is limited across science, technology, engineering, and mathematics disciplines, making the identification of professional development "best practices" and effective models for change difficult. We aimed to determine the extent to which postdoctoral fellows (i.e., future biology faculty) believed in and implemented evidence-based pedagogies after completion of a 2-yr professional development program, Faculty Institutes for Reforming Science Teaching (FIRST IV). Postdocs (PDs) attended a 2-yr training program during which they completed self-report assessments of their beliefs about teaching and gains in pedagogical knowledge and experience, and they provided copies of class assessments and video recordings of their teaching. The PDs reported greater use of learner-centered compared with teacher-centered strategies. These data were consistent with the results of expert reviews of teaching videos. The majority of PDs (86%) received video ratings that documented active engagement of students and implementation of learner-centered classrooms. Despite practice of higher-level cognition in class sessions, the items used by the PDs on their assessments of learning focused on lower-level cognitive skills. We attributed the high success of the FIRST IV program to our focus on inexperienced teachers, an iterative process of teaching practice and reflection, and development of and teaching a full course.

Teaching Assistant Professional Development in Biology: Designed for and Driven by Multidimensional Data.

Graduate teaching assistants (TAs) are increasingly responsible for instruction in undergraduate science, technology, engineering, and mathematics (STEM) courses. Various professional development (PD) programs have been developed and implemented to prepare TAs for this role, but data about effectiveness are lacking and are derived almost exclusively from self-reported surveys. In this study, we describe the design of a reformed PD (RPD) model and apply Kirkpatrick's Evaluation Framework to evaluate multiple outcomes of TA PD before, during, and after implementing RPD. This framework allows evaluation that includes both direct measures and self-reported data. In RPD, TAs created and aligned learning objectives and assessments and incorporated more learner-centered instructional practices in their teaching. However, these data are inconsistent with TAs' self-reported perceptions about RPD and suggest that single measures are insufficient to evaluate TA PD programs.

The other half of the story: effect size analysis in quantitative research.

Statistical significance testing is the cornerstone of quantitative research, but studies that fail to report measures of effect size are potentially missing a robust part of the analysis. We provide a rationale for why effect size measures should be included in quantitative discipline-based education research. Examples from both biological and educational research demonstrate the utility of effect size for evaluating practical significance. We also provide details about some effect size indices that are paired with common statistical significance tests used in educational research and offer general suggestions for interpreting effect size measures. Finally, we discuss some inherent limitations of effect size measures and provide further recommendations about reporting confidence intervals.

Forecasting alpine vegetation change using repeat sampling and a novel modeling approach.

Global change affects alpine ecosystems by, among many effects, by altering plant distributions and community composition. However, forecasting alpine vegetation change is challenged by a scarcity of studies observing change in fixed plots spanning decadal-time scales. We present in this article a probabilistic modeling approach that forecasts vegetation change on Niwot Ridge, CO using plant abundance data collected from marked plots established in 1971 and resampled in 1991 and 2001. Assuming future change can be inferred from past change, we extrapolate change for 100 years from 1971 and correlate trends for each plant community with time series environmental data (1971-2001). Models predict a decreased extent of Snowbed vegetation and an increased extent of Shrub Tundra by 2071. Mean annual maximum temperature and nitrogen deposition were the primary a posteriori correlates of plant community change. This modeling effort is useful for generating hypotheses of future vegetation change that can be tested with future sampling efforts.

Multi-decadal changes in tundra environments and ecosystems: synthesis of the International Polar Year-Back to the Future project (IPY-BTF).

Understanding the responses of tundra systems to global change has global implications. Most tundra regions lack sustained environmental monitoring and one of the only ways to document multi-decadal change is to resample historic research sites. The International Polar Year (IPY) provided a unique opportunity for such research through the Back to the Future (BTF) project (IPY project #512). This article synthesizes the results from 13 papers within this Ambio Special Issue. Abiotic changes include glacial recession in the Altai Mountains, Russia; increased snow depth and hardness, permafrost warming, and increased growing season length in sub-arctic Sweden; drying of ponds in Greenland; increased nutrient availability in Alaskan tundra ponds, and warming at most locations studied. Biotic changes ranged from relatively minor plant community change at two sites in Greenland to moderate change in the Yukon, and to dramatic increases in shrub and tree density on Herschel Island, and in subarctic Sweden. The population of geese tripled at one site in northeast Greenland where biomass in non-grazed plots doubled. A model parameterized using results from a BTF study forecasts substantial declines in all snowbeds and increases in shrub tundra on Niwot Ridge, Colorado over the next century. In general, results support and provide improved capacities for validating experimental manipulation, remote sensing, and modeling studies.

Model-based reasoning: using visual tools to reveal student learning.

Using visual models is common in science and should become more common in classrooms. Our research group has developed and completed studies on the use of a visual modeling tool, the Concept Connector. This modeling tool consists of an online concept mapping Java applet that has automatic scoring functions we refer to as Robograder. The Concept Connector enables students in large introductory science courses to visualize their thinking through online model building. The Concept Connector's flexible scoring system, based on tested grading schemes as well as instructor input, has enabled >1,000 physiology students to build maps of their ideas about plant and animal physiology with the guidance of automatic and immediate online scoring of homework. Criterion concept maps developed by instructors in this project contain numerous expert-generated or "correct" propositions connecting two concept words together with a linking phrase. In this study, holistic algorithms were used to test automated methods of scoring concept maps that might work as well as a human grader.

Just the facts? Introductory undergraduate biology courses focus on low-level cognitive skills.

Introductory biology courses are widely criticized for overemphasizing details and rote memorization of facts. Data to support such claims, however, are surprisingly scarce. We sought to determine whether this claim was evidence-based. To do so we quantified the cognitive level of learning targeted by faculty in introductory-level biology courses. We used Bloom's Taxonomy of Educational Objectives to assign cognitive learning levels to course goals as articulated on syllabi and individual items on high-stakes assessments (i.e., exams and quizzes). Our investigation revealed the following: 1) assessment items overwhelmingly targeted lower cognitive levels, 2) the cognitive level of articulated course goals was not predictive of the cognitive level of assessment items, and 3) there was no influence of course size or institution type on the cognitive levels of assessments. These results support the claim that introductory biology courses emphasize facts more than higher-order thinking.

Learner-centered inquiry in undergraduate biology: positive relationships with long-term student achievement.

We determined short- and long-term correlates of a revised introductory biology curriculum on understanding of biology as a process of inquiry and learning of content. In the original curriculum students completed two traditional lecture-based introductory courses. In the revised curriculum students completed two new learner-centered, inquiry-based courses. The new courses differed significantly from those of the original curriculum through emphases on critical thinking, collaborative work, and/or inquiry-based activities. Assessments were administered to compare student understanding of the process of biological science and content knowledge in the two curricula. More seniors who completed the revised curriculum had high-level profiles on the Views About Science Survey for Biology compared with seniors who completed the original curriculum. Also as seniors, students who completed the revised curriculum scored higher on the standardized Biology Field Test. Our results showed that an intense inquiry-based learner-centered learning experience early in the biology curriculum was associated with long-term improvements in learning. We propose that students learned to learn science in the new courses which, in turn, influenced their learning in subsequent courses. Studies that determine causal effects of learner-centered inquiry-based approaches, rather than correlative relationships, are needed to test our proposed explanation.

1, 2, 3, 4: infusing quantitative literacy into introductory biology.

Biology of the twenty-first century is an increasingly quantitative science. Undergraduate biology education therefore needs to provide opportunities for students to develop fluency in the tools and language of quantitative disciplines. Quantitative literacy (QL) is important for future scientists as well as for citizens, who need to interpret numeric information and data-based claims regarding nearly every aspect of daily life. To address the need for QL in biology education, we incorporated quantitative concepts throughout a semester-long introductory biology course at a large research university. Early in the course, we assessed the quantitative skills that students bring to the introductory biology classroom and found that students had difficulties in performing simple calculations, representing data graphically, and articulating data-driven arguments. In response to students' learning needs, we infused the course with quantitative concepts aligned with the existing course content and learning objectives. The effectiveness of this approach is demonstrated by significant improvement in the quality of students' graphical representations of biological data. Infusing QL in introductory biology presents challenges. Our study, however, supports the conclusion that it is feasible in the context of an existing course, consistent with the goals of college biology education, and promotes students' development of important quantitative skills.