Beyond active learning: Using 3-Dimensional learning to create scientifically authentic, student-centered classrooms.

In recent years, much of the emphasis for transformation of introductory STEM courses has focused on "active learning", and while this approach has been shown to produce more equitable outcomes for students, the construct of "active learning" is somewhat ill-defined and is often used as a "catch-all" that can encompass a wide range of pedagogical techniques. Here we present an alternative approach for how to think about the transformation of STEM courses that focuses instead on what students should know and what they can do with that knowledge. This approach, known as three-dimensional learning (3DL), emerged from the National Academy's "A Framework for K-12 Science Education", which describes a vision for science education that centers the role of constructing productive causal accounts for phenomena. Over the past 10 years, we have collected data from introductory biology, chemistry, and physics courses to assess the impact of such a transformation on higher education courses. Here we report on an analysis of video data of class sessions that allows us to characterize these sessions as active, 3D, neither, or both 3D and active. We find that 3D classes are likely to also involve student engagement (i.e. be active), but the reverse is not necessarily true. That is, focusing on transformations involving 3DL also tends to increase student engagement, whereas focusing solely on student engagement might result in courses where students are engaged in activities that do not involve meaningful engagement with core ideas of the discipline.

Environmental influences and individual characteristics that affect learner-centered teaching practices.

Research-based teaching practices can improve student learning outcomes in a variety of complex educational environments. The implementation of learner-centered teaching practices in STEM can both benefit from or be constrained by different factors related to individual instructors and the teaching environment. Additionally, we know little of how the instructional climate varies across institutions and how this climate affects teaching practices. Our study sought to examine the relative importance of environmental influences and individual characteristics on learner-centered teaching practices across institutions. We also assessed differences in our study population and departmental climate for 35 US higher education institutions across the country. We found that self-efficacy in teaching and professional development exert a strong influence on faculty teaching practices in biology. While departmental climate did not emerge as a significant predictor of teaching practices, there was consistently low support for teaching, and institution size was negatively correlated with leadership and evaluation of effective teaching. We also found that intensive professional development programs, such as the Faculty Institutes for Reforming Science Teaching IV program, may prepare instructors to teach learner-centered courses in different collegial teaching climates. Our results suggest that through cultivating self-efficacy and participating in iterative professional development, instructors can implement effective teaching practices in a variety of institutional environments.

Early-career faculty practice learner-centered teaching up to 9 years after postdoctoral professional development.

The goal of STEM professional development for teaching is that participants continue to practice what they learn in the long term. However, we do not know if the outcomes are achieved and ultimately persist. We tracked postdoctoral participants from the Faculty Institutes for Reforming Science Teaching (FIRST) IV program into their current positions as early-career biology faculty. We assessed their teaching approaches, practices, and student perceptions of the learning environment over 6 to 9 years after finishing the program. Simultaneously, we evaluated paired faculty in the same departments. We found that professional development outcomes persisted over time and across a career transition. FIRST IV faculty maintained their learner-centered practices and were more learner-centered than their peers. Last, we found that teaching approaches were correlated with teaching practices in all faculty participants. These results provide evidence for the success of the FIRST IV program and the long-term persistence of professional development outcomes.

Characterizing college science instruction: The Three-Dimensional Learning Observation Protocol.

The importance of improving STEM education is of perennial interest, and to this end, the education community needs ways to characterize transformation efforts. Three-dimensional learning (3DL) is one such approach to transformation, in which core ideas of the discipline, scientific practices, and crosscutting concepts are combined to support student development of disciplinary expertise. We have previously reported on an approach to the characterization of assessments, the Three-Dimensional Learning Assessment Protocol (3D-LAP), that can be used to identify whether assessments have the potential to engage students in 3DL. Here we present the development of a companion, the Three-Dimensional Learning Observation Protocol (3D-LOP), an observation protocol that can reliably distinguish between instruction that has potential for engagement with 3DL and instruction that does not. The 3D-LOP goes beyond other observation protocols, because it is intended not only to characterize the pedagogical approaches being used in the instructional environment, but also to identify whether students are being asked to engage with scientific practices, core ideas, and crosscutting concepts. We demonstrate herein that the 3D-LOP can be used reliably to code for the presence of 3DL; further, we present data that show the utility of the 3D-LOP in differentiating between instruction that has the potential to promote 3DL from instruction that does not. Our team plans to continue using this protocol to evaluate outcomes of instructional transformation projects. We also propose that the 3D-LOP can be used to support practitioners in developing curricular materials and selecting instructional strategies to promote engagement in three-dimensional instruction.

Evaluating the extent of a large-scale transformation in gateway science courses.

We evaluate the impact of an institutional effort to transform undergraduate science courses using an approach based on course assessments. The approach is guided by A Framework for K-12 Science Education and focuses on scientific and engineering practices, crosscutting concepts, and core ideas, together called three-dimensional learning. To evaluate the extent of change, we applied the Three-dimensional Learning Assessment Protocol to 4 years of chemistry, physics, and biology course exams. Changes in exams differed by discipline and even by course, apparently depending on an interplay between departmental culture, course organization, and perceived course ownership, demonstrating the complex nature of transformation in higher education. We conclude that while transformation must be supported at all organizational levels, ultimately, change is controlled by factors at the course and departmental levels.

A Problem-Sorting Task Detects Changes in Undergraduate Biological Expertise over a Single Semester.

Calls for undergraduate biology reform share similar goals: to produce people who can organize, use, connect, and communicate about biological knowledge. Achieving these goals requires students to gain disciplinary expertise. Experts organize, access, and apply disciplinary knowledge differently than novices, and expertise is measurable. By asking introductory biology students to sort biological problems, we investigated whether they changed how they organized and linked biological ideas over one semester of introductory biology. We administered the Biology Card Sorting Task to 751 students enrolled in their first or second introductory biology course focusing on either cellular-molecular or organismal-population topics, under structured or unstructured sorting conditions. Students used a combination of superficial, deep, and yet-uncharacterized ways of organizing and connecting biological knowledge. In some cases, this translated to more expert-like ways of organizing knowledge over a single semester, best predicted by whether students were enrolled in their first or second semester of biology and by the sorting condition completed. In addition to illuminating differences between novices and experts, our results show that card sorting is a robust way of detecting changes in novices' biological expertise-even in heterogeneous populations of novice biology students over the time span of a single semester.

Characterizing College Science Assessments: The Three-Dimensional Learning Assessment Protocol.

Many calls to improve science education in college and university settings have focused on improving instructor pedagogy. Meanwhile, science education at the K-12 level is undergoing significant changes as a result of the emphasis on scientific and engineering practices, crosscutting concepts, and disciplinary core ideas. This framework of "three-dimensional learning" is based on the literature about how people learn science and how we can help students put their knowledge to use. Recently, similar changes are underway in higher education by incorporating three-dimensional learning into college science courses. As these transformations move forward, it will become important to assess three-dimensional learning both to align assessments with the learning environment, and to assess the extent of the transformations. In this paper we introduce the Three-Dimensional Learning Assessment Protocol (3D-LAP), which is designed to characterize and support the development of assessment tasks in biology, chemistry, and physics that align with transformation efforts. We describe the development process used by our interdisciplinary team, discuss the validity and reliability of the protocol, and provide evidence that the protocol can distinguish between assessments that have the potential to elicit evidence of three-dimensional learning and those that do not.

Assessing faculty professional development in STEM higher education: Sustainability of outcomes.

We tested the effectiveness of Faculty Institutes for Reforming Science Teaching IV (FIRST), a professional development program for postdoctoral scholars, by conducting a study of program alumni. Faculty professional development programs are critical components of efforts to improve teaching and learning in the STEM (Science, Technology, Engineering, and Mathematics) disciplines, but reliable evidence of the sustained impacts of these programs is lacking. We used a paired design in which we matched a FIRST alumnus employed in a tenure-track position with a non-FIRST faculty member at the same institution. The members of a pair taught courses that were of similar size and level. To determine whether teaching practices of FIRST participants were more learner-centered than those of non-FIRST faculty, we compared faculty perceptions of their teaching strategies, perceptions of environmental factors that influence teaching, and actual teaching practice. Non-FIRST and FIRST faculty reported similar perceptions of their teaching strategies and teaching environment. FIRST faculty reported using active learning and interactive engagement in lecture sessions more frequently compared with non-FIRST faculty. Ratings from external reviewers also documented that FIRST faculty taught class sessions that were learner-centered, contrasting with the teacher-centered class sessions of most non-FIRST faculty. Despite marked differences in teaching practice, FIRST and non-FIRST participants used assessments that targeted lower-level cognitive skills. Our study demonstrated the effectiveness of the FIRST program and the empirical utility of comparison groups, where groups are well matched and controlled for contextual variables (for example, departments), for evaluating the effectiveness of professional development for subsequent teaching practices.

Functional mismatch in a bumble bee pollination mutualism under climate change.

Ecological partnerships, or mutualisms, are globally widespread, sustaining agriculture and biodiversity. Mutualisms evolve through the matching of functional traits between partners, such as tongue length of pollinators and flower tube depth of plants. Long-tongued pollinators specialize on flowers with deep corolla tubes, whereas shorter-tongued pollinators generalize across tube lengths. Losses of functional guilds because of shifts in global climate may disrupt mutualisms and threaten partner species. We found that in two alpine bumble bee species, decreases in tongue length have evolved over 40 years. Co-occurring flowers have not become shallower, nor are small-flowered plants more prolific. We argue that declining floral resources because of warmer summers have favored generalist foraging, leading to a mismatch between shorter-tongued bees and the longer-tubed plants they once pollinated.

Breaking the cycle: future faculty begin teaching with learner-centered strategies after professional development.

The availability of reliable evidence for teaching practices after professional development is limited across science, technology, engineering, and mathematics disciplines, making the identification of professional development "best practices" and effective models for change difficult. We aimed to determine the extent to which postdoctoral fellows (i.e., future biology faculty) believed in and implemented evidence-based pedagogies after completion of a 2-yr professional development program, Faculty Institutes for Reforming Science Teaching (FIRST IV). Postdocs (PDs) attended a 2-yr training program during which they completed self-report assessments of their beliefs about teaching and gains in pedagogical knowledge and experience, and they provided copies of class assessments and video recordings of their teaching. The PDs reported greater use of learner-centered compared with teacher-centered strategies. These data were consistent with the results of expert reviews of teaching videos. The majority of PDs (86%) received video ratings that documented active engagement of students and implementation of learner-centered classrooms. Despite practice of higher-level cognition in class sessions, the items used by the PDs on their assessments of learning focused on lower-level cognitive skills. We attributed the high success of the FIRST IV program to our focus on inexperienced teachers, an iterative process of teaching practice and reflection, and development of and teaching a full course.

Teaching Assistant Professional Development in Biology: Designed for and Driven by Multidimensional Data.

Graduate teaching assistants (TAs) are increasingly responsible for instruction in undergraduate science, technology, engineering, and mathematics (STEM) courses. Various professional development (PD) programs have been developed and implemented to prepare TAs for this role, but data about effectiveness are lacking and are derived almost exclusively from self-reported surveys. In this study, we describe the design of a reformed PD (RPD) model and apply Kirkpatrick's Evaluation Framework to evaluate multiple outcomes of TA PD before, during, and after implementing RPD. This framework allows evaluation that includes both direct measures and self-reported data. In RPD, TAs created and aligned learning objectives and assessments and incorporated more learner-centered instructional practices in their teaching. However, these data are inconsistent with TAs' self-reported perceptions about RPD and suggest that single measures are insufficient to evaluate TA PD programs.

The other half of the story: effect size analysis in quantitative research.

Statistical significance testing is the cornerstone of quantitative research, but studies that fail to report measures of effect size are potentially missing a robust part of the analysis. We provide a rationale for why effect size measures should be included in quantitative discipline-based education research. Examples from both biological and educational research demonstrate the utility of effect size for evaluating practical significance. We also provide details about some effect size indices that are paired with common statistical significance tests used in educational research and offer general suggestions for interpreting effect size measures. Finally, we discuss some inherent limitations of effect size measures and provide further recommendations about reporting confidence intervals.

Onset of alcohol or substance use disorders following treatment for adolescent depression.

OBJECTIVE: This study tested whether positive response to short-term treatment for adolescent major depressive disorder (MDD) would have the secondary benefit of preventing subsequent alcohol use disorders (AUD) or substance use disorders (SUD). METHOD: For 5 years, we followed 192 adolescents (56.2% female; 20.8% minority) who had participated in the Treatment for Adolescents with Depression Study (TADS; TADS Team, 2004) and who had no prior diagnoses of AUD or SUD. TADS initial treatments were cognitive behavior therapy (CBT), fluoxetine alone (FLX), the combination of CBT and FLX (COMB), or clinical management with pill placebo (PBO). We used both the original TADS treatment response rating and a more restrictive symptom count rating. During follow-up, diagnostic interviews were completed at 6- or 12-month intervals to assess onset of AUD or SUD as well as MDD recovery and recurrence. RESULTS: Achieving a positive response to MDD treatment was unrelated to subsequent AUD but predicted a lower rate of subsequent SUD, regardless of the measure of positive response (11.65% vs. 24.72%, or 10.0% vs. 24.5%, respectively). Type of initial MDD treatment was not related to either outcome. Prior to depression treatment, greater involvement with alcohol or drugs predicted later AUD or SUD, as did older age (for AUD) and more comorbid disorders (for SUD). Among those with recurrent MDD and AUD, AUD preceded MDD recurrence in 24 of 25 cases. CONCLUSION: Effective short-term adolescent depression treatment significantly reduces the rate of subsequent SUD but not AUD. Alcohol or drug use should be assessed prior to adolescent MDD treatment and monitored even after MDD recovery.

Forecasting alpine vegetation change using repeat sampling and a novel modeling approach.

Global change affects alpine ecosystems by, among many effects, by altering plant distributions and community composition. However, forecasting alpine vegetation change is challenged by a scarcity of studies observing change in fixed plots spanning decadal-time scales. We present in this article a probabilistic modeling approach that forecasts vegetation change on Niwot Ridge, CO using plant abundance data collected from marked plots established in 1971 and resampled in 1991 and 2001. Assuming future change can be inferred from past change, we extrapolate change for 100 years from 1971 and correlate trends for each plant community with time series environmental data (1971-2001). Models predict a decreased extent of Snowbed vegetation and an increased extent of Shrub Tundra by 2071. Mean annual maximum temperature and nitrogen deposition were the primary a posteriori correlates of plant community change. This modeling effort is useful for generating hypotheses of future vegetation change that can be tested with future sampling efforts.

Multi-decadal changes in tundra environments and ecosystems: synthesis of the International Polar Year-Back to the Future project (IPY-BTF).

Understanding the responses of tundra systems to global change has global implications. Most tundra regions lack sustained environmental monitoring and one of the only ways to document multi-decadal change is to resample historic research sites. The International Polar Year (IPY) provided a unique opportunity for such research through the Back to the Future (BTF) project (IPY project #512). This article synthesizes the results from 13 papers within this Ambio Special Issue. Abiotic changes include glacial recession in the Altai Mountains, Russia; increased snow depth and hardness, permafrost warming, and increased growing season length in sub-arctic Sweden; drying of ponds in Greenland; increased nutrient availability in Alaskan tundra ponds, and warming at most locations studied. Biotic changes ranged from relatively minor plant community change at two sites in Greenland to moderate change in the Yukon, and to dramatic increases in shrub and tree density on Herschel Island, and in subarctic Sweden. The population of geese tripled at one site in northeast Greenland where biomass in non-grazed plots doubled. A model parameterized using results from a BTF study forecasts substantial declines in all snowbeds and increases in shrub tundra on Niwot Ridge, Colorado over the next century. In general, results support and provide improved capacities for validating experimental manipulation, remote sensing, and modeling studies.

Model-based reasoning: using visual tools to reveal student learning.

Using visual models is common in science and should become more common in classrooms. Our research group has developed and completed studies on the use of a visual modeling tool, the Concept Connector. This modeling tool consists of an online concept mapping Java applet that has automatic scoring functions we refer to as Robograder. The Concept Connector enables students in large introductory science courses to visualize their thinking through online model building. The Concept Connector's flexible scoring system, based on tested grading schemes as well as instructor input, has enabled >1,000 physiology students to build maps of their ideas about plant and animal physiology with the guidance of automatic and immediate online scoring of homework. Criterion concept maps developed by instructors in this project contain numerous expert-generated or "correct" propositions connecting two concept words together with a linking phrase. In this study, holistic algorithms were used to test automated methods of scoring concept maps that might work as well as a human grader.

Recovery and recurrence following treatment for adolescent major depression.

CONTEXT: Major depressive disorder in adolescents is common and impairing. Efficacious treatments have been developed, but little is known about longer-term outcomes, including recurrence. OBJECTIVES: To determine whether adolescents who responded to short-term treatments or who received the most efficacious short-term treatment would have lower recurrence rates, and to identify predictors of recovery and recurrence. DESIGN: Naturalistic follow-up study. SETTING: Twelve academic sites in the United States. PARTICIPANTS: One hundred ninety-six adolescents (86 males and 110 females) randomized to 1 of 4 short-term interventions (fluoxetine hydrochloride treatment, cognitive behavioral therapy, their combination, or placebo) in the Treatment for Adolescents With Depression Study were followed up for 5 years after study entry (44.6% of the original Treatment for Adolescents With Depression Study sample). MAIN OUTCOME MEASURES: Recovery was defined as absence of clinically significant major depressive disorder symptoms on the Schedule for Affective Disorders and Schizophrenia for School-Age Children-Present and Lifetime Version interview for at least 8 weeks, and recurrence was defined as a new episode of major depressive disorder following recovery. RESULTS: Almost all participants (96.4%) recovered from their index episode of major depressive disorder during the follow-up period. Recovery by 2 years was significantly more likely for short-term treatment responders (96.2%) than for partial responders or nonresponders (79.1%) (P < .001) but was not associated with having received the most efficacious short-term treatment (the combination of fluoxetine and cognitive behavioral therapy). Of the 189 participants who recovered, 88 (46.6%) had a recurrence. Recurrence was not predicted by full short-term treatment response or by original treatment. However, full or partial responders were less likely to have a recurrence (42.9%) than were nonresponders (67.6%) (P = .03). Sex predicted recurrence (57.0% among females vs 32.9% among males; P = .02). CONCLUSIONS: Almost all depressed adolescents recovered. However, recurrence occurs in almost half of recovered adolescents, with higher probability in females in this age range. Further research should identify and address the vulnerabilities to recurrence that are more common among young women.

Just the facts? Introductory undergraduate biology courses focus on low-level cognitive skills.

Introductory biology courses are widely criticized for overemphasizing details and rote memorization of facts. Data to support such claims, however, are surprisingly scarce. We sought to determine whether this claim was evidence-based. To do so we quantified the cognitive level of learning targeted by faculty in introductory-level biology courses. We used Bloom's Taxonomy of Educational Objectives to assign cognitive learning levels to course goals as articulated on syllabi and individual items on high-stakes assessments (i.e., exams and quizzes). Our investigation revealed the following: 1) assessment items overwhelmingly targeted lower cognitive levels, 2) the cognitive level of articulated course goals was not predictive of the cognitive level of assessment items, and 3) there was no influence of course size or institution type on the cognitive levels of assessments. These results support the claim that introductory biology courses emphasize facts more than higher-order thinking.