An Empirical Validation of the Regression Point Displacement Design Using Within-Study Comparison Logic: Emerging Possibilities and Cautions.

BACKGROUND: The regression point displacement (RPD) design is a quasi-experiment (QE) that aims to control many threats to internal validity. Though it has existed for several decades, RPD has only recently begun to answer applied research questions in lieu of stronger QEs. OBJECTIVES: Our primary objective was to implement within-study comparison (WSC) logic to create RPD replicates and to determine conditions under which RPD might provide estimates comparable to those found in validating experiments. RESEARCH DESIGN: We utilize three randomized controlled trials (two cluster-level, one individual-level), artificially decomposing or creating cluster structures, to create multiple RPDs. We compare results in each RPD treatment group to a fixed set of control groups to gauge the congruence of these repeated RPD realizations with results found in these three RCTs. RESULTS: RPD's performance was uneven. Using multiple criteria, we found that RPDs successfully predicted the direction of the RCT's intervention effect but inconsistently fell within the .10 SD threshold. A scant 13% of RPD results were statistically significant at either the .05 or .01 alpha-level. RPD results were within the 95% confidence interval of RCTs around half the time, and false negative rates were substantially higher than false positive rates. CONCLUSIONS: RPD consistently underestimates treatment effects in validating RCTs. We analyze reasons for this insensitivity and offer practical suggestions to improve the chances RPD will correctly identify favorable results. We note that the synthetic, "decomposition of cluster RCTs," WSC design represents a prototype for evaluating other QEs.

Evaluating Coincident Relationships Between Obesity Incidence and Normal Weight Incidence From Birth Through Kindergarten for US Children.

PURPOSE: We examine the concurrent relationship between obesity incidence and normal weight status incidence and prevalence in children between 9 months and kindergarten. DESIGN: Multistage, probability sample from the Early Childhood Longitudinal Study-Birth cohort. SETTING: United States. PARTICIPANTS: Representative sample of US preschool children (n = 9950) followed from birth through kindergarten. MEASURES: From direct, anthropometric measures, we reported prevalence and incidence rates across 4 follow-up periods. ANALYSIS: In addition to prevalence and incidence rates, we reported risk ratios based on multiple definitions and estimated predicted probabilities of obesity and normal weight status using clinically meaningful body mass index (BMI)-for-age percentiles. RESULTS: Obesity prevalence (13%-20%) was much smaller than normal weight status prevalence (66%-70%). Lower socioeconomic status, Hispanic, and non-Hispanic black children had greater risk of obesity. During 9 months to kindergarten, obesity incidence decreased two-thirds (15.6%), while normal weight status incidence decreased almost one-half (44.6%). Coincidently, normal weight status incidence (ranged from 23% to 45%) was consistently and substantially higher than obesity incidence (ranged from 5% to 15%). During 4 years to kindergarten, the obesity risk for overweight children was 13 times higher than that for normal weight status children. CONCLUSION: Overall rates of obese and normal weight incidence were substantial at 9 months, trended lower, but remained high through kindergarten. At 4 years to kindergarten, children with relatively high initial BMI were very likely to become obese but far less likely to achieve normal weight status.

Helping Hospitals Improve Patient Centeredness: Assessing the Impact of Feedback Following a Best Practices Workshop.

Regularly reported patient surveys are an important dimension of hospital quality management. This study investigates whether providing hospital staff with interim feedback on patient survey results following a best practices workshop can help hospitals improve patient centeredness. Standardized surveys with consecutive patient samples were administered in accredited breast cancer center (BCC) hospitals in one German state (18 million inhabitants), over a 6-month period, in 2012. Two studies were conducted by applying a combination of regression point displacement (RPD) and interrupted time series (ITS) designs. In Study 1, 2 of the 27 hospitals that had previously participated in a best practices workshop to discuss patient-centeredness issues were randomly chosen and were provided interim feedback of patient survey results and workshop minutes. In Study 2, 4 randomly chosen hospitals of 32 that had not participated in the workshop also received interim feedback but no workshop minutes. Control hospitals in both studies neither received feedback nor workshop minutes. The impact of interim feedback was evaluated by applying graphical assessments and multiple regression analyses. Both graphical assessments (locally weighted scatterplot smoothing (LOESS) lines, RPD plots) suggested an effect of interim feedback. Multiple regression results did not unambiguously support these findings. The suggested design approach may prove particularly useful to assess effects in pilot studies, when resources are not available to conduct a randomized study or when its conduct is contingent on initial, positive evidence.

Failed Warnings: Evaluating the Impact of Academic Probation Warning Letters on Student Achievement.

BACKGROUND: Academic probation is a nearly universal but underresearched policy practiced at most postsecondary institutions. OBJECTIVES: To evaluate the impact of probation warning letters on students' academic performance. RESEARCH DESIGN: Employing the inferentially strong regression discontinuity design, we evaluated the impact of two versions of warning letters (U.S. mail and email), noting their impact on next semester grades. SUBJECTS: Probation and nonprobation students at a large, Midwestern college enrolled during two, successive fall-winter semester pairs (n > 17,000, for each pair). MEASURES: Fall and winter grade point averages (GPAs) were identified for each individual student in the study sample. RESULTS: Using both parametric and nonparametric analyses, we found that neither delivery method, paper or electronic, had a consistent, significant impact on subsequent GPA or odds of a GPA ≥ 2.0 during the next semester. Four of the eight measures of effect for GPA were small and positive (0.02-0.15; one significant positive outcome), and four were negative (-0.02 to -0.08; one significant negative result). A separate set of analyses that excluded students who took a single course led to further inconsistency in results. Supplementary analyses that excluded students scoring far from the cut-point yielded results consistent with a no-difference conclusion. Our findings also indicated that, after being placed on probation, only a small percentage of students were able to avoid movement to the next stage of academic sanction. CONCLUSIONS: Warning letters notifying students of probation status, lacking a staff-focused intervention, had little impact on academic performance.

Early childhood healthy and obese weight status: potentially protective benefits of breastfeeding and delaying solid foods.

The aim of this study was to assess the relationship between breastfeeding and postponing introduction to solid food (SF) on children's obesity and healthy weight status (WS), at 2 and 4 years. Drawing upon a nationally representative sample of children from the Early Childhood Longitudinal Study-Birth Cohort, we estimated the magnitude of the relationship between children's WS and early feeding practices. Contingency tables and multinomial logistic regression were used to analyze obese and healthy WS for breastfed and never breastfed children and examine three timing categories for SF introduction. With both percentages and odds, breastfeeding and delaying introduction to SF until 4 months were associated with lower obesity rates and higher, healthy WS rates (typically 5-10%). Analyses of feeding practice combinations revealed that when children were not breastfed, obesity odds decreased when SF introduction was postponed until 4 months. Obesity odds were further reduced when SF delay was combined with breastfeeding. Consistent increases in healthy WS were also observed. Benefits were stable across both follow-up periods. Breastfeeding and delaying complementary foods yielded consistently and substantially lower likelihood of obesity and greater probability of healthy WS. Health policies targeting early feeding practices represent promising interventions to decrease preschool obesity and promote healthy WS.

Evaluating effects of developmental education for college students using a regression discontinuity design.

BACKGROUND: Annually, American colleges and universities provide developmental education (DE) to millions of underprepared students; however, evaluation estimates of DE benefits have been mixed. OBJECTIVES: Using a prototypic exemplar of DE, our primary objective was to investigate the utility of a replicative evaluative framework for assessing program effectiveness. RESEARCH DESIGN: Within the context of the regression discontinuity (RD) design, this research examined the effectiveness of a DE program for five, sequential cohorts of first-time college students. Discontinuity estimates were generated for individual terms and cumulatively, across terms. SUBJECTS: Participants were 3,589 first-time community college students. MEASURES: DE program effects were measured by contrasting both college-level English grades and a dichotomous measure of pass/fail, for DE and non-DE students. RESULTS: Parametric and nonparametric estimates of overall effect were positive for continuous and dichotomous measures of achievement (grade and pass/fail). The variability of program effects over time was determined by tracking results within individual terms and cumulatively, across terms. Applying this replication strategy, DE's overall impact was modest (an effect size of approximately .20) but quite consistent, based on parametric and nonparametric estimation approaches. A meta-analysis of five RD results yielded virtually the same estimate as the overall, parametric findings. Subset analysis, though tentative, suggested that males benefited more than females, while academic gains were comparable for different ethnicities. CONCLUSION: The cumulative, within-study comparison, replication approach offers considerable potential for the evaluation of new and existing policies, particularly when effects are relatively small, as is often the case in applied settings.

U.S. children's preschool weight status trajectories: patterns from 9-month, 2-year, and 4-year Early Childhood Longitudinal Study-Birth cohort data.

PURPOSE: To track changes in U.S. children's early weight status (normal, at-risk, or obese) from infancy through preschool. DESIGN: Patterns of change in individual children's weight status are established using three time points from the Early Childhood Longitudinal Study-Birth cohort. SETTING: United States. SUBJECTS: Nationally representative sample of U.S. preschool children surveyed at 9 months (n = 8900), with follow-up at 2 years (n = 7500) and preschool (n = 7000). MEASURES: Individual children's weight status was determined from Centers for Disease Control and Prevention growth charts at multiple time periods. ANALYSIS: Layered, categorical data analysis of changes in weight status patterns at three time periods during infancy through preschool. RESULTS: Young children whose early weight status was normal tend to retain a normal weight status and not to develop an unfavorable status (at-risk, obese). In contrast, children who were obese at an early age were more likely to have an at-risk or obese weight category at a later age. Overall, both favorable and unfavorable early weight statuses were highly associated with subsequent status at preschool. CONCLUSION: Early weight status can provide important information relevant for early entry point prevention and treatment of childhood obesity.

Young children's weight trajectories and associated risk factors: results from the Early Childhood Longitudinal Study-Birth Cohort.

PURPOSE: To estimate the prevalence of U.S. children's overweight risk and obesity at age 9 months and at age 2 years, to assess weight changes between the two periods, and to examine relationships between weight status (i.e., normal, at risk, or obese) changes and demographic variables. DESIGN: Analyses of children's early weight trajectories and related demographic characteristics from the Early Childhood Longitudinal Study-Birth Cohort (ECLS-B) are presented. SETTING: United States. SUBJECTS: The 9-month-old (n  =  8900) and 2-year-old (n  =  7500) ECLS-B waves were used to generate nationally representative estimates of obese and at-risk children born in 2001. MEASURES: Measures included child's sex, race/ethnicity, socioeconomic status, community locale, geographic region, and weight status. ANALYSIS: Logistic and multinomial logistic regression models were used to determine the odds of children's demographic characteristics being related to weight persistence, loss, or gain. RESULTS: Approximately one-third of U.S. children were either at risk or obese at 9 months (31.9%) and at 2 years (34.3%). Some children were at greater risk (e.g., Hispanics and low socioeconomic status children), while others had reduced risk (e.g., females and Asian/Pacific Islanders). Additional results from two trajectory models generally corroborated patterns of status change due to weight gain. CONCLUSIONS: Between age 9 months and age 2 years, U.S. children consistently moved toward less desirable weight status. Obesity risk was not uniform across demographic subgroups, suggesting that health policy might focus on those children at greatest risk.