Publication bias, statistical power and reporting practices in the Journal of Sports Sciences: potential barriers to replicability.

Two factors that decrease the replicability of studies in the scientific literature are publication bias and studies with underpowered desgins. One way to ensure that studies have adequate statistical power to detect the effect size of interest is by conducting a-priori power analyses. Yet, a previous editorial published in the Journal of Sports Sciences reported a median sample size of 19 and the scarce usage of a-priori power analyses. We meta-analysed 89 studies from the same journal to assess the presence and extent of publication bias, as well as the average statistical power, by conducting a z-curve analysis. In a larger sample of 174 studies, we also examined a) the usage, reporting practices and reproducibility of a-priori power analyses; and b) the prevalence of reporting practices of t-statistic or F-ratio, degrees of freedom, exact p-values, effect sizes and confidence intervals. Our results indicate that there was some indication of publication bias and the average observed power was low (53% for significant and non-significant findings and 61% for only significant findings). Finally, the usage and reporting practices of a-priori power analyses as well as statistical results including test statistics, effect sizes and confidence intervals were suboptimal.

Assessing the Evidential Value of Mental Fatigue and Exercise Research.

It has often been reported that mental exertion, presumably leading to mental fatigue, can negatively affect exercise performance; however, recent findings have questioned the strength of the effect. To further complicate this issue, an overlooked problem might be the presence of publication bias in studies using underpowered designs, which is known to inflate false positive report probability and effect size estimates. Altogether, the presence of bias is likely to reduce the evidential value of the published literature on this topic, although it is unknown to what extent. The purpose of the current work was to assess the evidential value of studies published to date on the effect of mental exertion on exercise performance by assessing the presence of publication bias and the observed statistical power achieved by these studies. A traditional meta-analysis revealed a Cohen's dz effect size of - 0.54, 95% CI [- 0.68, - 0.40], p < .001. However, when we applied methods for estimating and correcting for publication bias (based on funnel plot asymmetry and observed p-values), we found that the bias-corrected effect size became negligible with most of publication-bias methods and decreased to - 0.36 in the more optimistic of all the scenarios. A robust Bayesian meta-analysis found strong evidence in favor of publication bias, BFpb > 1000, and inconclusive evidence in favor of the effect, adjusted dz = 0.01, 95% CrI [- 0.46, 0.37], BF10 = 0.90. Furthermore, the median observed statistical power assuming the unadjusted meta-analytic effect size (i.e., - 0.54) as the true effect size was 39% (min = 19%, max = 96%), indicating that, on average, these studies only had a 39% chance of observing a significant result if the true effect was Cohen's dz = - 0.54. If the more optimistic adjusted effect size (- 0.36) was assumed as the true effect, the median statistical power was just 20%. We conclude that the current literature is a useful case study for illustrating the dangers of conducting underpowered studies to detect the effect size of interest.

Adding Intermittent Vibration to Varied-intensity Work Intervals: No Extra Benefit.

Varied-intensity work intervals have been shown to induce higher fractions of maximal oxygen uptake during high-intensity interval training compared with constant-intensity work intervals. We assessed whether varied-intensity work intervals combined with intermittent vibration could further increase cyclists' fraction of maximal oxygen uptake to potentially optimise adaptive stimulus. Thirteen cyclists (V̇O2max: 69.7±7.1 ml·kg-1·min-1) underwent a performance assessment and two high-intensity interval training sessions. Both comprised six 5-minute varied-intensity work intervals within which the work rate was alternated between 100% (3×30-second blocks, with or without vibration) and 77% of maximal aerobic power (always without vibration). Adding vibration to varied-intensity work intervals did not elicit a longer time above ninety percent of maximal oxygen uptake (415±221 versus 399±209 seconds, P=0.69). Heart rate- and perceptual-based training-load metrics were also not affected (all P≥0.59). When considering individual work intervals, no between-condition differences were found (fraction of maximal oxygen uptake, P=0.34; total oxygen uptake, P=0.053; mean minute ventilation, P=0.079; mean heart rate, P=0.88; blood lactate concentration, P=0.53; ratings of perceived exertion, P=0.29). Adding intermittent vibration to varied-intensity work intervals does not increase the fraction of maximal oxygen uptake elicited. Whether intermittent exposure to vibration can enhance cyclists' adaptive stimulus triggered by high-intensity interval training remains to be determined.

Proposal of a Selection Protocol for Replication of Studies in Sports and Exercise Science.

INTRODUCTION: To improve the rigor of science, experimental evidence for scientific claims ideally needs to be replicated repeatedly with comparable analyses and new data to increase the collective confidence in the veracity of those claims. Large replication projects in psychology and cancer biology have evaluated the replicability of their fields but no collaborative effort has been undertaken in sports and exercise science. We propose to undertake such an effort here. As this is the first large replication project in this field, there is no agreed-upon protocol for selecting studies to replicate. Criticism of previous selection protocols include claims they were non-randomised and non-representative. Any selection protocol in sports and exercise science must be representative to provide an accurate estimate of replicability of the field. Our aim is to produce a protocol for selecting studies to replicate for inclusion in a large replication project in sports and exercise science. METHODS: The proposed selection protocol uses multiple inclusion and exclusion criteria for replication study selection, including: the year of publication and citation rankings, research disciplines, study types, the research question and key dependent variable, study methods and feasibility. Studies selected for replication will be stratified into pools based on instrumentation and expertise required, and will then be allocated to volunteer laboratories for replication. Replication outcomes will be assessed using a multiple inferential strategy and descriptive information will be reported regarding the final number of included and excluded studies, and original author responses to requests for raw data.

Replication concerns in sports and exercise science: a narrative review of selected methodological issues in the field.

Known methodological issues such as publication bias, questionable research practices and studies with underpowered designs are known to decrease the replicability of study findings. The presence of such issues has been widely established across different research fields, especially in psychology. Their presence raised the first concerns that the replicability of study findings could be low and led researchers to conduct large replication projects. These replication projects revealed that a significant portion of original study findings could not be replicated, giving rise to the conceptualization of the replication crisis. Although previous research in the field of sports and exercise science has identified the first warning signs, such as an overwhelming proportion of significant findings, small sample sizes and lack of data availability, their possible consequences for the replicability of our field have been overlooked. We discuss the consequences of the above issues on the replicability of our field and offer potential solutions to improve replicability.

Running Your Best Triathlon Race.

Negative or evenly paced racing strategies often lead to more favorable performance outcomes for endurance athletes. However, casual inspection of race split times and observational studies both indicate that elite triathletes competing in Olympic-distance triathlon typically implement a positive pacing strategy during the last of the 3 disciplines, the 10-km run. To address this apparent contradiction, the authors examined data from 14 International Triathlon Union elite races over 3 consecutive years involving a total of 725 male athletes. Analyses of race results confirm that triathletes typically implement a positive running pace strategy, running the first lap of the standard 4-lap circuit substantially faster than laps 2 (∼7%), 3 (∼9%), and 4 (∼12%). Interestingly, mean running pace in lap 1 had a substantially lower correlation with 10-km run time (r = .82) than both laps 2 and 3. Overall triathlon race performance (ranking) was best associated with run performance (r = .82) compared with the swim and cycle sections. Lower variability in race pace during the 10-km run was also reflective of more successful run times. Given that overall race outcome is mainly explained by the 10-km run performance, with top run performances associated with a more evenly paced strategy, triathletes (and their coaches) should reevaluate their pacing strategy during the run section.

Optimizing Interval Training Through Power-Output Variation Within the Work Intervals.

PURPOSE: Maximal oxygen uptake (V˙O2max) is a key determinant of endurance performance. Therefore, devising high-intensity interval training (HIIT) that maximizes stress of the oxygen-transport and -utilization systems may be important to stimulate further adaptation in athletes. The authors compared physiological and perceptual responses elicited by work intervals matched for duration and mean power output but differing in power-output distribution. METHODS: Fourteen cyclists (V˙O2max 69.2 [6.6] mL·kg-1·min-1) completed 3 laboratory visits for a performance assessment and 2 HIIT sessions using either varied-intensity or constant-intensity work intervals. RESULTS: Cyclists spent more time at >90%V˙O2max during HIIT with varied-intensity work intervals (410 [207] vs 286 [162] s, P = .02), but there were no differences between sessions in heart-rate- or perceptual-based training-load metrics (all P ≥ .1). When considering individual work intervals, minute ventilation (V˙E) was higher in the varied-intensity mode (F = 8.42, P = .01), but not respiratory frequency, tidal volume, blood lactate concentration [La], ratings of perceived exertion, or cadence (all F ≤ 3.50, ≥ .08). Absolute changes (Δ) between HIIT sessions were calculated per work interval, and Δ total oxygen uptake was moderately associated with ΔV˙E (r = .36, P = .002). CONCLUSIONS: In comparison with an HIIT session with constant-intensity work intervals, well-trained cyclists sustain higher fractions of V˙O2max when work intervals involved power-output variations. This effect is partially mediated by an increased oxygen cost of hyperpnea and not associated with a higher [La], perceived exertion, or training-load metrics.