Assessing the evidential value of mental fatigue and exercise research

Abstract

It has been reported repeatedly that mental fatigue can negatively affect exercise performance, but recent findings have questioned the strength of the effect. To further complicate this issue, an overlooked problem might be the presence of publication bias in a body of literature with studies using underpowered designs which is known to inflate false positive report probability and effect size estimates. Altogether, the presence of bias in the literature is likely to reduce the evidential value of the published literature on this topic, although it is unknown to which extent. The purpose of the current work was to assess the evidential value of studies published up to date on the effect of mental fatigue on exercise performance by assessing the presence of publication bias and the observed statistical power achieved by these studies. A traditional meta-analysis revealed a Cohen’s d_zeffect size of –0.49, 95% CI [–0.63, –0.34], p < 0.001. However, when we applied methods for estimating and correcting for publication bias (such as small-study effects), we found that the bias-corrected effect size decreased to –0.17. Furthermore, the median observed statistical power assuming the meta-analytic effect size (i.e., –0.49) as the true effect size was 34% (min = 16%, max = 97%), indicating that on average these studies only had a 34% chance of observing a significant result if the true effect was Cohen’s d_z = –0.49. If the adjusted effect size (–0.17) was assumed as the true effect, the median statistical power was just 0.09%. We conclude that the evidence for the effect of the mental fatigue effect is a useful case study for illustrating the dangers of small-study effects.