Abstract
In Sport and Exercise Science (SES), 82% of published studies report significant findings indicating these studies have a robust research design that examines true effects with high statistical power. However, small sample sizes and underpowered designs are common in SES. Related literature in psychology reports that only 37% of replicated effects yield similar results to the original. Replication studies in SES are still rare, raising concerns about the overwhelming reported significant findings despite the small sample sizes. PURPOSE: To replicate the findings of a published SES study and compare results. METHODS: Based on the criteria published by Murphy et al., 2023, the Effects of Caffeine on Vertical Jump Height and Execution in Collegiate Athletes by Bloms et al., 2015 was selected as the original study. The replication study, mirroring the original design, utilized a single-blind, randomized, crossover approach. Like the original, participants engaged in training sessions lasting 8-20 hours per week and were administered a caffeine dosage of 5 mg/kg or a placebo. Following a 60-minute interval post-caffeine ingestion, squat jump (SJ) and countermovement jump (CMJ) were performed. Key distinctions between the replication and original studies encompassed athlete levels, force platform types, and analysis software, as well as the range of sports included, alongside a sample size disparity (40 vs. 25). A paired t-test compared jump height (cm) under both caffeine and placebo conditions. RESULTS: The original study showed that caffeine consumption compared to placebo (μcaffeine (SD) vs. μplacebo (SD)) improved both SJ (34.5 (6.7) vs. 32.8 (6.2), p = .001) and CMJ (37.9 (7.4) vs. 36.4 (6.9), p = .001) heights. In the replication study, caffeine consumption did not improve SJ (29.8 (7) vs. 29.5 (7), 0.2 ± 2.7cm, t38 = 0.56, p = 0.58, dz = 0.1, 95%CI [-.23, .40]) nor CMJ (32.4 (8) vs. 31.7 (8), 0.7 ± 3.2cm, t39 = 1.34, p = .19, dz = 0.2, 95%CI [-.10, .52]). The z-test indicated that the replication effect size for SJ (dz = 0.09) was significantly smaller than the original effect size (dz = 0.75) (z = 2.39, p = 0.008). Similarly, the replication effect size for CMJ (dz = 0.21) was significantly smaller than the original effect size (dz = 0.75) (z = 1.94, p = 0.03). CONCLUSION: The observed replicability bias in SES, indicated by the divergence of replication study results from the original findings, points towards potential issues of publication bias and false positives within the field. This underscores the need for cautious interpretation of reported significant findings and a call for more rigorous research practices in SES.
Recommended Citation
Papadakis, Zacharias and Murphy, Jennifer
(2024)
"Replication Crisis in Sport and Exercise Science,"
International Journal of Exercise Science: Conference Proceedings: Vol. 15:
Iss.
4, Article 13.
Available at:
https://digitalcommons.wku.edu/ijesab/vol15/iss4/13