Meta-Analysis of Variation in Sport and Exercise Science
Examples of Application Within Resistance Training Research
DOI:
https://doi.org/10.51224/SRXIV.214Keywords:
meta-analysis, intraindividual variability, variability, resistance training, applied statisticsAbstract
Meta-analysis has become commonplace within sport and exercise science for synthesising and summarising empirical studies. However, most research in the field focuses upon mean effects; particularly the effects of interventions to improve outcomes such as fitness or performance. It is well known that there is considerable variability in response to interventions. Hence, interest has increased in exploring precision or personalised exercise approaches. Not only is the mean often affected by interventions, but variances may also be impacted. Exploration of variances in studies such as randomised controlled trials (RCTs) can yield insight into the existence of interindividual heterogeneity in response to interventions and help determine the degree of generalisability of effects. Yet, larger samples sizes than those used for typical mean effects are required when probing variances. Thus, for a field with small samples such as sport and exercise science, exploration of variance through a meta-analytic framework is appealing. Given the value of embracing and exploring variation alongside mean effects in sport and exercise science, yet its lack of application in research synthesis by way of meta-analysis, we introduce and discuss effect size approaches and models for meta-analysis of variation using relatable examples from resistance training RCTs.
Metrics
References
Atkinson, G., & Batterham, A. M. (2015). True and false interindividual differences in the physiological response to an intervention. Experimental Physiology, 100(6), 577–588. https://doi.org/10.1113/EP085070
Atkinson, G., Williamson, P., & Batterham, A. M. (2019). Issues in the determination of ‘responders’ and ‘non-responders’ in physiological research. Experimental Physiology, 104(8), 1215–1225. https://doi.org/10.1113/EP087712
Barr, D. J., Levy, R., Scheepers, C., & Tily, H. J. (2013). Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of Memory and Language, 68(3), 10.1016/j.jml.2012.11.001. https://doi.org/10.1016/j.jml.2012.11.001
Bernárdez-Vázquez, R., Raya-González, J., Castillo, D., & Beato, M. (2022). Resistance Training Variables for Optimization of Muscle Hypertrophy: An Umbrella Review. Frontiers in Sports and Active Living, 4. https://www.frontiersin.org/articles/10.3389/fspor.2022.949021
Bonafiglia, J. T., Swinton, P. A., Ross, R., Johannsen, N. M., Martin, C. K., Church, T. S., Slentz, C. A., Ross, L. M., Kraus, W. E., Walsh, J. J., Kenny, G. P., Goldfield, G. S., Prud’homme, D., Sigal, R. J., Earnest, C. P., & Gurd, B. J. (2022). Interindividual Differences in Trainability and Moderators of Cardiorespiratory Fitness, Waist Circumference, and Body Mass Responses: A Large-Scale Individual Participant Data Meta-analysis. Sports Medicine (Auckland, N.Z.). https://doi.org/10.1007/s40279-022-01725-9
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2021). Introduction to Meta-Analysis. John Wiley & Sons.
Caldwell, A., & Vigotsky, A. D. (2020). A case against default effect sizes in sport and exercise science. PeerJ, 8, e10314. https://doi.org/10.7717/peerj.10314
Carpinelli, R. N. (2017). Interindividual heterogeneity of adaptations to resistance training. Medicina Sportiva Practica, 18(4), 79–94.
Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Routledge. https://doi.org/10.4324/9780203771587
Cortés Martínez, J. (2021). Constant effect in randomized clinical trials with quantitative outcome: A methodological review. TDX (Tesis Doctorals En Xarxa). https://upcommons.upc.edu/handle/2117/349575
Curtis, P. S., & Wang, X. (1998). A meta-analysis of elevated CO2 effects on woody plant mass, form, and physiology. Oecologia, 113(3), 299–313. https://doi.org/10.1007/s004420050381
Deb, S. K., Brown, D. R., Gough, L. A., Mclellan, C. P., Swinton, P. A., Andy Sparks, S., & Mcnaughton, L. R. (2018). Quantifying the effects of acute hypoxic exposure on exercise performance and capacity: A systematic review and meta-regression. European Journal of Sport Science, 18(2), 243–256. https://doi.org/10.1080/17461391.2017.1410233
Esteves, G. P., Swinton, P., Sale, C., James, R. M., Artioli, G. G., Roschel, H., Gualano, B., Saunders, B., & Dolan, E. (2021). Individual Participant Data Meta-Analysis Provides No Evidence of Intervention Response Variation in Individuals Supplementing With Beta-Alanine. International Journal of Sport Nutrition and Exercise Metabolism, 31(4), 305–313. https://doi.org/10.1123/ijsnem.2021-0038
Exner, R. J., Patel, M. H., Whitener, D. V., Buckner, S. L., Jessee, M. B., & Dankel, S. J. (2022). Does performing resistance exercise to failure homogenize the training stimulus by accounting for differences in local muscular endurance? European Journal of Sport Science, 1–10. https://doi.org/10.1080/17461391.2021.2023657
Fisher, J., Steele, J., Wolf, M., Korakakis, P. A., Smith, D., & Giessing, J. (2022). The Role of Supervision in Resistance Training; an Exploratory Systematic Review and Meta-Analysis: International Journal of Strength and Conditioning, 2(1), Article 1. https://doi.org/10.47206/ijsc.v2i1.101
Glass, G. V. (1976). Primary, Secondary, and Meta-Analysis of Research. Educational Researcher, 5(10), 3–8. https://doi.org/10.2307/1174772
Gurevitch, J., Morrison, J. A., & Hedges, L. V. (2000). The Interaction between Competition and Predation: A Meta-analysis of Field Experiments. The American Naturalist, 155(4), 435–453. https://doi.org/10.1086/303337
Hagger, M. (2022). Meta-analysis. International Review of Sport and Exercise Psychology, 15(1), 120–151. https://doi.org/10.1080/1750984X.2021.1966824
Halliday, T. M., Savla, J., Marinik, E. L., Hedrick, V. E., Winett, R. A., & Davy, B. M. (2017). Resistance training is associated with spontaneous changes in aerobic physical activity but not overall diet quality in adults with prediabetes. Physiology & Behavior, 177, 49–56. https://doi.org/10.1016/j.physbeh.2017.04.013
Halperin, I., Malleron, T., Har-Nir, I., Androulakis-Korakakis, P., Wolf, M., Fisher, J., & Steele, J. (2022). Accuracy in Predicting Repetitions to Task Failure in Resistance Exercise: A Scoping Review and Exploratory Meta-analysis. Sports Medicine, 52(2), 377–390. https://doi.org/10.1007/s40279-021-01559-x
Hecksteden, A., Kraushaar, J., Scharhag-Rosenberger, F., Theisen, D., Senn, S., & Meyer, T. (2015). Individual response to exercise training—A statistical perspective. Journal of Applied Physiology (Bethesda, Md.: 1985), 118(12), 1450–1459. https://doi.org/10.1152/japplphysiol.00714.2014
Hedges, L. V., Gurevitch, J., & Curtis, P. S. (1999). The Meta-Analysis of Response Ratios in Experimental Ecology. Ecology, 80(4), 1150–1156. https://doi.org/10.1890/0012-9658(1999)080[1150:TMAORR]2.0.CO;2
Hedges, L. V., & Nowell, A. (1995). Sex differences in mental test scores, variability, and numbers of high-scoring individuals. Science (New York, N.Y.), 269(5220), 41–45. https://doi.org/10.1126/science.7604277
Hedges, L. V., & Olkin, I. (2014). Statistical Methods for Meta-Analysis. Academic Press.
Heidel, K. A., Novak, Z. J., & Dankel, S. J. (2022). Machines and free weight exercises: A systematic review and meta-analysis comparing changes in muscle size, strength, and power. The Journal of Sports Medicine and Physical Fitness, 62(8), 1061–1070. https://doi.org/10.23736/S0022-4707.21.12929-9
Hopkins, W. G. (2015). Individual responses made easy. Journal of Applied Physiology (Bethesda, Md.: 1985), 118(12), 1444–1446. https://doi.org/10.1152/japplphysiol.00098.2015
Hrubeniuk, T. J., Bonafiglia, J. T., Bouchard, D. R., Gurd, B. J., & Sénéchal, M. (2022). Directions for Exercise Treatment Response Heterogeneity and Individual Response Research. International Journal of Sports Medicine, 43(1), 11–22. https://doi.org/10.1055/a-1548-7026
Kelley, G. A. (2022). Precision exercise medicine in rheumatology: Don’t put the cart before the horse. Clinical Rheumatology, 41(8), 2277–2279. https://doi.org/10.1007/s10067-022-06260-6
Kelley, G. A., Kelley, K. S., & Callahan, L. F. (2022). Are There Interindividual Differences in Anxiety as a Result of Aerobic Exercise Training in Adults with Fibromyalgia? An Ancillary Meta-analysis of Randomized Controlled Trials. Archives of Physical Medicine and Rehabilitation, S0003-9993(22)00007-7. https://doi.org/10.1016/j.apmr.2021.12.019
Kelley, G. A., Kelley, K. S., & Pate, R. R. (2020). Are There Inter-Individual Differences in Fat Mass and Percent Body Fat as a Result of Aerobic Exercise Training in Overweight and Obese Children and Adolescents? A Meta-Analytic Perspective. Childhood Obesity (Print), 16(5), 301–306. https://doi.org/10.1089/chi.2020.0056
Lajeunesse, M. J. (2015). Bias and correction for the log response ratio in ecological meta-analysis. Ecology, 96(8), 2056–2063. https://doi.org/10.1890/14-2402.1
Mills, H. L., Higgins, J. P. T., Morris, R. W., Kessler, D., Heron, J., Wiles, N., Davey Smith, G., & Tilling, K. (2021). Detecting Heterogeneity of Intervention Effects Using Analysis and Meta-analysis of Differences in Variance Between Trial Arms. Epidemiology (Cambridge, Mass.), 32(6), 846–854. https://doi.org/10.1097/EDE.0000000000001401
Morris, S. B. (2008). Estimating Effect Sizes From Pretest-Posttest-Control Group Designs. Organizational Research Methods, 11(2), 364–386. https://doi.org/10.1177/1094428106291059
Morris, W. F., Hufbauer, R. A., Agrawal, A. A., Bever, J. D., Borowicz, V. A., Gilbert, G. S., Maron, J. L., Mitchell, C. E., Parker, I. M., Power, A. G., Torchin, M. E., & Vázquez, D. P. (2007). Direct and interactive effects of enemies and mutualists on plant performance: A meta-analysis. Ecology, 88(4), 1021–1029. https://doi.org/10.1890/06-0442
Nakagawa, S., & Cuthill, I. C. (2007). Effect size, confidence interval and statistical significance: A practical guide for biologists. Biological Reviews of the Cambridge Philosophical Society, 82(4), 591–605. https://doi.org/10.1111/j.1469-185X.2007.00027.x
Nakagawa, S., Poulin, R., Mengersen, K., Reinhold, K., Engqvist, L., Lagisz, M., & Senior, A. M. (2015). Meta-analysis of variation: Ecological and evolutionary applications and beyond. Methods in Ecology and Evolution, 6(2), 143–152. https://doi.org/10.1111/2041-210X.12309
Nakagawa, S., & Schielzeth, H. (2012). The mean strikes back: Mean-variance relationships and heteroscedasticity. Trends in Ecology & Evolution, 27(9), 474–475; author reply 475-476. https://doi.org/10.1016/j.tree.2012.04.003
Nuzzo, J., Pinto, M., Nosaka, K., & Steele, J. (2022). How much stronger are muscles eccentrically than concentrically? : Meta-analysis of the influences of sex, age, joint action, and velocity. SportRxiv. https://doi.org/10.51224/SRXIV.197
Pickering, C., & Kiely, J. (2019). Do Non-Responders to Exercise Exist-and If So, What Should We Do About Them? Sports Medicine (Auckland, N.Z.), 49(1), 1–7. https://doi.org/10.1007/s40279-018-01041-1
Plackett, R. L. (1958). Studies in the History of Probability and Statistics: VII. The Principle of the Arithmetic Mean. Biometrika, 45(1/2), 130–135. https://doi.org/10.2307/2333051
Polito, M. D., Papst, R. R., & Farinatti, P. (2021). Moderators of strength gains and hypertrophy in resistance training: A systematic review and meta-analysis. Journal of Sports Sciences, 39(19), 2189–2198. https://doi.org/10.1080/02640414.2021.1924978
Raudenbush, S. W., & Bryk, A. S. (1987). Examining Correlates of Diversity. Journal of Educational Statistics, 12(3), 241–269. https://doi.org/10.3102/10769986012003241
Ross, R., Goodpaster, B. H., Koch, L. G., Sarzynski, M. A., Kohrt, W. M., Johannsen, N. M., Skinner, J. S., Castro, A., Irving, B. A., Noland, R. C., Sparks, L. M., Spielmann, G., Day, A. G., Pitsch, W., Hopkins, W. G., & Bouchard, C. (2019). Precision exercise medicine: Understanding exercise response variability. British Journal of Sports Medicine, 53(18), 1141–1153. https://doi.org/10.1136/bjsports-2018-100328
Senior, A. M., Viechtbauer, W., & Nakagawa, S. (2020). Revisiting and expanding the meta-analysis of variation: The log coefficient of variation ratio. Research Synthesis Methods, 11(4), 553–567. https://doi.org/10.1002/jrsm.1423
Steele, J., Fisher, J. P., Giessing, J., Androulakis-Korakakis, P., Wolf, M., Kroeske, B., & Reuters, R. (2022). Long-Term Time-Course of Strength Adaptation to Minimal Dose Resistance Training Through Retrospective Longitudinal Growth Modeling. Research Quarterly for Exercise and Sport, 0(0), 1–18. https://doi.org/10.1080/02701367.2022.2070592
Steele, J., Fisher, J., Skivington, M., Dunn, C., Arnold, J., Tew, G., Batterham, A. M., Nunan, D., O’Driscoll, J. M., Mann, S., Beedie, C., Jobson, S., Smith, D., Vigotsky, A., Phillips, S., Estabrooks, P., & Winett, R. (2017). A higher effort-based paradigm in physical activity and exercise for public health: Making the case for a greater emphasis on resistance training. BMC Public Health, 17(1), 300. https://doi.org/10.1186/s12889-017-4209-8
Steele, J., Malleron, T., Har-Nir, I., Androulakis-Korakakis, P., Wolf, M., Fisher, J. P., & Halperin, I. (2022). Are Trainees Lifting Heavy Enough? Self-Selected Loads in Resistance Exercise: A Scoping Review and Exploratory Meta-analysis. Sports Medicine. https://doi.org/10.1007/s40279-022-01717-9
Steele, J., Plotkin, D., Van Every, D., Rosa, A., Zambrano, H., Mendelovits, B., Carrasquillo-Mercado, M., Grgic, J., & Schoenfeld, B. J. (2021). Slow and Steady, or Hard and Fast? A Systematic Review and Meta-Analysis of Studies Comparing Body Composition Changes between Interval Training and Moderate Intensity Continuous Training. Sports (Basel, Switzerland), 9(11), 155. https://doi.org/10.3390/sports9110155
Swinton, P. A., Hemingway, B. S., Saunders, B., Gualano, B., & Dolan, E. (2018). A Statistical Framework to Interpret Individual Response to Intervention: Paving the Way for Personalized Nutrition and Exercise Prescription. Frontiers in Nutrition, 5, 41. https://doi.org/10.3389/fnut.2018.00041
Swinton, P. A., Katherine, B., Andy, H., Leon, G., John, P., Rodrigo, R. A., Patrick, M., & Andrew, M. (2022). Interpreting magnitude of change in strength and conditioning: Effect size selection, threshold values and Bayesian updating Journal of Sports Sciences. Journal of Sports Sciences.
Taylor, L. R. (1961). Aggregation, Variance and the Mean. Nature, 189(4766), Article 4766. https://doi.org/10.1038/189732a0
Tenan, M. S., Vigotsky, A. D., & Caldwell, A. R. (2020). Comment on: “A Method to Stop Analyzing Random Error and Start Analyzing Differential Responders to Exercise”. Sports Medicine, 50(2), 431–434. https://doi.org/10.1007/s40279-019-01249-9
Usui, T., Macleod, M. R., McCann, S. K., Senior, A. M., & Nakagawa, S. (2021). Meta-analysis of variation suggests that embracing variability improves both replicability and generalizability in preclinical research. PLOS Biology, 19(5), e3001009. https://doi.org/10.1371/journal.pbio.3001009
Viechtbauer, W. (2010). Conducting Meta-Analyses in R with the metafor Package. Journal of Statistical Software, 36, 1–48. https://doi.org/10.18637/jss.v036.i03
Vigotsky, A., Nuckols, G. L., Fisher, J., Heathers, J., Krieger, J., Schoenfeld, B. J., Giessing, J., & Steele, J. (2020). Improbable data patterns in the work of Barbalho et al. SportRxiv. https://doi.org/10.31236/osf.io/sg3wm
Yang, Y., Hillebrand, H., Lagisz, M., Cleasby, I., & Nakagawa, S. (2022). Low statistical power and overestimated anthropogenic impacts, exacerbated by publication bias, dominate field studies in global change biology. Global Change Biology, 28(3), 969–989. https://doi.org/10.1111/gcb.15972
Downloads
Posted
Versions
- 2023-11-13 (4)
- 2023-09-15 (3)
- 2023-09-12 (2)
- 2022-10-26 (1)
Categories
License
Copyright (c) 2022 James Steele, James P. Fisher, Dave Smith, Brad J Schoenfeld, Yefeng Yang, Shinichi Nakagawa
This work is licensed under a Creative Commons Attribution 4.0 International License.
