DOI of the published article http://dx.doi.org/10.1080/02640414.2023.2286748
Meta-Analysis of Variation in Sport and Exercise Science
Examples of Application Within Resistance Training Research
DOI:
https://doi.org/10.51224/SRXIV.214Keywords:
meta-analysis, intraindividual variability, variability, resistance training, applied statisticsAbstract
Meta-analysis has become commonplace within sport and exercise science for synthesising and summarising empirical studies. However, most research in the field focuses upon mean effects; particularly the effects of interventions to improve outcomes such as fitness or performance. It is thought that individual responses to interventions vary considerably. Hence, interest has increased in exploring precision or personalised exercise approaches. Not only is the mean often affected by interventions, but variation may also be impacted. Exploration of variation in studies such as randomised controlled trials (RCTs) can yield insight into interindividual heterogeneity in response to interventions and help determine generalisability of effects. Yet, larger samples sizes than those used for typical mean effects are required when probing variation. Thus, in a field with small samples such as sport and exercise science, exploration of variation through a meta-analytic framework is appealing. Despite the value of embracing and exploring variation alongside mean effects in sport and exercise science it is rarely applied to research synthesis through meta-analysis.We introduce and evaluate different effect size calculations along with models for meta-analysis of variation using relatable examples from resistance training RCTs.
Metrics
References
Atkinson, G., & Batterham, A. M. (2015). True and false interindividual differences in the physiological response to an intervention. Experimental Physiology, 100(6), 577–588. https://doi.org/10.1113/EP085070
Atkinson, G., Williamson, P., & Batterham, A. M. (2019). Issues in the determination of ‘responders’ and ‘non-responders’ in physiological research. Experimental Physiology, 104(8), 1215–1225. https://doi.org/10.1113/EP087712
Barr, D. J., Levy, R., Scheepers, C., & Tily, H. J. (2013). Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of Memory and Language, 68(3), 10.1016/j.jml.2012.11.001. https://doi.org/10.1016/j.jml.2012.11.001
Bernárdez-Vázquez, R., Raya-González, J., Castillo, D., & Beato, M. (2022). Resistance Training Variables for Optimization of Muscle Hypertrophy: An Umbrella Review. Frontiers in Sports and Active Living, 4. https://www.frontiersin.org/articles/10.3389/fspor.2022.949021
Bonafiglia, J. T., Swinton, P. A., Ross, R., Johannsen, N. M., Martin, C. K., Church, T. S., Slentz, C. A., Ross, L. M., Kraus, W. E., Walsh, J. J., Kenny, G. P., Goldfield, G. S., Prud’homme, D., Sigal, R. J., Earnest, C. P., & Gurd, B. J. (2022). Interindividual Differences in Trainability and Moderators of Cardiorespiratory Fitness, Waist Circumference, and Body Mass Responses: A Large-Scale Individual Participant Data Meta-analysis. Sports Medicine (Auckland, N.Z.). https://doi.org/10.1007/s40279-022-01725-9
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2010). A basic introduction to fixed-effect and random-effects models for meta-analysis. Research Synthesis Methods, 1(2), 97–111. https://doi.org/10.1002/jrsm.12
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2021). Introduction to Meta-Analysis. John Wiley & Sons.
Box, G. E. P. (1976). Science and Statistics. Journal of the American Statistical Association, 71(356), 791–799. https://doi.org/10.1080/01621459.1976.10480949
Caldwell, A., & Vigotsky, A. D. (2020). A case against default effect sizes in sport and exercise science. PeerJ, 8, e10314. https://doi.org/10.7717/peerj.10314
Carpinelli, R. N. (2017). Interindividual heterogeneity of adaptations to resistance training. Medicina Sportiva Practica, 18(4), 79–94.
Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Routledge. https://doi.org/10.4324/9780203771587
Cortés Martínez, J. (2021). Constant effect in randomized clinical trials with quantitative outcome: A methodological review. TDX (Tesis Doctorals En Xarxa). https://upcommons.upc.edu/handle/2117/349575
Curtis, P. S., & Wang, X. (1998). A meta-analysis of elevated CO2 effects on woody plant mass, form, and physiology. Oecologia, 113(3), 299–313. https://doi.org/10.1007/s004420050381
Deb, S. K., Brown, D. R., Gough, L. A., Mclellan, C. P., Swinton, P. A., Andy Sparks, S., & Mcnaughton, L. R. (2018). Quantifying the effects of acute hypoxic exposure on exercise performance and capacity: A systematic review and meta-regression. European Journal of Sport Science, 18(2), 243–256. https://doi.org/10.1080/17461391.2017.1410233
Esteves, G. P., Swinton, P., Sale, C., James, R. M., Artioli, G. G., Roschel, H., Gualano, B., Saunders, B., & Dolan, E. (2021). Individual Participant Data Meta-Analysis Provides No Evidence of Intervention Response Variation in Individuals Supplementing With Beta-Alanine. International Journal of Sport Nutrition and Exercise Metabolism, 31(4), 305–313. https://doi.org/10.1123/ijsnem.2021-0038
Exner, R. J., Patel, M. H., Whitener, D. V., Buckner, S. L., Jessee, M. B., & Dankel, S. J. (2022). Does performing resistance exercise to failure homogenize the training stimulus by accounting for differences in local muscular endurance? European Journal of Sport Science, 1–10. https://doi.org/10.1080/17461391.2021.2023657
Fisher, J., Steele, J., Wolf, M., Korakakis, P. A., Smith, D., & Giessing, J. (2022). The Role of Supervision in Resistance Training; an Exploratory Systematic Review and Meta-Analysis: International Journal of Strength and Conditioning, 2(1), Article 1. https://doi.org/10.47206/ijsc.v2i1.101
Glass, G. V. (1976). Primary, Secondary, and Meta-Analysis of Research. Educational Researcher, 5(10), 3–8. https://doi.org/10.2307/1174772
Gurevitch, J., Morrison, J. A., & Hedges, L. V. (2000). The Interaction between Competition and Predation: A Meta-analysis of Field Experiments. The American Naturalist, 155(4), 435–453. https://doi.org/10.1086/303337
Hagger, M. (2006). Meta-analysis in sport and exercise research: Review, recent developments, and recommendations. European Journal of Sport Science, 6(2), 103–115. https://doi.org/10.1080/17461390500528527
Hagger, M. (2022). Meta-analysis. International Review of Sport and Exercise Psychology, 15(1), 120–151. https://doi.org/10.1080/1750984X.2021.1966824
Halliday, T. M., Savla, J., Marinik, E. L., Hedrick, V. E., Winett, R. A., & Davy, B. M. (2017). Resistance training is associated with spontaneous changes in aerobic physical activity but not overall diet quality in adults with prediabetes. Physiology & Behavior, 177, 49–56. https://doi.org/10.1016/j.physbeh.2017.04.013
Halperin, I., Malleron, T., Har-Nir, I., Androulakis-Korakakis, P., Wolf, M., Fisher, J., & Steele, J. (2022). Accuracy in Predicting Repetitions to Task Failure in Resistance Exercise: A Scoping Review and Exploratory Meta-analysis. Sports Medicine, 52(2), 377–390. https://doi.org/10.1007/s40279-021-01559-x
Hecksteden, A., Kraushaar, J., Scharhag-Rosenberger, F., Theisen, D., Senn, S., & Meyer, T. (2015). Individual response to exercise training—A statistical perspective. Journal of Applied Physiology (Bethesda, Md.: 1985), 118(12), 1450–1459. https://doi.org/10.1152/japplphysiol.00714.2014
Hedges, L. V., Gurevitch, J., & Curtis, P. S. (1999). The Meta-Analysis of Response Ratios in Experimental Ecology. Ecology, 80(4), 1150–1156. https://doi.org/10.1890/0012-9658(1999)080[1150:TMAORR]2.0.CO;2
Hedges, L. V., & Nowell, A. (1995). Sex differences in mental test scores, variability, and numbers of high-scoring individuals. Science (New York, N.Y.), 269(5220), 41–45. https://doi.org/10.1126/science.7604277
Hedges, L. V., & Olkin, I. (2014). Statistical Methods for Meta-Analysis. Academic Press.
Hedges, L. V., Tipton, E., & Johnson, M. C. (2010). Robust variance estimation in meta-regression with dependent effect size estimates. Research Synthesis Methods, 1(1), 39–65. https://doi.org/10.1002/jrsm.5
Heidel, K. A., Novak, Z. J., & Dankel, S. J. (2022). Machines and free weight exercises: A systematic review and meta-analysis comparing changes in muscle size, strength, and power. The Journal of Sports Medicine and Physical Fitness, 62(8), 1061–1070. https://doi.org/10.23736/S0022-4707.21.12929-9
Hopkins, W. G. (2015). Individual responses made easy. Journal of Applied Physiology (Bethesda, Md.: 1985), 118(12), 1444–1446. https://doi.org/10.1152/japplphysiol.00098.2015
Hrubeniuk, T. J., Bonafiglia, J. T., Bouchard, D. R., Gurd, B. J., & Sénéchal, M. (2022). Directions for Exercise Treatment Response Heterogeneity and Individual Response Research. International Journal of Sports Medicine, 43(1), 11–22. https://doi.org/10.1055/a-1548-7026
Kass, R. E., & Raftery, A. E. (1995). Bayes Factors. Journal of the American Statistical Association, 90(430), 773–795. https://doi.org/10.1080/01621459.1995.10476572
Kelley, G. A. (2022). Precision exercise medicine in rheumatology: Don’t put the cart before the horse. Clinical Rheumatology, 41(8), 2277–2279. https://doi.org/10.1007/s10067-022-06260-6
Kelley, G. A., Kelley, K. S., & Callahan, L. F. (2022). Are There Interindividual Differences in Anxiety as a Result of Aerobic Exercise Training in Adults with Fibromyalgia? An Ancillary Meta-analysis of Randomized Controlled Trials. Archives of Physical Medicine and Rehabilitation, S0003-9993(22)00007-7. https://doi.org/10.1016/j.apmr.2021.12.019
Kelley, G. A., Kelley, K. S., & Pate, R. R. (2020). Are There Inter-Individual Differences in Fat Mass and Percent Body Fat as a Result of Aerobic Exercise Training in Overweight and Obese Children and Adolescents? A Meta-Analytic Perspective. Childhood Obesity (Print), 16(5), 301–306. https://doi.org/10.1089/chi.2020.0056
Lajeunesse, M. J. (2011). On the meta-analysis of response ratios for studies with correlated and multi-group designs. Ecology, 92(11), 2049–2055. https://doi.org/10.1890/11-0423.1
Lajeunesse, M. J. (2015). Bias and correction for the log response ratio in ecological meta-analysis. Ecology, 96(8), 2056–2063. https://doi.org/10.1890/14-2402.1
Mills, H. L., Higgins, J. P. T., Morris, R. W., Kessler, D., Heron, J., Wiles, N., Davey Smith, G., & Tilling, K. (2021). Detecting Heterogeneity of Intervention Effects Using Analysis and Meta-analysis of Differences in Variance Between Trial Arms. Epidemiology (Cambridge, Mass.), 32(6), 846–854. https://doi.org/10.1097/EDE.0000000000001401
Morris, S. B. (2008). Estimating Effect Sizes From Pretest-Posttest-Control Group Designs. Organizational Research Methods, 11(2), 364–386. https://doi.org/10.1177/1094428106291059
Morris, W. F., Hufbauer, R. A., Agrawal, A. A., Bever, J. D., Borowicz, V. A., Gilbert, G. S., Maron, J. L., Mitchell, C. E., Parker, I. M., Power, A. G., Torchin, M. E., & Vázquez, D. P. (2007). Direct and interactive effects of enemies and mutualists on plant performance: A meta-analysis. Ecology, 88(4), 1021–1029. https://doi.org/10.1890/06-0442
Nakagawa, S., & Cuthill, I. C. (2007). Effect size, confidence interval and statistical significance: A practical guide for biologists. Biological Reviews of the Cambridge Philosophical Society, 82(4), 591–605. https://doi.org/10.1111/j.1469-185X.2007.00027.x
Nakagawa, S., Poulin, R., Mengersen, K., Reinhold, K., Engqvist, L., Lagisz, M., & Senior, A. M. (2015). Meta-analysis of variation: Ecological and evolutionary applications and beyond. Methods in Ecology and Evolution, 6(2), 143–152. https://doi.org/10.1111/2041-210X.12309
Nakagawa, S., & Schielzeth, H. (2012). The mean strikes back: Mean-variance relationships and heteroscedasticity. Trends in Ecology & Evolution, 27(9), 474–475; author reply 475-476. https://doi.org/10.1016/j.tree.2012.04.003
Nuzzo, J., Pinto, M. D., Nosaka, K., & Steele, J. (2023a). The Eccentric:Concentric Strength Ratio of Human Skeletal Muscle In Vivo: Meta-analysis of the Influences of Sex, Age, Joint Action, and Velocity. Sports Medicine, 53(6), 1125–1136. https://doi.org/10.1007/s40279-023-01851-y
Nuzzo, J., Pinto, M., Nosaka, K., & Steele, J. (2022). How much stronger are muscles eccentrically than concentrically? : Meta-analysis of the influences of sex, age, joint action, and velocity. SportRxiv. https://doi.org/10.51224/SRXIV.197
Nuzzo, J., Pinto, M., Nosaka, K., & Steele, J. (2023b). Maximal number of repetitions at percentages of the one repetition maximum: A meta-regression and moderator analysis of sex, age, training status, and exercise. SportRxiv. https://doi.org/10.51224/SRXIV.291
Olsson-Collentine, A., Van Aert, R. C. M., Bakker, M., & Wicherts, J. (2023). Meta-analyzing the multiverse: A peek under the hood of selective reporting. Psychological Methods. https://doi.org/10.1037/met0000559
Pickering, C., & Kiely, J. (2019). Do Non-Responders to Exercise Exist-and If So, What Should We Do About Them? Sports Medicine (Auckland, N.Z.), 49(1), 1–7. https://doi.org/10.1007/s40279-018-01041-1
Plackett, R. L. (1958). Studies in the History of Probability and Statistics: VII. The Principle of the Arithmetic Mean. Biometrika, 45(1/2), 130–135. https://doi.org/10.2307/2333051
Polito, M. D., Papst, R. R., & Farinatti, P. (2021). Moderators of strength gains and hypertrophy in resistance training: A systematic review and meta-analysis. Journal of Sports Sciences, 39(19), 2189–2198. https://doi.org/10.1080/02640414.2021.1924978
Raudenbush, S. W., & Bryk, A. S. (1987). Examining Correlates of Diversity. Journal of Educational Statistics, 12(3), 241–269. https://doi.org/10.3102/10769986012003241
Robinson, Z., Helms, E., Trexler, E., Steele, J., Hall, M., Huang, C.-J., & Zourdos, M. (2023). Optimizing Research Methodology for the Detection of Individual Response Variation in Resistance Training. SportRxiv. https://doi.org/10.51224/SRXIV.340
Ross, R., Goodpaster, B. H., Koch, L. G., Sarzynski, M. A., Kohrt, W. M., Johannsen, N. M., Skinner, J. S., Castro, A., Irving, B. A., Noland, R. C., Sparks, L. M., Spielmann, G., Day, A. G., Pitsch, W., Hopkins, W. G., & Bouchard, C. (2019). Precision exercise medicine: Understanding exercise response variability. British Journal of Sports Medicine, 53(18), 1141–1153. https://doi.org/10.1136/bjsports-2018-100328
Senior, A. M., Viechtbauer, W., & Nakagawa, S. (2020). Revisiting and expanding the meta-analysis of variation: The log coefficient of variation ratio. Research Synthesis Methods, 11(4), 553–567. https://doi.org/10.1002/jrsm.1423
Steegen, S., Tuerlinckx, F., Gelman, A., & Vanpaemel, W. (2016). Increasing Transparency Through a Multiverse Analysis. Perspectives on Psychological Science, 11(5), 702–712. https://doi.org/10.1177/1745691616658637
Steele, J., Fisher, J. P., Giessing, J., Androulakis-Korakakis, P., Wolf, M., Kroeske, B., & Reuters, R. (2022). Long-Term Time-Course of Strength Adaptation to Minimal Dose Resistance Training Through Retrospective Longitudinal Growth Modeling. Research Quarterly for Exercise and Sport, 0(0), 1–18. https://doi.org/10.1080/02701367.2022.2070592
Steele, J., Fisher, J., Skivington, M., Dunn, C., Arnold, J., Tew, G., Batterham, A. M., Nunan, D., O’Driscoll, J. M., Mann, S., Beedie, C., Jobson, S., Smith, D., Vigotsky, A., Phillips, S., Estabrooks, P., & Winett, R. (2017). A higher effort-based paradigm in physical activity and exercise for public health: Making the case for a greater emphasis on resistance training. BMC Public Health, 17(1), 300. https://doi.org/10.1186/s12889-017-4209-8
Steele, J., Malleron, T., Har-Nir, I., Androulakis-Korakakis, P., Wolf, M., Fisher, J. P., & Halperin, I. (2022). Are Trainees Lifting Heavy Enough? Self-Selected Loads in Resistance Exercise: A Scoping Review and Exploratory Meta-analysis. Sports Medicine. https://doi.org/10.1007/s40279-022-01717-9
Steele, J., Plotkin, D., Van Every, D., Rosa, A., Zambrano, H., Mendelovits, B., Carrasquillo-Mercado, M., Grgic, J., & Schoenfeld, B. J. (2021). Slow and Steady, or Hard and Fast? A Systematic Review and Meta-Analysis of Studies Comparing Body Composition Changes between Interval Training and Moderate Intensity Continuous Training. Sports (Basel, Switzerland), 9(11), 155. https://doi.org/10.3390/sports9110155
Swinton, P. A., Hemingway, B. S., Saunders, B., Gualano, B., & Dolan, E. (2018). A Statistical Framework to Interpret Individual Response to Intervention: Paving the Way for Personalized Nutrition and Exercise Prescription. Frontiers in Nutrition, 5, 41. https://doi.org/10.3389/fnut.2018.00041
Swinton, P. A., Katherine, B., Andy, H., Leon, G., John, P., Rodrigo, R. A., Patrick, M., & Andrew, M. (2022). Interpreting magnitude of change in strength and conditioning: Effect size selection, threshold values and Bayesian updating Journal of Sports Sciences. Journal of Sports Sciences.
Taylor, L. R. (1961). Aggregation, Variance and the Mean. Nature, 189(4766), Article 4766. https://doi.org/10.1038/189732a0
Tenan, M. S., Vigotsky, A. D., & Caldwell, A. R. (2020). Comment on: “A Method to Stop Analyzing Random Error and Start Analyzing Differential Responders to Exercise”. Sports Medicine, 50(2), 431–434. https://doi.org/10.1007/s40279-019-01249-9
Usui, T., Macleod, M. R., McCann, S. K., Senior, A. M., & Nakagawa, S. (2021). Meta-analysis of variation suggests that embracing variability improves both replicability and generalizability in preclinical research. PLOS Biology, 19(5), e3001009. https://doi.org/10.1371/journal.pbio.3001009
Van den Noortgate, W., López-López, J. A., Marín-Martínez, F., & Sánchez-Meca, J. (2013). Three-level meta-analysis of dependent effect sizes. Behavior Research Methods, 45(2), 576–594. https://doi.org/10.3758/s13428-012-0261-6
Viechtbauer, W. (2010). Conducting Meta-Analyses in R with the metafor Package. Journal of Statistical Software, 36, 1–48. https://doi.org/10.18637/jss.v036.i03
Vigotsky, A., Nuckols, G. L., Fisher, J., Heathers, J., Krieger, J., Schoenfeld, B. J., Giessing, J., & Steele, J. (2020). Improbable data patterns in the work of Barbalho et al. SportRxiv. https://doi.org/10.31236/osf.io/sg3wm
Wagenmakers, E.-J. (2007). A practical solution to the pervasive problems ofp values. Psychonomic Bulletin & Review, 14(5), 779–804. https://doi.org/10.3758/BF03194105
Wolf, M., Androulakis-Korakakis, P., Fisher, J., Schoenfeld, B., & Steele, J. (2023). Partial Vs Full Range of Motion Resistance Training: A Systematic Review and Meta-Analysis. International Journal of Strength and Conditioning, 3(1), Article 1. https://doi.org/10.47206/ijsc.v3i1.182
Yang, Y., Hillebrand, H., Lagisz, M., Cleasby, I., & Nakagawa, S. (2022). Low statistical power and overestimated anthropogenic impacts, exacerbated by publication bias, dominate field studies in global change biology. Global Change Biology, 28(3), 969–989. https://doi.org/10.1111/gcb.15972
Downloads
Posted
Versions
- 2023-11-13 (4)
- 2023-09-15 (3)
- 2023-09-12 (2)
- 2022-10-26 (1)
Categories
License
Copyright (c) 2022 James Steele, James P. Fisher, Dave Smith, Brad J Schoenfeld, Yefeng Yang, Shinichi Nakagawa
This work is licensed under a Creative Commons Attribution 4.0 International License.
