Preprint has been published in a journal as an article
DOI of the published article http://dx.doi.org/10.1080/02640414.2023.2286748
Preprint / Version 4

Meta-Analysis of Variation in Sport and Exercise Science

Examples of Application Within Resistance Training Research

##article.authors##

  • James Steele Solent University
  • James P. Fisher Solent University
  • Dave Smith Manchester Metropolitan University
  • Brad J Schoenfeld CUNY Lehman College
  • Yefeng Yang University of New South Wales
  • Shinichi Nakagawa University of New South Wales

DOI:

https://doi.org/10.51224/SRXIV.214

Keywords:

meta-analysis, intraindividual variability, variability, resistance training, applied statistics

Abstract

Meta-analysis has become commonplace within sport and exercise science for synthesising and summarising empirical studies. However, most research in the field focuses upon mean effects; particularly the effects of interventions to improve outcomes such as fitness or performance. It is thought that individual responses to interventions vary considerably. Hence, interest has increased in exploring precision or personalised exercise approaches. Not only is the mean often affected by interventions, but variation may also be impacted. Exploration of variation in studies such as randomised controlled trials (RCTs) can yield insight into interindividual heterogeneity in response to interventions and help determine generalisability of effects. Yet, larger samples sizes than those used for typical mean effects are required when probing variation. Thus, in a field with small samples such as sport and exercise science, exploration of variation through a meta-analytic framework is appealing. Despite the value of embracing and exploring variation alongside mean effects in sport and exercise science it is rarely applied to research synthesis through meta-analysis.We introduce and evaluate different effect size calculations along with models for meta-analysis of variation using relatable examples from resistance training RCTs.

Metrics

Metrics Loading ...

References

Atkinson, G., & Batterham, A. M. (2015). True and false interindividual differences in the physiological response to an intervention. Experimental Physiology, 100(6), 577–588. https://doi.org/10.1113/EP085070

Atkinson, G., Williamson, P., & Batterham, A. M. (2019). Issues in the determination of ‘responders’ and ‘non-responders’ in physiological research. Experimental Physiology, 104(8), 1215–1225. https://doi.org/10.1113/EP087712

Barr, D. J., Levy, R., Scheepers, C., & Tily, H. J. (2013). Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of Memory and Language, 68(3), 10.1016/j.jml.2012.11.001. https://doi.org/10.1016/j.jml.2012.11.001

Bernárdez-Vázquez, R., Raya-González, J., Castillo, D., & Beato, M. (2022). Resistance Training Variables for Optimization of Muscle Hypertrophy: An Umbrella Review. Frontiers in Sports and Active Living, 4. https://www.frontiersin.org/articles/10.3389/fspor.2022.949021

Bonafiglia, J. T., Swinton, P. A., Ross, R., Johannsen, N. M., Martin, C. K., Church, T. S., Slentz, C. A., Ross, L. M., Kraus, W. E., Walsh, J. J., Kenny, G. P., Goldfield, G. S., Prud’homme, D., Sigal, R. J., Earnest, C. P., & Gurd, B. J. (2022). Interindividual Differences in Trainability and Moderators of Cardiorespiratory Fitness, Waist Circumference, and Body Mass Responses: A Large-Scale Individual Participant Data Meta-analysis. Sports Medicine (Auckland, N.Z.). https://doi.org/10.1007/s40279-022-01725-9

Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2010). A basic introduction to fixed-effect and random-effects models for meta-analysis. Research Synthesis Methods, 1(2), 97–111. https://doi.org/10.1002/jrsm.12

Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2021). Introduction to Meta-Analysis. John Wiley & Sons.

Box, G. E. P. (1976). Science and Statistics. Journal of the American Statistical Association, 71(356), 791–799. https://doi.org/10.1080/01621459.1976.10480949

Caldwell, A., & Vigotsky, A. D. (2020). A case against default effect sizes in sport and exercise science. PeerJ, 8, e10314. https://doi.org/10.7717/peerj.10314

Carpinelli, R. N. (2017). Interindividual heterogeneity of adaptations to resistance training. Medicina Sportiva Practica, 18(4), 79–94.

Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Routledge. https://doi.org/10.4324/9780203771587

Cortés Martínez, J. (2021). Constant effect in randomized clinical trials with quantitative outcome: A methodological review. TDX (Tesis Doctorals En Xarxa). https://upcommons.upc.edu/handle/2117/349575

Curtis, P. S., & Wang, X. (1998). A meta-analysis of elevated CO2 effects on woody plant mass, form, and physiology. Oecologia, 113(3), 299–313. https://doi.org/10.1007/s004420050381

Deb, S. K., Brown, D. R., Gough, L. A., Mclellan, C. P., Swinton, P. A., Andy Sparks, S., & Mcnaughton, L. R. (2018). Quantifying the effects of acute hypoxic exposure on exercise performance and capacity: A systematic review and meta-regression. European Journal of Sport Science, 18(2), 243–256. https://doi.org/10.1080/17461391.2017.1410233

Esteves, G. P., Swinton, P., Sale, C., James, R. M., Artioli, G. G., Roschel, H., Gualano, B., Saunders, B., & Dolan, E. (2021). Individual Participant Data Meta-Analysis Provides No Evidence of Intervention Response Variation in Individuals Supplementing With Beta-Alanine. International Journal of Sport Nutrition and Exercise Metabolism, 31(4), 305–313. https://doi.org/10.1123/ijsnem.2021-0038

Exner, R. J., Patel, M. H., Whitener, D. V., Buckner, S. L., Jessee, M. B., & Dankel, S. J. (2022). Does performing resistance exercise to failure homogenize the training stimulus by accounting for differences in local muscular endurance? European Journal of Sport Science, 1–10. https://doi.org/10.1080/17461391.2021.2023657

Fisher, J., Steele, J., Wolf, M., Korakakis, P. A., Smith, D., & Giessing, J. (2022). The Role of Supervision in Resistance Training; an Exploratory Systematic Review and Meta-Analysis: International Journal of Strength and Conditioning, 2(1), Article 1. https://doi.org/10.47206/ijsc.v2i1.101

Glass, G. V. (1976). Primary, Secondary, and Meta-Analysis of Research. Educational Researcher, 5(10), 3–8. https://doi.org/10.2307/1174772

Gurevitch, J., Morrison, J. A., & Hedges, L. V. (2000). The Interaction between Competition and Predation: A Meta-analysis of Field Experiments. The American Naturalist, 155(4), 435–453. https://doi.org/10.1086/303337

Hagger, M. (2006). Meta-analysis in sport and exercise research: Review, recent developments, and recommendations. European Journal of Sport Science, 6(2), 103–115. https://doi.org/10.1080/17461390500528527

Hagger, M. (2022). Meta-analysis. International Review of Sport and Exercise Psychology, 15(1), 120–151. https://doi.org/10.1080/1750984X.2021.1966824

Halliday, T. M., Savla, J., Marinik, E. L., Hedrick, V. E., Winett, R. A., & Davy, B. M. (2017). Resistance training is associated with spontaneous changes in aerobic physical activity but not overall diet quality in adults with prediabetes. Physiology & Behavior, 177, 49–56. https://doi.org/10.1016/j.physbeh.2017.04.013

Halperin, I., Malleron, T., Har-Nir, I., Androulakis-Korakakis, P., Wolf, M., Fisher, J., & Steele, J. (2022). Accuracy in Predicting Repetitions to Task Failure in Resistance Exercise: A Scoping Review and Exploratory Meta-analysis. Sports Medicine, 52(2), 377–390. https://doi.org/10.1007/s40279-021-01559-x

Hecksteden, A., Kraushaar, J., Scharhag-Rosenberger, F., Theisen, D., Senn, S., & Meyer, T. (2015). Individual response to exercise training—A statistical perspective. Journal of Applied Physiology (Bethesda, Md.: 1985), 118(12), 1450–1459. https://doi.org/10.1152/japplphysiol.00714.2014

Hedges, L. V., Gurevitch, J., & Curtis, P. S. (1999). The Meta-Analysis of Response Ratios in Experimental Ecology. Ecology, 80(4), 1150–1156. https://doi.org/10.1890/0012-9658(1999)080[1150:TMAORR]2.0.CO;2

Hedges, L. V., & Nowell, A. (1995). Sex differences in mental test scores, variability, and numbers of high-scoring individuals. Science (New York, N.Y.), 269(5220), 41–45. https://doi.org/10.1126/science.7604277

Hedges, L. V., & Olkin, I. (2014). Statistical Methods for Meta-Analysis. Academic Press.

Hedges, L. V., Tipton, E., & Johnson, M. C. (2010). Robust variance estimation in meta-regression with dependent effect size estimates. Research Synthesis Methods, 1(1), 39–65. https://doi.org/10.1002/jrsm.5

Heidel, K. A., Novak, Z. J., & Dankel, S. J. (2022). Machines and free weight exercises: A systematic review and meta-analysis comparing changes in muscle size, strength, and power. The Journal of Sports Medicine and Physical Fitness, 62(8), 1061–1070. https://doi.org/10.23736/S0022-4707.21.12929-9

Hopkins, W. G. (2015). Individual responses made easy. Journal of Applied Physiology (Bethesda, Md.: 1985), 118(12), 1444–1446. https://doi.org/10.1152/japplphysiol.00098.2015

Hrubeniuk, T. J., Bonafiglia, J. T., Bouchard, D. R., Gurd, B. J., & Sénéchal, M. (2022). Directions for Exercise Treatment Response Heterogeneity and Individual Response Research. International Journal of Sports Medicine, 43(1), 11–22. https://doi.org/10.1055/a-1548-7026

Kass, R. E., & Raftery, A. E. (1995). Bayes Factors. Journal of the American Statistical Association, 90(430), 773–795. https://doi.org/10.1080/01621459.1995.10476572

Kelley, G. A. (2022). Precision exercise medicine in rheumatology: Don’t put the cart before the horse. Clinical Rheumatology, 41(8), 2277–2279. https://doi.org/10.1007/s10067-022-06260-6

Kelley, G. A., Kelley, K. S., & Callahan, L. F. (2022). Are There Interindividual Differences in Anxiety as a Result of Aerobic Exercise Training in Adults with Fibromyalgia? An Ancillary Meta-analysis of Randomized Controlled Trials. Archives of Physical Medicine and Rehabilitation, S0003-9993(22)00007-7. https://doi.org/10.1016/j.apmr.2021.12.019

Kelley, G. A., Kelley, K. S., & Pate, R. R. (2020). Are There Inter-Individual Differences in Fat Mass and Percent Body Fat as a Result of Aerobic Exercise Training in Overweight and Obese Children and Adolescents? A Meta-Analytic Perspective. Childhood Obesity (Print), 16(5), 301–306. https://doi.org/10.1089/chi.2020.0056

Lajeunesse, M. J. (2011). On the meta-analysis of response ratios for studies with correlated and multi-group designs. Ecology, 92(11), 2049–2055. https://doi.org/10.1890/11-0423.1

Lajeunesse, M. J. (2015). Bias and correction for the log response ratio in ecological meta-analysis. Ecology, 96(8), 2056–2063. https://doi.org/10.1890/14-2402.1

Mills, H. L., Higgins, J. P. T., Morris, R. W., Kessler, D., Heron, J., Wiles, N., Davey Smith, G., & Tilling, K. (2021). Detecting Heterogeneity of Intervention Effects Using Analysis and Meta-analysis of Differences in Variance Between Trial Arms. Epidemiology (Cambridge, Mass.), 32(6), 846–854. https://doi.org/10.1097/EDE.0000000000001401

Morris, S. B. (2008). Estimating Effect Sizes From Pretest-Posttest-Control Group Designs. Organizational Research Methods, 11(2), 364–386. https://doi.org/10.1177/1094428106291059

Morris, W. F., Hufbauer, R. A., Agrawal, A. A., Bever, J. D., Borowicz, V. A., Gilbert, G. S., Maron, J. L., Mitchell, C. E., Parker, I. M., Power, A. G., Torchin, M. E., & Vázquez, D. P. (2007). Direct and interactive effects of enemies and mutualists on plant performance: A meta-analysis. Ecology, 88(4), 1021–1029. https://doi.org/10.1890/06-0442

Nakagawa, S., & Cuthill, I. C. (2007). Effect size, confidence interval and statistical significance: A practical guide for biologists. Biological Reviews of the Cambridge Philosophical Society, 82(4), 591–605. https://doi.org/10.1111/j.1469-185X.2007.00027.x

Nakagawa, S., Poulin, R., Mengersen, K., Reinhold, K., Engqvist, L., Lagisz, M., & Senior, A. M. (2015). Meta-analysis of variation: Ecological and evolutionary applications and beyond. Methods in Ecology and Evolution, 6(2), 143–152. https://doi.org/10.1111/2041-210X.12309

Nakagawa, S., & Schielzeth, H. (2012). The mean strikes back: Mean-variance relationships and heteroscedasticity. Trends in Ecology & Evolution, 27(9), 474–475; author reply 475-476. https://doi.org/10.1016/j.tree.2012.04.003

Nuzzo, J., Pinto, M. D., Nosaka, K., & Steele, J. (2023a). The Eccentric:Concentric Strength Ratio of Human Skeletal Muscle In Vivo: Meta-analysis of the Influences of Sex, Age, Joint Action, and Velocity. Sports Medicine, 53(6), 1125–1136. https://doi.org/10.1007/s40279-023-01851-y

Nuzzo, J., Pinto, M., Nosaka, K., & Steele, J. (2022). How much stronger are muscles eccentrically than concentrically? : Meta-analysis of the influences of sex, age, joint action, and velocity. SportRxiv. https://doi.org/10.51224/SRXIV.197

Nuzzo, J., Pinto, M., Nosaka, K., & Steele, J. (2023b). Maximal number of repetitions at percentages of the one repetition maximum: A meta-regression and moderator analysis of sex, age, training status, and exercise. SportRxiv. https://doi.org/10.51224/SRXIV.291

Olsson-Collentine, A., Van Aert, R. C. M., Bakker, M., & Wicherts, J. (2023). Meta-analyzing the multiverse: A peek under the hood of selective reporting. Psychological Methods. https://doi.org/10.1037/met0000559

Pickering, C., & Kiely, J. (2019). Do Non-Responders to Exercise Exist-and If So, What Should We Do About Them? Sports Medicine (Auckland, N.Z.), 49(1), 1–7. https://doi.org/10.1007/s40279-018-01041-1

Plackett, R. L. (1958). Studies in the History of Probability and Statistics: VII. The Principle of the Arithmetic Mean. Biometrika, 45(1/2), 130–135. https://doi.org/10.2307/2333051

Polito, M. D., Papst, R. R., & Farinatti, P. (2021). Moderators of strength gains and hypertrophy in resistance training: A systematic review and meta-analysis. Journal of Sports Sciences, 39(19), 2189–2198. https://doi.org/10.1080/02640414.2021.1924978

Raudenbush, S. W., & Bryk, A. S. (1987). Examining Correlates of Diversity. Journal of Educational Statistics, 12(3), 241–269. https://doi.org/10.3102/10769986012003241

Robinson, Z., Helms, E., Trexler, E., Steele, J., Hall, M., Huang, C.-J., & Zourdos, M. (2023). Optimizing Research Methodology for the Detection of Individual Response Variation in Resistance Training. SportRxiv. https://doi.org/10.51224/SRXIV.340

Ross, R., Goodpaster, B. H., Koch, L. G., Sarzynski, M. A., Kohrt, W. M., Johannsen, N. M., Skinner, J. S., Castro, A., Irving, B. A., Noland, R. C., Sparks, L. M., Spielmann, G., Day, A. G., Pitsch, W., Hopkins, W. G., & Bouchard, C. (2019). Precision exercise medicine: Understanding exercise response variability. British Journal of Sports Medicine, 53(18), 1141–1153. https://doi.org/10.1136/bjsports-2018-100328

Senior, A. M., Viechtbauer, W., & Nakagawa, S. (2020). Revisiting and expanding the meta-analysis of variation: The log coefficient of variation ratio. Research Synthesis Methods, 11(4), 553–567. https://doi.org/10.1002/jrsm.1423

Steegen, S., Tuerlinckx, F., Gelman, A., & Vanpaemel, W. (2016). Increasing Transparency Through a Multiverse Analysis. Perspectives on Psychological Science, 11(5), 702–712. https://doi.org/10.1177/1745691616658637

Steele, J., Fisher, J. P., Giessing, J., Androulakis-Korakakis, P., Wolf, M., Kroeske, B., & Reuters, R. (2022). Long-Term Time-Course of Strength Adaptation to Minimal Dose Resistance Training Through Retrospective Longitudinal Growth Modeling. Research Quarterly for Exercise and Sport, 0(0), 1–18. https://doi.org/10.1080/02701367.2022.2070592

Steele, J., Fisher, J., Skivington, M., Dunn, C., Arnold, J., Tew, G., Batterham, A. M., Nunan, D., O’Driscoll, J. M., Mann, S., Beedie, C., Jobson, S., Smith, D., Vigotsky, A., Phillips, S., Estabrooks, P., & Winett, R. (2017). A higher effort-based paradigm in physical activity and exercise for public health: Making the case for a greater emphasis on resistance training. BMC Public Health, 17(1), 300. https://doi.org/10.1186/s12889-017-4209-8

Steele, J., Malleron, T., Har-Nir, I., Androulakis-Korakakis, P., Wolf, M., Fisher, J. P., & Halperin, I. (2022). Are Trainees Lifting Heavy Enough? Self-Selected Loads in Resistance Exercise: A Scoping Review and Exploratory Meta-analysis. Sports Medicine. https://doi.org/10.1007/s40279-022-01717-9

Steele, J., Plotkin, D., Van Every, D., Rosa, A., Zambrano, H., Mendelovits, B., Carrasquillo-Mercado, M., Grgic, J., & Schoenfeld, B. J. (2021). Slow and Steady, or Hard and Fast? A Systematic Review and Meta-Analysis of Studies Comparing Body Composition Changes between Interval Training and Moderate Intensity Continuous Training. Sports (Basel, Switzerland), 9(11), 155. https://doi.org/10.3390/sports9110155

Swinton, P. A., Hemingway, B. S., Saunders, B., Gualano, B., & Dolan, E. (2018). A Statistical Framework to Interpret Individual Response to Intervention: Paving the Way for Personalized Nutrition and Exercise Prescription. Frontiers in Nutrition, 5, 41. https://doi.org/10.3389/fnut.2018.00041

Swinton, P. A., Katherine, B., Andy, H., Leon, G., John, P., Rodrigo, R. A., Patrick, M., & Andrew, M. (2022). Interpreting magnitude of change in strength and conditioning: Effect size selection, threshold values and Bayesian updating Journal of Sports Sciences. Journal of Sports Sciences.

Taylor, L. R. (1961). Aggregation, Variance and the Mean. Nature, 189(4766), Article 4766. https://doi.org/10.1038/189732a0

Tenan, M. S., Vigotsky, A. D., & Caldwell, A. R. (2020). Comment on: “A Method to Stop Analyzing Random Error and Start Analyzing Differential Responders to Exercise”. Sports Medicine, 50(2), 431–434. https://doi.org/10.1007/s40279-019-01249-9

Usui, T., Macleod, M. R., McCann, S. K., Senior, A. M., & Nakagawa, S. (2021). Meta-analysis of variation suggests that embracing variability improves both replicability and generalizability in preclinical research. PLOS Biology, 19(5), e3001009. https://doi.org/10.1371/journal.pbio.3001009

Van den Noortgate, W., López-López, J. A., Marín-Martínez, F., & Sánchez-Meca, J. (2013). Three-level meta-analysis of dependent effect sizes. Behavior Research Methods, 45(2), 576–594. https://doi.org/10.3758/s13428-012-0261-6

Viechtbauer, W. (2010). Conducting Meta-Analyses in R with the metafor Package. Journal of Statistical Software, 36, 1–48. https://doi.org/10.18637/jss.v036.i03

Vigotsky, A., Nuckols, G. L., Fisher, J., Heathers, J., Krieger, J., Schoenfeld, B. J., Giessing, J., & Steele, J. (2020). Improbable data patterns in the work of Barbalho et al. SportRxiv. https://doi.org/10.31236/osf.io/sg3wm

Wagenmakers, E.-J. (2007). A practical solution to the pervasive problems ofp values. Psychonomic Bulletin & Review, 14(5), 779–804. https://doi.org/10.3758/BF03194105

Wolf, M., Androulakis-Korakakis, P., Fisher, J., Schoenfeld, B., & Steele, J. (2023). Partial Vs Full Range of Motion Resistance Training: A Systematic Review and Meta-Analysis. International Journal of Strength and Conditioning, 3(1), Article 1. https://doi.org/10.47206/ijsc.v3i1.182

Yang, Y., Hillebrand, H., Lagisz, M., Cleasby, I., & Nakagawa, S. (2022). Low statistical power and overestimated anthropogenic impacts, exacerbated by publication bias, dominate field studies in global change biology. Global Change Biology, 28(3), 969–989. https://doi.org/10.1111/gcb.15972

Posted

2022-10-26 — Updated on 2023-11-13

Versions