Preprint / Version 1

The effects of variance-covariance misspecification in randomised controlled trials with high frequency data collection.

A simulation study in sport science.


  • Paul Swinton



Statistical power, Linear mixed models, covariance structure


Due to problems of low statistical power in sport science with pre-post randomised controlled trials (RCTs), future studies may seek to conduct high frequency data collection to achieve adequate statistical power with feasible sample sizes. Increasing frequency of data collection requires more sophisticated statistical analyses that may have to account for increased complexity in variance-covariance structures. The purpose of this simulation study was to investigate the effects of misspecifying variance-covariance structures when fitting linear mixed models (LMMs) to realistic data generated across a range of intervention durations and measurement frequencies.


Simulated data were created to compare two interventions across either eight or sixteen weeks, with measurements made either once, twice or three times per week. Data were generated according to four models with different variance-covariance structures and fixed effects held constant. The data generating models included: 1) random intercept only; 2) correlated random intercepts and slopes; 3)&4) correlated random intercepts and slopes with autoregressive level one residuals (=0.3, or =0.6). The effects of both underspecifying and overspecifying variance-covariance structures with fitted LMMs were investigated focusing on type I errors of the average treatment effect when set to zero, and statistical power of the average treatment effect when set to a small but non-zero value. Statistical power of likelihood ratio tests to distinguish variance-covariance structures was also estimated. Monte Carlo simulations were performed across the different conditions with 5000 iterations included in each analysis.  


The results showed that underspecifying the variance-covariance structure by fitting a random intercept LMM when more complex structures existed created a high percentage of type I errors that ranged from ~7 to 40%. Percentages of type I errors increased with greater amounts of data from both a longer intervention and increased number of measurements per week. Results also demonstrated that statistical power decreased when the true data generating mechanism included autoregressive level one residuals compared to just correlated random intercepts and slopes. The decline in statistical power was influenced by both the amount of data available and the degree of autocorrelation. Statistical power dropped to its lowest (~0.55) when combining the least amount of data (e.g. eight weeks with one measurement per week) and highest autocorrelation (=0.6). Statistical power of likelihood ratio tests to correctly identify the existence of random slopes was influenced by intervention duration, with low statistical power (0.22 to 0.44) for a short intervention, and high statistical power (>0.99) for a long intervention.

In conclusion, correct specification of the variance-covariance structure is likely to be important for future RCTs in sport science conducted with high frequency measurements. Complex variance-covariance structures particularly with correlated level one residuals may require substantive increases in the number of participants recruited to obtain high statistical power. Researchers should be cautious of omitting random slopes where required given high percentages of type I errors this may cause. Caution is also required when testing for random slopes particularly with relatively small amounts of data that may be achieved with short interventions.


Metrics Loading ...


Hecksteden A, Faude O, Meyer T, Donath L. How to construct, conduct and analyze an exercise training study? Frontiers in physiology. 2018;9:389083.

Swinton, PA. Murphy, A. Comparative effect size distributions in strength and conditioning and implications for future research: A meta-analysis. 2022. Pre-print available from SportRχiv.

Swinton, PA. Is it time to rethink pre-post randomised controlled trials in strength and conditioning? A review of statistical approaches with derivations and simulations. 2023. Pre-print available from SportRχiv.

Abt G, Boreham C, Davison G, Jackson R, Nevill A, Wallace E, Williams M. Power, precision, and sample size estimation in sport and exercise science research. Journal of Sports Sciences. 2020;38(17):1933-5.

Bernards JR, Sato K, Haff GG, Bazyler CD. Current research and statistical practices in sport science and a need for change. Sports. 2017;5(4):87.

Götz FM, Gosling SD, Rentfrow PJ. Small effects: The indispensable foundation for a cumulative psychological science. Perspectives on psychological science. 2022;17(1):205-15.

Swinton, PA. Adequate statistical power in strength and conditioning may be achieved through longer interventions and high frequency outcome measurement. 2023. Pre-print available from SportRχiv.

McElreath R. Statistical rethinking: A Bayesian course with examples in R and Stan. Chapman and Hall/CRC; 2018.

Brown VA. An introduction to linear mixed-effects modeling in R. Advances in Methods and Practices in Psychological Science. 2021;4(1):2515245920960351.

Mirman D. Growth curve analysis and visualization using R. CRC press; 2017.

Hoffman L. Longitudinal analysis: Modeling within-person fluctuation and change. Routledge; 2015.

Swinton, PA. The influence of baseline capability on intervention effects in strength and conditioning: A review of concepts and methods with meta-analysis. Pre-print available from SportRχiv.

Kutner MH, Nachtsheim CJ, Neter J, Li W. Applied linear statistical models. McGraw-hill; 2005.

Browne W, Goldstein H. MCMC sampling for a multilevel model with nonindependent residuals within and between cluster units. Journal of Educational and Behavioral Statistics. 2010;35(4):453-73.

Blozis SA, Craft M. Alternative covariance structures in mixed-effects models: Addressing intra-and inter-individual heterogeneity. Behavior Research Methods. 2023:1-20.

Fitzmaurice GM, Laird NM, Ware JH. Applied longitudinal analysis. John Wiley & Sons; 2012.

Kwok OM, West SG, Green SB. The impact of misspecifying the within-subject covariance structure in multiwave longitudinal multilevel models: A Monte Carlo study. Multivariate Behavioral Research. 2007;42(3):557-92.

Balsalobre-Fernández C, Kipp K. Use of machine-learning and load–velocity profiling to estimate 1-repetition maximums for two variations of the bench-press exercise. Sports. 2021;9(3):39.

Fernandes JF, Dingley AF, Garcia-Ramos A, Perez-Castilla A, Tufano JJ, Twist C. Prediction of one repetition maximum using reference minimum velocity threshold values in young and middle-aged resistance-trained males. Behavioral Sciences. 2021;11(5):71.

Pérez-Castilla A, Fernandes JF, Garcia-Ramos A. Validity of the bench press one-repetition maximum test predicted through individualized load-velocity relationship using different repetition criteria and minimal velocity thresholds. Isokinetics and Exercise Science. 2021;29(4):369-77.

R Core Team (2023). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria.

Weston S, Analytics R. doParallel: Foreach parallel adaptor for the parallel package. R package version. 2019;1:15.

Barr DJ, Levy R, Scheepers C, Tily HJ. Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of memory and language. 2013;68(3):255-78.

Barr DJ. Random effects structure for testing interactions in linear mixed-effects models. Frontiers in psychology. 2013;4:54057.

Scandola M, Tidoni E. Reliability and feasibility of Linear Mixed Models in fully crossed experimental designs. Advances in Methods and Practices in Psychological Science. 2024;7(1):25152459231214454.

Swinton, PA. Assessing individual response to training in sport and exercise. A conceptual and statistical review. 2023. Pre-print available from SportRχiv.

Hrubeniuk TJ, Bonafiglia JT, Bouchard DR, Gurd BJ, Sénéchal M. Directions for exercise treatment response heterogeneity and individual response research. International Journal of Sports Medicine. 2021:11-22.

Ross R, Goodpaster BH, Koch LG, Sarzynski MA, Kohrt WM, Johannsen NM, Skinner JS, Castro A, Irving BA, Noland RC, Sparks LM. Precision exercise medicine: understanding exercise response variability. British journal of sports medicine. 2019;53(18):1141-53.

Steele J, Fisher JP, Smith D, Schoenfeld BJ, Yang Y, Nakagawa S. Meta-analysis of variation in sport and exercise science: examples of application within resistance training research. Journal of Sports Sciences. 2023;41(17):1617-34.

Kelley GA, Kelley KS, Stauffer BL. Effects of resistance training on body weight and body composition in older adults: An inter-individual response difference meta-analysis of randomized controlled trials. Science Progress. 2023;106(2):00368504231179062.

Bonafiglia JT, Swinton PA, Ross R, Johannsen NM, Martin CK, Church TS, Slentz CA, Ross LM, Kraus WE, Walsh JJ, Kenny GP. Interindividual differences in trainability and moderators of cardiorespiratory fitness, waist circumference, and body mass responses: a large-scale individual participant data meta-analysis. Sports Medicine. 2022;52(12):2837-51.

Carlson K, Magnusen M, Walters P. Effect of various training modalities on vertical jump. Research in sports medicine. 2009;17(2):84-94.

Claudino JG, Cronin J, Mezêncio B, McMaster DT, McGuigan M, Tricoli V, Amadio AC, Serrão JC. The countermovement jump to monitor neuromuscular status: A meta-analysis. Journal of science and medicine in sport. 2017;20(4):397-402.

Ruddock AD, Winter EM. Jumping depends on impulse not power. Journal of sports sciences. 2016;34(6):584-5.

Hughes S, Warmenhoven J, Haff GG, Chapman DW, Nimphius S. Countermovement jump and squat jump force-time curve analysis in control and fatigue conditions. The Journal of Strength & Conditioning Research. 2022;36(10):2752-61.

Oldham PD. A note on the analysis of repeated measurements of the same subjects. Journal of chronic diseases. 1962;15(10):969-77.

Blance A, Tu YK, Gilthorpe MS. A multilevel modelling solution to mathematical coupling. Statistical Methods in Medical Research. 2005;14(6):553-65.