Better statistical reporting does not lead to statistical rigour: lessons from two decades of pseudoreplication in mouse-model studies of neurological disorders.

BACKGROUND

Accurately determining the sample size ("N") of a dataset is a key consideration for experimental design. Misidentification of sample size can lead to pseudoreplication, a process of artificially inflating the number of experimental replicates which systematically underestimates variability, overestimates effect sizes and invalidates statistical tests performed on the data. While many journals have adopted stringent requirements with regard to statistical reporting over the last decade, it remains unknown whether such efforts have had a meaningful impact on statistical rigour.

METHODS

Here, we evaluated the prevalence of this type of statistical error among neuroscience studies involving animal models of Fragile-X Syndrome (FXS) and those using animal models of neurological disorders at large published between 2001 and 2024.

RESULTS

We found that pseudoreplication was present in the majority of publication, increasing over time despite marked improvements in statistical reporting over the last decade. This trend generalised beyond the FXS literature to rodent studies of neurological disorders at large between 2012 and 2024, suggesting that pseudoreplication remains a widespread issue in the literature.

LIMITATIONS

The scope of this study was limited to rodent-model studies of neurological disorders which had the potential for being pseudoreplicated, by allowing repeat observations from individual animals. We did not consider reviews or articles whose experimental design could not allow for pseudoreplication, for example studies which reported only behavioural results, or studies which did not use inferential statistics.

CONCLUSIONS

These observations identify an urgent need for better standards in experimental design and increased vigilance for this type of error during peer review. While reporting standards have significantly improved over the past two decades, this alone has not been enough to curb the prevalence of pseudoreplication. We offer suggestions for how this can be remedied as well as quantifying the severity of this particular type of statistical error. Although the examined literature concerns a specific neuroscience-related area of research, the implications of pseudoreplication apply to all fields of empirical research.