Analysing multitrait-multimethod data with structural equation models for ordinal variables applying the WLSMV estimator: what sample size is needed for valid results?

Convergent and discriminant validity of psychological constructs can best be examined in the framework of multitrait-multimethod (MTMM) analysis. To gain information at the level of single items, MTMM models for categorical variables have to be applied. The CTC(M-1) model is presented as an example of an MTMM model for ordinal variables. Based on an empirical application of the CTC(M-1) model, a complex simulation study was conducted to examine the sample size requirements of the robust weighted least squares mean- and variance-adjusted chi(2) test of model fit (WLSMV estimator) implemented in Mplus. In particular, the simulation study analysed the chi(2) approximation, the parameter estimation bias, the standard error bias, and the reliability of the WLSMV estimator depending on the varying number of items per trait-method unit (ranging from 2 to 8) and varying sample sizes (250, 500, 750, and 1000 observations). The results showed that the WLSMV estimator provided a good -- albeit slightly liberal -- chi(2) approximation and stable and reliable parameter estimates for models of reasonable complexity (2-4 items) and small sample sizes (at least 250 observations). When more complex models with 5 or more items were analysed, larger sample sizes of at least 500 observations were needed. The most complex model with 9 trait-method units and 8 items (72 observed variables) requires sample sizes of at least 1000 observations.