An analysis of dimensionality using factor analysis (true-score theory) and Rasch measurement: what is the difference? Which method is better?

One often hears the question asked, "For questionnaire data measuring a variable, what difference does it make to use factor analysis/principal components analysis (true-score theory) or Rasch measurement in testing for dimensionality?" This paper reports both factor analysis and Rasch measurement analysis for two sets of data. One set of data measures social anxiety for primary school students (N=436, I=10) and the second measures attitude to mathematics for primary-aged students (N=774, I=10). For both sets of data, the factor analysis suggests that the scores are reliable, and that inferences can be made that are valid for measuring school anxiety and attitude to mathematics. For both sets of data analyzed with Rasch measurement techniques, the reliability of the measures, the dimensionality of the measures, and the initial conceptualisation of the items, are called into question. It suggests that one cannot make valid inferences from the measures that were initially set up for true-score theory. The Rasch analysis suggests that items intended to measure a variable should be initially developed on a conceptualized scale from easy to hard, and that students should answer the items from this perspective, so that the Rasch analysis of the data tests this conceptualisation, and a linear scale can be created based on a mathematical measurement model with consistent units (logits).