Competing Magnetic Phases in SbTe ( = Ho and Tb).

The interplay between a topological electronic structure and magnetism may result in intricate physics. In this work, we describe a case of rather peculiar coexistence or competition of several magnetic phases below seemingly single antiferromagnetic transition in SbTe ( = Ho and Tb) topological semimetals, the magnetic members of the ZrSiS/PbFCl structure type (space group 4/). Neutron diffraction experiments reveal a complex multi-step order below = 3.8 K ( = Ho) and = 6.4 K ( = Tb). Magnetic phases can be described using four propagation vectors = (/ 0 0) and = (/ 0 /) at a base temperature of 1.7 K, which transform into incommensurate vectors ' = (/ - δ 0 0) and = (/ - δ 0 /) at elevated temperatures in both compounds. Together with the refined models of magnetic structures, we present the group theoretical analysis of magnetic symmetry of the proposed solutions. These results prompt further investigations of the relation between the electronic structure of those semimetals and the determined antiferromagnetic ordering existing therein.