Competing antiferromagnetic-ferromagnetic states in a Kitaev honeycomb magnet.

The Kitaev model is a rare example of an analytically solvable and physically instantiable Hamiltonian yielding a topological quantum spin liquid ground state. Here we report signatures of Kitaev spin liquid physics in the honeycomb magnet LiCoSbO, built of high-spin (Co) ions, in contrast to the more typical low-spin electron configurations in the presence of large spin-orbit coupling. Neutron powder diffraction measurements, heat capacity, and magnetization studies support the development of a long-range antiferromagnetic order space group of , below at . The magnetic entropy recovered between and 50 K is estimated to be , in good agreement with the value expected for systems close to a Kitaev quantum spin liquid state. The temperature-dependent magnetic order parameter demonstrates a value of 0.19(3), consistent with XY anisotropy and in-plane ordering, with Ising-like interactions between layers. Further, we observe a spin-flop-driven crossover to ferromagnetic order with space group of under an applied magnetic field of at . Magnetic structure analysis demonstrates these magnetic states are competing at finite applied magnetic fields even below the spin-flop transition. Both the compass model, a quantitative comparison of the specific heat of LiCoSbO, and related honeycomb cobaltates to the anisotropic Kitaev model further support proximity to a Kitaev spin liquid state. This material demonstrates the rich playground of high-spin systems for spin liquid candidates and complements known Ir- and Ru-based materials.