Suppression of antiferromagnetic order by strain-enhanced frustration in honeycomb cobaltate.

Layered honeycomb cobaltates are predicted as promising for realizing the Kitaev quantum spin liquid, a many-body quantum entangled ground state characterized by fractional excitations. However, they exhibit antiferromagnetic ordering at low temperatures, hindering the expected quantum state. We demonstrate that controlling the trigonal distortion of CoO octahedra is crucial to suppress antiferromagnetic order through enhancing frustration in layered honeycomb cobaltates. Using heterostructure engineering on CuCoSbO thin films, we adjust the trigonal distortion of CoO octahedra and the resulting trigonal crystal field. The original Néel temperature of 16 kelvin in bulk CuCoSbO decreases (increases) to 7.8 kelvin (22.7 kelvin) in strained CuCoSbO films by decreasing (increasing) the magnitude of the trigonal crystal fields. The first-principles calculation suggests the enhancement of geometrical frustration as the origin of the suppression of antiferromagnetism. This finding supports the potential of layered honeycomb cobaltate heterostructures and strain engineering in realizing extremely elusive quantum phases of matter.