Force sensing beyond standard quantum limit with optomechanical "soft" mode induced by nonlinear interaction.

We consider an optomechanical (OM) system that consists of a mechanical and an optical mode interacting through linear and quadratic OM dispersive couplings. The system is operated in unresolved sideband limit with a high quality factor mechanical resonator. Such a system acts as a parametrically driven oscillator, giving access to an intensity-assisted tunability of the spring constant. This enables the operation of the OM system in its "soft mode," wherein the mechanical spring softens and responds with a lower resonance frequency. We show that this soft mode can be exploited to nonlinearize backaction noise, which yields higher force sensitivity beyond the conventional standard quantum limit.