Bilayer membranes composed of cholesterol and phospholipids exhibit diverse forms of nonideal mixing. In particular, many previous studies document macroscopic liquid-liquid phase separation as well as nanometer-scale heterogeneity in membranes of phosphatidylcholine (PC) lipids and cholesterol. Here, we present experimental measurements of cholesterol chemical potential (μ) in binary membranes containing dioleoyl PC (DOPC), 1-palmitoyl-2-oleoyl PC (POPC), or dipalmitoyl PC (DPPC), and in ternary membranes of DOPC and DPPC, referenced to crystalline cholesterol. μ is the thermodynamic quantity that dictates the availability of cholesterol to bind other factors, and notably must be equal between coexisting phases of a phase separated mixture. It is simply related to concentration under conditions of ideal mixing, but is far from ideal for the majority of lipid mixtures investigated here. Measurements of μ can vary with phospholipid composition by 1.5 kT at constant cholesterol mole fraction implying a more than fivefold change in its availability for binding receptors and other reactions. Experimental measurements are fit to thermodynamic models including cholesterol-DPPC complexes or pairwise interactions between lipid species to provide intuition about the magnitude of interactions. These findings reinforce that μ depends on membrane composition overall, suggesting avenues for cells to alter the availability of cholesterol without varying cholesterol concentration.