Reversible phosphorylation relies on highly regulated kinases and phosphatases that target specific substrates to control diverse cellular processes. Here, we address how protein phosphatase activity is directed to the correct substrates under the correct conditions. The serine/threonine phosphatase SpoIIE from Bacillus subtilis, a member of the widespread protein phosphatase 2C (PP2C) family of phosphatases, is activated by movement of a conserved α-helical element in the phosphatase domain to create the binding site for the metal cofactor. We hypothesized that this conformational switch could provide a general mechanism for control of diverse members of the PP2C family of phosphatases. The B. subtilis phosphatase RsbU responds to different signals, acts on a different substrates, and produces a more graded response than SpoIIE. Using an unbiased genetic screen, we isolated mutants in the α-helical switch region of RsbU that are constitutively active, indicating conservation of the switch mechanism. Using phosphatase activity assays with phosphoprotein substrates, we found that both phosphatases integrate substrate recognition with activating signals to control metal-cofactor binding and substrate dephosphorylation. This integrated control provides a mechanism for PP2C family of phosphatases to produce specific responses by acting on the correct substrates, under the appropriate conditions.