The association between cause and effect is usually probabilistic. Memories triggered by ambiguous cues may be altered or biased into a more negative perception in psychiatric diseases. Understanding the formation and modulation of this probabilistic association is important for revealing the nature of aversive memory and alterations in brain diseases. We found that 50% conditioned and unconditioned stimuli (CS-US) association during Pavlovian fear conditioning results in reduced fear responses and neural spiking in the dorsomedial prefrontal cortex (dmPFC) due to enhanced inhibition from dmPFC parvalbumin (PV) neurons. Formation of probabilistic memory is associated with increased synaptic inputs to PV-neurons and requires activation of ventral hippocampus, which detects CS-US mismatch during conditioning. Stress prior to conditioning impairs the formation of probabilistic memory by abolishing PV-neuronal plasticity, while stress prior to memory retrieval reverts enhanced PV-neuron activity. In conclusion, PV-neurons tailor learned responses to fit brain state at the moment of retrieval.