Cells escape the need for mitogens at a restriction point several hours before entering S phase. The restriction point has been proposed to result from CDK4/6 initiating partial Rb phosphorylation to trigger a bistable switch whereby cyclin E-CDK2 and Rb mutually reinforce each other to induce Rb hyperphosphorylation. Here, using single-cell analysis, we unexpectedly found that cyclin E/A-CDK activity can only maintain Rb hyperphosphorylation starting at the onset of S phase and that CDK4/6 activity, but not cyclin E/A-CDK activity, is required to hyperphosphorylate Rb throughout G1 phase. Mitogen removal in G1 results in a gradual loss of CDK4/6 activity with a high likelihood of cells sustaining Rb hyperphosphorylation until S phase, at which point cyclin E/A-CDK activity takes over. Thus, it is short-term memory, or transient hysteresis, in CDK4/6 activity following mitogen removal that sustains Rb hyperphosphorylation, demonstrating a probabilistic rather than an irreversible molecular mechanism underlying the restriction point.