Many network models in computational neuroscience rise to the challenge of explaining behavioural phenomena ranging from microseconds to tens of seconds using components operating mostly on a time-scale of milliseconds. These models have in common that the underlying system has a memory, which implies that its output depends on its past input history. In this review we compare how such memory traces or delayed responses may be implemented in different brain areas supporting a diversity of functions.