Virtual reality (VR) and augmented reality (AR) are rapidly growing technologies. Both have been applied within neurosurgery for presurgical planning and intraoperative navigation, but VR and AR technology is particularly promising for the education of neurosurgical trainees. With the increasing demand for high impact yet efficient educational strategies, VR- and AR-based simulators allow neurosurgical residents to practice technical skills in a low-risk setting. Initial studies have confirmed that such simulators increase trainees' confidence, improve their understanding of operative anatomy, and enhance surgical techniques. Knowledge of the history and conceptual underpinnings of these technologies is useful to understand their current and future applications towards neurosurgical training. The technological precursors for VR and AR were introduced as early as the 1800s, and draw from the fields of entertainment, flight simulation, and education. However, computer software and processing speeds are needed to develop widespread VR- and AR-based surgical simulators, which have only been developed within the last 15 years. During that time, several devices had become rapidly adopted by neurosurgeons, and some programs had begun to incorporate them into the residency curriculum. With ever-improving technology, VR and AR are promising additions to a multi-modal training program, enabling neurosurgical residents to maximize their efforts in preparation for the operating room. In this review, we outline the historical development of the VR and AR systems that are used in neurosurgical training and discuss representative examples of the current technology.