Carotid artery atherosclerosis, the result of a multitude of vascular risk factors, is a promising marker for use in risk stratification. Recent evidence suggests that carotid artery atherosclerosis affects cognitive function and is an independent risk factor for the development of cognitive impairment. Both atherosclerosis and cognitive impairment develop over a prolonged period (years), and due to the aging population, markers to identify persons at risk are needed. Carotid artery atherosclerosis can easily be visualized using non-invasive ultrasound, potentially enabling early and intensified risk factor management to preserve cognitive function or delay further decline. However, the burden of atherosclerosis and temporal exposure required to pose a risk of cognitive impairment is unclear. This mini-review aims to explore the available evidence on the association between carotid atherosclerosis and cognition, and furthermore identify the remaining gaps in knowledge.