The relationship between cholesterol and cognitive function is unclear from the previous studies. This study was conducted to explore this association in older Chinese adults. Data were from the Shanghai Aging Study, comprising 3,836 residents aged 50 years or over in an urban community. Diagnoses of dementia and mild cognitive impairment were established according to the fourth edition of diagnostic and statistical manual of mental disorders (DSM-IV) and Petersen criteria. Multivariate logistic regression models, non-matched and propensity score (PS) matched, were used to examine the association between cholesterol levels and cognitive function. There was a significantly higher proportion of participants with low levels of total cholesterol (TC) and low density lipoprotein cholesterol (LDL-C) in the dementia group than in groups without dementia ( < 0.05). High LDL-C level was inversely associated with dementia, with a negative trend in the PS matched model. TC and high density lipoprotein cholesterol (HDL-C) were not significantly related to dementia in either non-matched models or PS matched models. Our result indicates that high level of LDL-C is inversely associated with dementia. High level of LDL-C may be considered as a potential protective factor against cognition decline.