Anosmia is one of the most prevalent symptoms of SARS-CoV-2 infection during the COVID-19 pandemic. However, the cellular mechanism behind the sudden loss of smell has not yet been investigated. The initial step of odour detection takes place in the pseudostratified olfactory epithelium (OE) mainly composed of olfactory sensory neurons surrounded by supporting cells known as sustentacular cells. The olfactory neurons project their axons to the olfactory bulb in the central nervous system offering a potential pathway for pathogens to enter the central nervous system by bypassing the blood brain barrier. In the present study, we explored the impact of SARS-CoV-2 infection on the olfactory system in golden Syrian hamsters. We observed massive damage of the OE as early as 2 days post nasal instillation of SARS-CoV-2, resulting in a major loss of cilia necessary for odour detection. These damages were associated with infection of a large proportion of sustentacular cells but not of olfactory neurons, and we did not detect any presence of the virus in the olfactory bulbs. We observed massive infiltration of immune cells in the OE and lamina propria of infected animals, which may contribute to the desquamation of the OE. The OE was partially restored 14 days post infection. Anosmia observed in COVID-19 patient is therefore likely to be linked to a massive and fast desquamation of the OE following sustentacular cells infection with SARS-CoV-2 and subsequent recruitment of immune cells in the OE and lamina propria.