Mercury ion (Hg) is one of the most toxic heavy metal ions and lowering the detection limit of Hg is always a challenge in analytical chemistry and environmental analysis. In this work, sulfhydryl functionalized carbon quantum dots (HS-CQDs) were synthesized through a one-pot hydrothermal method. The obtained HS-CQDs were able to detect mercury ions Hg rapidly and sensitively through fluorescence quenching, which may be ascribed to the formation of nonfluorescent ground-state complexes and electron transfer reaction between HS-CQDs and Hg. A modification of the HS-CQD surface by -SH was confirmed using Fourier transform infrared spectroscopy (FTIR) and X-ray photoelectron spectroscopy (XPS). The HS-CQDs sensing system obtained a good linear relationship over a Hg concentration ranging from 0.45 μM to 2.1 μM with a detection limit of 12 nM. Delightfully, the sensor has been successfully used to detect Hg in real samples with satisfactory results. This means that the sensor has the potential to be used for testing actual samples.