Self-healing hydrogels like tissues or organs which are repaired automatically in response to damage show great promise. However, it remains a challenge to develop novel functional nanoparticles as crosslinkers to prepare tough and self-healing nanocomposite hydrogels. Here, we report the preparation of water-soluble ultrasmall aluminum hydroxide nanoparticles with a diameter of 2-3 nm through a simple sol-gel method. Furthermore, a tough nanocomposite hydrogel is prepared by the in situ copolymerization of acylamide and 2-acrylamido-2-methyl propane sulfonic acid in the presence of aluminum hydroxide nanoparticles. The resulting hydrogels exhibit high compressive strength of 18.9 MPa and an elongation at break of ∼2100%. Importantly, the Al-NC gel displayed a high self-healing efficiency of 86% without any external stimulus at room temperature. Moreover, we found an interesting multi-hierarchical porous morphology of the Al-NC gel depending on the contents of the aluminum hydroxide nanoparticles. The tough nanocomposite hydrogel might provide a novel promising avenue for designing advanced self-healable soft materials for various biomedical applications.