Generative chemical language models (CLMs) have demonstrated success in learning language-based molecular representations for de novo drug design. Here, we integrate structure-based drug design (SBDD) principles with CLMs to go from protein structure to novel small-molecule ligands, without a priori knowledge of ligand chemistry. Using Augmented Hill-Climb, we successfully optimise multiple objectives within a practical timeframe, including protein-ligand complementarity. Resulting de novo molecules contain known or promising adenosine A receptor ligand chemistry that is not available in commercial vendor libraries, accessing commercially novel areas of chemical space. Experimental validation demonstrates a binding hit rate of 88%, with 50% having confirmed functional activity, including three nanomolar ligands and two novel chemotypes. The two strongest binders are co-crystallised with the A receptor, revealing their binding mechanisms that can be used to inform future iterations of structure-based de novo design, closing the AI SBDD loop.