Phenolic acids are widely found in fruits and vegetables. The inhibitory effect of phenolic acids on α-amylase, a key enzyme for starch digestion, has attracted the attention of researchers. To further investigate the effects of different substituents on the benzene ring of phenolic acid on the inhibition of α-amylase activity, in vitro experiments and molecular docking were used. The structure-activity relationships of 17 phenolic acids with benzoic acid as the parent nucleus were analyzed by determining their half inhibitory concentration (IC) toward α-amylase. The results showed that 2,3,4-trihydroxybenzoic acid had the strongest inhibitory effect on α-amylase with an IC value of 17.30 ± 0.73 mM. According to the structure-activity analysis, the hydroxyl group at the 2-position on the benzene ring had a strong positive effect on the inhibitory activity of α-amylase, while methoxylation at the 2-position and hydroxylation at the 5-position had a negative effect. Molecular docking revealed that hydrogen bonding and hydrophobic interactions were involved in the inhibition, with hydrogen bonding being the primary force. These findings provide a more comprehensive understanding of phenolic acids as inhibitors of α-amylase and provide new ideas for the design of dietary formulations for diabetic patients.