In recent decades, we have witnessed a progressive decline in male fertility. This is partly related to the increased prevalence of chronic diseases (e.g., obesity and diabetes mellitus) and risky lifestyle behaviors. These conditions alter male fertility through various non-genetic mechanisms. However, there is increasing evidence that they are also capable of causing sperm epigenetic alterations, which, in turn, can cause infertility. Furthermore, these modifications could be transmitted to offspring, altering their general and reproductive health. Therefore, these epigenetic modifications could represent one of the causes of the progressive decline in sperm count recorded in recent decades. This review focuses on highlighting epigenetic modifications at the sperm level induced by non-genetic causes of infertility. In detail, the effects on DNA methylation, histone modifications, and the expression profiles of non-coding RNAs are evaluated. Finally, a focus on the risk of transgenerational inheritance is presented. Our narrative review aims to demonstrate how certain conditions can alter gene expression, potentially leading to the transmission of anomalies to future generations. It emphasizes the importance of the early detection and treatment of reversible conditions (such as obesity and varicocele) and the modification of risky lifestyle behaviors. Addressing these issues is crucial for individual health, in preserving fertility, and in ensuring the well-being of future generations.