Kirsten rat sarcoma viral oncogene homolog (KRAS) protein inhibitors are a promising class of therapeutics, but research on molecules that effectively penetrate the blood-brain barrier (BBB) remains limited, which is crucial for treating central nervous system (CNS) malignancies. Although molecular generation models have recently advanced drug discovery, they often overlook the complexity of biological and chemical factors, leaving room for improvement. In this study, we present a structure-constrained molecular generation workflow designed to optimize lead compounds for both drug efficacy and drug absorption properties. Our approach utilizes a variational autoencoder (VAE) generative model integrated with reinforcement learning for multi-objective optimization. This method specifically aims to enhance BBB permeability (BBBp) while maintaining high-affinity substructures of KRAS inhibitors. To support this, we incorporate a specialized KRAS BBB predictor based on active learning and an affinity predictor employing comparative learning models. Additionally, we introduce two novel metrics, the knowledge-integrated reproduction score (KIRS) and the composite diversity score (CDS), to assess structural performance and biological relevance. Retrospective validation with KRAS inhibitors, AMG510 and MRTX849, demonstrates the framework's effectiveness in optimizing BBBp and highlights its potential for real-world drug development applications. This study provides a robust framework for accelerating the structural enhancement of lead compounds, advancing the drug development process across diverse targets.