In medicinal chemistry programs it is key to design and make compounds that are efficacious and safe. This is a long, complex, and difficult multiparameter optimization process, often including several properties with orthogonal trends. New methods for the automated design of compounds against profiles of multiple properties are thus of great value. Here we present a fragment-based reinforcement learning approach based on an actor-critic model, for the generation of novel molecules with optimal properties. The actor and the critic are both modeled with bidirectional long short-term memory (LSTM) networks. The AI method learns how to generate new compounds with desired properties by starting from an initial set of lead molecules and then improving these by replacing some of their fragments. A balanced binary tree based on the similarity of fragments is used in the generative process to bias the output toward structurally similar molecules. The method is demonstrated by a case study showing that 93% of the generated molecules are chemically valid and more than a third satisfy the targeted objectives, while there were none in the initial set.