Traditional Chinese medicine (TCM) is characterized by synergistic therapeutic effect involving multiple compounds and targets, which provide potential new therapy for the treatment of complex cancer conditions. However, the main contributors and the underlying mechanisms of synergistic TCM cancer therapies remain largely undetermined. Machine learning now provides a new approach to determine synergistic compound combinations from complex components of TCM. In this study, a prediction model based on extreme gradient boosting (XGBoost) algorithm was constructed by integrating gene expression data of different cancer cell lines, targets information of natural compounds and drug response data. Radix Paeoniae Rubra (RPR) was selected as a model herbal sample to evaluate the reliability of the constructed model. The optimal XGBoost prediction model achieved a good performance with Mean Square Error (MSE) of 0.66, Mean Absolute Error (MAE) of 0.61, and the Root Mean Squared Error (RMSE) of 0.81 on test dataset. The superior synergistic anti-tumor combinations of D15 (Paeonol[Formula: see text][Formula: see text][Formula: see text]Ethyl gallate) and D13 (Paeoniflorin[Formula: see text][Formula: see text][Formula: see text]Paeonol) were successfully predicted from RPR and experimentally validated on MCF-7 cells. Moreover, the combination of D13 could work as a main contributor to a synergistic anti-proliferative activity in the compatibility of RPR and Cortex Moutan (CM). Our XGBoost model could be a reliable tool for the efficient prediction of synergistic anti-tumor multi-compound combinations from TCM.