The determination of transcriptome profiles that mediate immune therapy in cancer remains a major clinical and biological challenge. Despite responses induced by immune-check points inhibitors (ICIs) in diverse tumor types and all the big breakthroughs in cancer immunotherapy, most patients with solid tumors do not respond to ICI therapies. It still remains a big challenge to predict the ICI treatment response. Here, we propose a framework with multiple prior knowledge networks guided for immune checkpoints inhibitors prediction-DeepOmix-ICI (or ICInet for short). ICInet can predict the immune therapy response by leveraging geometric deep learning and prior biological knowledge graphs of gene-gene interactions. Here, we demonstrate more than 600 ICI-treated patients with ICI response data and gene expression profile to apply on ICInet. ICInet was used for ICI therapy responses prediciton across different cancer types-melanoma, gastric cancer and bladder cancer, which includes 7 cohorts from different data sources. ICInet is able to robustly generalize into multiple cancer types. Moreover, the performance of ICInet in those cancer types can outperform other ICI biomarkers in the clinic. Our model [area under the curve (AUC = 0.85)] generally outperformed other measures, including tumor mutational burden (AUC = 0.62) and programmed cell death ligand-1 score (AUC = 0.74). Therefore, our study presents a prior-knowledge guided deep learning method to effectively select immunotherapy-response-associated biomarkers, thereby improving the prediction of immunotherapy response for precision oncology.