Radiomics research is rapidly growing in recent years, but more concerns on radiomics reliability are also raised. This review attempts to update and overview the current status of radiomics reliability research in the ever expanding medical literature from the perspective of a single reliability metric of intraclass correlation coefficient (ICC). To conduct this systematic review, Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were followed. After literature search and selection, a total of 481 radiomics studies using CT, PET, or MRI, covering a wide range of subject and disease types, were included for review. In these highly heterogeneous studies, feature reliability to image segmentation was much more investigated than reliability to other factors, such as image acquisition, reconstruction, post-processing, and feature quantification. The reported ICCs also suggested high radiomics feature reliability to image segmentation. Image acquisition was found to introduce much more feature variability than image segmentation, in particular for MRI, based on the reported ICC values. Image post-processing and feature quantification yielded different levels of radiomics reliability and might be used to mitigate image acquisition-induced variability. Some common flaws and pitfalls in ICC use were identified, and suggestions on better ICC use were given. Due to the extremely high study heterogeneities and possible risks of bias, the degree of radiomics feature reliability that has been achieved could not yet be safely synthesized or derived in this review. More future researches on radiomics reliability are warranted.