Deep learning has emerged as a leading machine learning tool in object detection and has attracted attention with its achievements in progressing medical image analysis. Convolutional Neural Networks (CNNs) are the most preferred method of deep learning algorithms for this purpose and they have an essential role in the detection and potential early diagnosis of colon cancer. In this article, we hope to bring a perspective to progress in this area by reviewing deep learning practices for colon cancer analysis. This study first presents an overview of popular deep learning architectures used in colon cancer analysis. After that, all studies related to colon cancer analysis are collected under the field of colon cancer and deep learning, then they are divided into five categories that are detection, classification, segmentation, survival prediction, and inflammatory bowel diseases. Then, the studies collected under each category are summarized in detail and listed. We conclude our work with a summary of recent deep learning practices for colon cancer analysis, a critical discussion of the challenges faced, and suggestions for future research. This study differs from other studies by including 135 recent academic papers, separating colon cancer into five different classes, and providing a comprehensive structure. We hope that this study is beneficial to researchers interested in using deep learning techniques for the diagnosis of colon cancer.