PURPOSE: Esophageal cancer is a common malignant tumor in life, which seriously affects human health. In order to reduce the work intensity of doctors and improve detection accuracy, we proposed esophageal cancer detection using deep learning. The characteristics of deep learning: association and structure, activity and experience, essence and variation, migration and application, value and evaluation. METHOD: The improved Faster RCNN esophageal cancer detection in this paper introduces the online hard example mining (OHEM) mechanism into the system, and the experiment used 1520 gastrointestinal CT images from 421 patients. Then, we compare the overall performance of Inception-v2, Faster RCNN, and improved Faster RCNN through F-1 measure, mean average precision (mAP), and detection time. RESULTS: The experiment shows that the overall performance of the improved Faster RCNN is higher than the other two networks. The F-1 measure of our method reaches 95.71%, the mAP reaches 92.15%, and the detection time per CT is only 5.3s. CONCLUSION: Through comparative analysis on the esophageal cancer image data set, the experimental results show that the introduction of online hard example mining mechanism in the Faster RCNN algorithm can improve the detection accuracy to a certain extent.