Light traps have been widely used as effective tools to monitor multiple agricultural and forest insect pests simultaneously. However, the current detection methods of pests from light trapping images have several limitations, such as exhibiting extremely imbalanced class distribution, occlusion among multiple pest targets, and inter-species similarity. To address the problems, this study proposes an improved YOLOv3 model in combination with image enhancement to better detect crop pests in real agricultural environments. First, a dataset containing nine common maize pests is constructed after an image augmentation based on image cropping. Then, a linear transformation method is proposed to optimize the anchors generated by the k-means clustering algorithm, which can improve the matching accuracy between anchors and ground truths. In addition, two residual units are added to the second residual block of the original YOLOv3 network to obtain more information about the location of the underlying small targets, and one ResNet unit is used in the feature pyramid network structure to replace two DBL(Conv+BN+LeakyReLU) structures to enhance the reuse of pest features. Experiment results show that the mAP and mRecall of our proposed method are improved by 6.3% and 4.61%, respectively, compared with the original YOLOv3. The proposed method outperforms other state-of-the-art methods (SSD, Faster-rcnn, and YOLOv4), indicating that the proposed method achieves the best detection performance, which can provide an effective model for the realization of intelligent monitoring of maize pests.