The integrated fast detection technology for electric bikes, riders, helmets, and license plates is of great significance for maintaining traffic safety. YOLOv5 is one of the most advanced single-stage object detection algorithms. However, it is difficult to deploy on embedded systems, such as unmanned aerial vehicles (UAV), with limited memory and computing resources because of high computational load and high memory requirements. In this paper, a lightweight YOLOv5 model (SG-YOLOv5) is proposed for the fast detection of the helmet and license plate of electric bikes, by introducing two mechanisms to improve the original YOLOv5. Firstly, the YOLOv5s backbone network and the Neck part are lightened by combining the two lightweight networks, ShuffleNetv2 and GhostNet, included. Secondly, by adopting an Add-based feature fusion method, the number of parameters and the floating-point operations (FLOPs) are effectively reduced. On this basis, a scene-based non-truth suppression method is proposed to eliminate the interference of pedestrian heads and license plates on parked vehicles, and then the license plates of the riders without helmets can be located through the inclusion relation of the target boxes and can be extracted. To verify the performance of the SG-YOLOv5, the experiments are conducted on a homemade RHNP dataset, which contains four categories: rider, helmet, no-helmet, and license plate. The results show that, the SG-YOLOv5 has the same mean average precision (mAP0.5) as the original; the number of model parameters, the FLOPs, and the model file size are reduced by 90.8%, 80.5%, and 88.8%, respectively. Additionally, the number of frames per second (FPS) is 2.7 times higher than that of the original. Therefore, the proposed SG-YOLOv5 can effectively achieve the purpose of lightweight and improve the detection speed while maintaining great detection accuracy.