In order to realize the intelligent online yield estimation of tomato in the plant factory with artificial lighting (PFAL), a recognition method of tomato red fruit and green fruit based on improved yolov3 deep learning model was proposed to count and estimate tomato fruit yield under natural growth state. According to the planting environment and facility conditions of tomato plants, a computer vision system for fruit counting and yield estimation was designed and the new position loss function was based on the generalized intersection over union (GIoU), which improved the traditional YOLO algorithm loss function. Meanwhile, the scale invariant feature could promote the description precision of the different shapes of fruits. Based on the construction and labeling of the sample image data, the K-means clustering algorithm was used to obtain nine prior boxes of different specifications which were assigned according to the hierarchical level of the feature map. The experimental results of model training and evaluation showed that the mean average precision (mAP) of the improved detection model reached 99.3%, which was 2.7% higher than that of the traditional YOLOv3 model, and the processing time for a single image declined to 15 ms. Moreover, the improved YOLOv3 model had better identification effects for dense and shaded fruits. The research results can provide yield estimation methods and technical support for the research and development of intelligent control system for planting fruits and vegetables in plant factories, greenhouses and fields.