A machine learning based method for tracking of simultaneously imaged neural activity and body posture of freely moving maggot.

To understand neural basis of animal behavior, it is necessary to monitor neural activity and behavior in freely moving animal before building relationship between them. Here we use light sheet fluorescence microscope (LSFM) combined with microfluidic chip to simultaneously capture neural activity and body movement in small freely behaving Drosophila larva. We develop a transfer learning based method to simultaneously track the continuously changing body posture and activity of neurons that move together using a sub-region tracking network with a precise landmark estimation network for the inference of target landmark trajectory. Based on the tracking of each labelled neuron, the activity of the neuron indicated by fluorescent intensity is calculated. For each video, annotation of only 20 frames in a video is sufficient to yield human-level accuracy for all other frames. The validity of this method is further confirmed by reproducing the activity pattern of PMSIs (period-positive median segmental interneurons) and larval movement as previously reported. Using this method, we disclosed the correlation between larval movement and left-right asymmetry in activity of a group of unidentified neurons labelled by R52H01-Gal4 and further confirmed the roles of these neurons in bilateral balance of body contraction during larval crawling by genetic inhibition of these neurons. Our method provides a new tool for accurate extraction of neural activities and movement of freely behaving small-size transparent animals.