T lymphocytes migrate into organs and interact with local cells to perform their functions. How human T lymphocytes communicate with organ-specific cells and participate in pathobiological processes remains unresolved. Brain infiltration of T lymphocytes is associated with multiple neurological disorders. Thus, to characterize the behavior of human T lymphocytes reaching the human brain, we performed time-lapse microscopy on human CD8+ T lymphocytes co-cultured with either primary human astrocytes or neurons. Using traditional manual and visual assessment of microscopy data, we identified distinct CD8+ T lymphocyte motility behaviors. However, such characterization is time and labor-intensive. In this work, we trained and validated a machine-learning model for the automated classification of behaviors of CD8+ T lymphocytes interacting with astrocytes and neurons. A balanced random forest was trained for the binary classification of established classes of cell behaviors (synapse vs. kinapse) as well as visually identified behaviors (scanning, dancing, and poking). Feature selection was performed during 3-fold cross-validation using the minimum redundancy maximum relevance algorithm. Results show promising performances when tested on a held-out dataset of CD8+ T lymphocytes interacting with astrocytes with a new experimenter and a held-out independent dataset of CD8+ T lymphocytes interacting with neurons. When tested on the independent CD8+ T cell-neuron dataset, the final model achieved a binary classification accuracy of 0.82 and a 3-class accuracy of 0.79. This novel automated classification approach could significantly reduce the time required to label cell motility behaviors while facilitating the identification of interactions of T lymphocytes with multiple cell types.