Martvel George, Riemer Stefanie
Tech4Animals Lab, University of Haifa, Abba Khoushy Ave 199, 3498838, Haifa, Israel.
Messerli Research Institute, University of Veterinary Medicine, Veterinärplatz 1, 1210, Vienna, Austria.
Sci Rep. 2025 Sep 2;15(1):32331. doi: 10.1038/s41598-025-15741-y.
Automated analysis of facial expressions is a vibrant field in human affective computing, while research in nonhuman animals is still in its early stages. Compared to labour-intensive manual coding, automation can provide a more reliable and objective alternative, eliminating subjectivity and bias. However, using automated approaches of facial analysis in nonhuman animals "in the wild", i.e. outside of controlled laboratory conditions, is a challenge given the nature of noisy datasets. Here we present the first study using a fully automated analysis of facial landmarks associated with different emotional states in a morphologically diverse sample of pet dogs. We applied a novel AI-pipeline to study fear expressions of dogs in their home environment, analysing owner-provided video recordings during a real-life firework situation on New Year's Eve in comparison to a control evening without fireworks. Using a static geometric morphometrics-inspired analysis, the pipeline allows for quantifying dog facial expressions in an extremely noisy and diverse "in the wild" dataset, encompassing various breeds, angles and environments. We used an automated facial landmark system of 36 dog facial landmarks based on the Dog Facial Action Coding System. Due to the great variety in morphology of the included dogs, landmarks denoting the ear pinnae were excluded. Nonetheless, landmarks relating to the base of the ears differentiated most strongly between the conditions, suggesting backwards-drawn ears as the best indicator of the firework condition, which is in agreement with manually coded data. Additionally, the firework condition was associated with more mouth-opening, possibly reflecting panting in a subset of dogs. We conclude that automated analysis of dog facial expressions, based on the previously validated landmark system, is feasible in a diverse sample of pet dogs, paving the way towards automated emotion detection.
面部表情的自动分析是人类情感计算中一个充满活力的领域，而对非人类动物的研究仍处于早期阶段。与劳动密集型的人工编码相比，自动化可以提供一种更可靠、更客观的替代方法，消除主观性和偏差。然而，鉴于嘈杂数据集的性质，在非人类动物“在自然环境中”，即在受控实验室条件之外，使用面部分析的自动化方法是一项挑战。在这里，我们展示了第一项研究，该研究对形态多样的宠物狗样本中与不同情绪状态相关的面部标志进行了全自动分析。我们应用了一种新颖的人工智能流程来研究狗在其家庭环境中的恐惧表情，分析主人提供的除夕夜现实生活中烟花燃放时的视频记录，并与没有烟花的对照夜晚进行比较。使用受静态几何形态计量学启发的分析方法，该流程能够在一个极其嘈杂和多样的“自然环境中”数据集中量化狗的面部表情，该数据集涵盖了各种品种、角度和环境。我们使用了基于犬类面部动作编码系统的36个犬类面部标志的自动面部标志系统。由于所纳入的狗在形态上差异很大，表示耳廓的标志被排除在外。尽管如此，与耳根相关的标志在不同条件下的区分最为明显，表明耳朵向后拉是烟花燃放情况的最佳指标，这与人工编码数据一致。此外，烟花燃放情况与更多的张嘴动作相关，这可能反映了一部分狗的喘气情况。我们得出结论，基于先前经过验证的标志系统对狗的面部表情进行自动分析，在形态多样的宠物狗样本中是可行的，为自动情绪检测铺平了道路。
