IEEE Trans Neural Netw Learn Syst. 2024 Sep;35(9):12746-12759. doi: 10.1109/TNNLS.2023.3264647. Epub 2024 Sep 3.
Freezing of Gait (FoG) is a common symptom of Parkinson's disease (PD), manifesting as a brief, episodic absence, or marked reduction in walking, despite a patient's intention to move. Clinical assessment of FoG events from manual observations by experts is both time-consuming and highly subjective. Therefore, machine learning-based FoG identification methods would be desirable. In this article, we address this task as a fine-grained human action recognition problem based on vision inputs. A novel deep learning architecture, namely, higher order polynomial transformer (HP-Transformer), is proposed to incorporate pose and appearance feature sequences to formulate fine-grained FoG patterns. In particular, a higher order self-attention mechanism is proposed based on higher order polynomials. To this end, linear, bilinear, and trilinear transformers are formulated in pursuit of discriminative fine-grained representations. These representations are treated as multiple streams and further fused by a cross-order fusion strategy for FoG detection. Comprehensive experiments on a large in-house dataset collected during clinical assessments demonstrate the effectiveness of the proposed method, and an area under the receiver operating characteristic (ROC) curve (AUC) of 0.92 is achieved for detecting FoG.
冻结步态（Freezing of Gait，FoG）是帕金森病（Parkinson's disease，PD）的一种常见症状，表现为短暂的、间歇性的行走缺失或明显减少，尽管患者有移动的意愿。专家通过手动观察对 FoG 事件进行临床评估既费时又高度主观。因此，基于机器学习的 FoG 识别方法是可取的。在本文中，我们将此任务视为基于视觉输入的细粒度人体动作识别问题。提出了一种新的深度学习架构，即高阶多项式转换器（higher order polynomial transformer，HP-Transformer），用于合并姿势和外观特征序列，以形成细粒度的 FoG 模式。具体来说，基于高阶多项式提出了一种高阶自注意力机制。为此，制定了线性、双线性和三线性转换器，以追求有区别的细粒度表示。这些表示被视为多个流，并通过跨阶融合策略进一步融合，以进行 FoG 检测。在临床评估期间收集的大型内部数据集上进行的全面实验证明了该方法的有效性，用于检测 FoG 的接收者操作特征（receiver operating characteristic，ROC）曲线下面积（area under the receiver operating characteristic，AUC）达到 0.92。
