IEEE Trans Image Process. 2018 Apr;27(4):1888-1900. doi: 10.1109/TIP.2017.2779600. Epub 2017 Dec 4.
Hand detection is essential for many hand related tasks, e.g., recovering hand pose and understanding gesture. However, hand detection in uncontrolled environments is challenging due to the flexibility of wrist joint and cluttered background. We propose a convolutional neural network (CNN), which formulates in-plane rotation explicitly to solve hand detection and rotation estimation jointly. Our network architecture adopts the backbone of faster R-CNN to generate rectangular region proposals and extract local features. The rotation network takes the feature as input and estimates an in-plane rotation which manages to align the hand, if any in the proposal, to the upward direction. A derotation layer is then designed to explicitly rotate the local spatial feature map according to the rotation network and feed aligned feature map for detection. Experiments show that our method outperforms the state-of-the-art detection models on widely-used benchmarks, such as Oxford and Egohands database. Further analysis show that rotation estimation and classification can mutually benefit each other.
手的检测对于许多与手相关的任务至关重要，例如恢复手的姿势和理解手势。然而，由于手腕关节的灵活性和杂乱的背景，在不受控制的环境中进行手的检测具有挑战性。我们提出了一种卷积神经网络（CNN），它明确地将平面内旋转纳入其中，以联合解决手的检测和旋转估计问题。我们的网络架构采用了更快的 R-CNN 的骨干网络，生成矩形区域建议并提取局部特征。旋转网络将特征作为输入，估计一个平面内的旋转，从而将建议中的任何手部对齐到向上的方向。然后设计了一个去旋转层，根据旋转网络显式地旋转局部空间特征图，并为检测提供对齐的特征图。实验表明，我们的方法在广泛使用的基准测试（如牛津和 Egohands 数据库）上优于最先进的检测模型。进一步的分析表明，旋转估计和分类可以相互受益。
