Gao Yiming, Kuang Zhanghui, Li Guanbin, Zhang Wayne, Lin Liang
IEEE Trans Image Process. 2021;30:8306-8317. doi: 10.1109/TIP.2021.3093784. Epub 2021 Oct 5.
Human-object interaction detection that aims at detecting <human, verb, object> triplets is critical for the holistic human-centric scene understanding. Existing approaches ignore the modeling of correlations among hierarchical human parts and objects. In this work, we introduce a Hierarchical Reasoning Network (HRNet) to capture relations among human parts at multiple scales (including the holistic human, human region, and human keypoint levels) and objects via a unified graph. In particular, HRNet first constructs one multi-level human parts graph, each level of which consists of human parts at one specific scale, objects, and the unions of human part-object pairs as nodes, and their mutual visual and spatial layout relations as intra-level reasoning. To also capture the relations across scales, we further introduce inter-level reasoning between the nodes of two consecutive levels based on the prior of human body structure. The representations of graph nodes are propagated along intra-level and inter-level reasoning in turn during reasoning. Extensive experiments demonstrate our HRNet obtains new state-of-the-art results on three challenging HICO-DET, V-COCO and HOI-A benchmarks, validating the compelling effectiveness of the proposed method.
旨在检测<人、动词、物体>三元组的人与物体交互检测对于以人类为中心的整体场景理解至关重要。现有方法忽略了分层人体部位与物体之间相关性的建模。在这项工作中，我们引入了一种分层推理网络（HRNet），通过一个统一的图来捕捉多尺度（包括整体人体、人体区域和人体关键点级别）的人体部位与物体之间的关系。具体而言，HRNet首先构建一个多级人体部位图，其每一级由一个特定尺度的人体部位、物体以及人体部位-物体对的并集作为节点，以及它们相互的视觉和空间布局关系作为级内推理。为了捕捉跨尺度的关系，我们基于人体结构的先验知识，进一步引入了两个连续级别的节点之间的级间推理。在推理过程中，图节点的表示依次沿着级内和级间推理进行传播。大量实验表明，我们的HRNet在三个具有挑战性的HICO-DET、V-COCO和HOI-A基准测试中取得了新的最优结果，验证了所提方法的显著有效性。
