Yang Jie, Zeng Ailing, Ren Tianhe, Liu Shilong, Li Feng, Zhang Ruimao, Zhang Lei
IEEE Trans Pattern Anal Mach Intell. 2025 Jul;47(7):5636-5654. doi: 10.1109/TPAMI.2025.3555527.
Detecting keypoints on diverse objects is essential for fine-grained visual understanding and analysis. This paper introduces Enhanced Explicit Box Detection (ED-Pose++), an end-to-end framework that leverages cascade box regression to realize both conventional and interactive multi-object keypoint detection. Unlike traditional one-stage methods, ED-Pose++ innovatively redefines multi-object keypoint detection as a dual-phase explicit box detection, achieving a unified representation and regression optimization process. Specifically, an object detection decoder first extracts each object's position and global features, establishing a good initialization for subsequent keypoint detection. To bring in contextual information near keypoints, we also regard each keypoint as a small box to learn both positions and their related local contents. In practice, an object-to-keypoint detection decoder adopts a collaborative learning strategy between object and keypoint features, facilitating efficient information propagation between global and local perspectives. Rooted on the architecture, we further equip dual-phase box detection with an interactive mechanism that enables the model to refine its predictions based on limited user feedback. During training, we incorporate an error correction scheme to equip the model with an adept self-correction capability for use during inference. The comprehensive experiments demonstrate ED-Pose++'s superior performance in conventional multi-object keypoint detection tasks. For the first time, ED-Pose++ outperforms heatmap-based top-down approaches across various benchmarks, despite operating within a fully end-to-end architecture. The interactive variant also dramatically reduces more than 10 times the labeling effort of 2D keypoint annotation compared with manual-only annotation.
检测不同物体上的关键点对于细粒度视觉理解和分析至关重要。本文介绍了增强显式框检测（ED-Pose++），这是一个端到端框架，它利用级联框回归来实现传统的和交互式的多目标关键点检测。与传统的单阶段方法不同，ED-Pose++创新性地将多目标关键点检测重新定义为双阶段显式框检测，实现了统一的表示和回归优化过程。具体来说，一个目标检测解码器首先提取每个目标的位置和全局特征，为后续的关键点检测建立良好的初始化。为了引入关键点附近的上下文信息，我们还将每个关键点视为一个小框，以学习其位置及其相关的局部内容。在实践中，一个从目标到关键点的检测解码器采用目标和关键点特征之间的协作学习策略，促进全局和局部视角之间的高效信息传播。基于该架构，我们进一步为双阶段框检测配备了一种交互式机制，使模型能够根据有限的用户反馈来优化其预测。在训练期间，我们纳入了一种纠错方案，使模型在推理时具备熟练的自我纠错能力。综合实验证明了ED-Pose++在传统多目标关键点检测任务中的卓越性能。尽管是在完全端到端的架构内运行，但ED-Pose++首次在各种基准测试中超越了基于热图的自上而下方法。与仅手动标注相比，交互式变体还将2D关键点标注的标注工作量大幅减少了10倍以上。
