Keypoints as Dynamic Centroids for Unified Human Pose and Segmentation

2025/05/17 by Niaz Ahmad, Ahmad, Niaz, Jawad Khan +7
Computer Science · Engineering · #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Human Motion and Animation #Human Pose and Action Recognition #Robot Manipulation and Learning

paper · pdf · doi:10.48550/arxiv.2505.12130

openalex publication_date 2025/05/17 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

The dynamic movement of the human body presents a fundamental challenge for human pose estimation and body segmentation. State-of-the-art approaches primarily rely on combining keypoint heatmaps with segmentation masks but often struggle in scenarios involving overlapping joints or rapidly changing poses during instance-level segmentation. To address these limitations, we propose Keypoints as Dynamic Centroid (KDC), a new centroid-based representation for unified human pose estimation and instance-level segmentation. KDC adopts a bottom-up paradigm to generate keypoint heatmaps for both easily distinguishable and complex keypoints and improves keypoint detection and confidence scores by introducing KeyCentroids using a keypoint disk. It leverages high-confidence keypoints as dynamic centroids in the embedding space to generate MaskCentroids, allowing for swift clustering of pixels to specific human instances during rapid body movements in live environments. Our experimental evaluations on the CrowdPose, OCHuman, and COCO benchmarks demonstrate KDC's effectiveness and generalizability in challenging scenarios in terms of both accuracy and runtime performance. The implementation is available at: https://sites.google.com/view/niazahmad/projects/kdc.

Citations

• VISUALCENT: Visual Human Analysis using Dynamic Centroid Representation
• Occluded Human Pose Estimation based on Limb Joint Augmentation
• RTMO: Towards High-Performance One-Stage Real-Time Multi-Person Pose Estimation
• Group Pose: A Simple Baseline for End-to-End Multi-person Pose Estimation
• Rethinking pose estimation in crowds: overcoming the detection information-bottleneck and ambiguity
• Explicit Box Detection Unifies End-to-End Multi-Person Pose Estimation
• QueryPose: Sparse Multi-Person Pose Regression via Spatial-Aware Part-Level Query
• AlphaPose: Whole-Body Regional Multi-Person Pose Estimation and Tracking in Real-Time
• PoseTrans: A Simple Yet Effective Pose Transformation Augmentation for Human Pose Estimation
• Learning Quality-aware Representation for Multi-person Pose Regression
• The Center of Attention: Center-Keypoint Grouping via Attention for Multi-Person Pose Estimation
• Learning Local-Global Contextual Adaptation for Multi-Person Pose Estimation
• FCPose: Fully Convolutional Multi-Person Pose Estimation with Dynamic Instance-Aware Convolutions
• Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regression
• SIMPLE: SIngle-network with Mimicking and Point Learning for Bottom-up Human Pose Estimation
• Multi-Instance Pose Networks: Rethinking Top-Down Pose Estimation
• PI-Net: Pose Interacting Network for Multi-Person Monocular 3D Pose Estimation
• Graph-PCNN: Two Stage Human Pose Estimation with Graph Pose Refinement
• Gaussian kernel smoothing
• Learning Delicate Local Representations for Multi-Person Pose Estimation
• DirectPose: Direct End-to-End Multi-Person Pose Estimation
• HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimation
• Single-Stage Multi-Person Pose Machines
• Deep High-Resolution Representation Learning for Visual Recognition
• Deep High-Resolution Representation Learning for Visual Recognition
• Human Pose Estimation for Real-World Crowded Scenarios
• Multi-Person Pose Estimation with Enhanced Channel-wise and Spatial Information
• YOLACT: Real-time Instance Segmentation
• PifPaf: Composite Fields for Human Pose Estimation
• CrowdPose: Efficient Crowded Scenes Pose Estimation and A New Benchmark
• MultiPoseNet: Fast Multi-Person Pose Estimation using Pose Residual Network
• Simple Baselines for Human Pose Estimation and Tracking
• PersonLab: Person Pose Estimation and Instance Segmentation with a Bottom-Up, Part-Based, Geometric Embedding Model
• Path Aggregation Network for Instance Segmentation
• Integral Human Pose Regression
• Mask R-CNN
• Towards Accurate Multi-person Pose Estimation in the Wild
• RMPE: Regional Multi-person Pose Estimation
• Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields
• DeeperCut: A Deeper, Stronger, and Faster Multi-Person Pose Estimation Model
• Instance-sensitive Fully Convolutional Networks
• Convolutional Pose Machines
• Deep Residual Learning for Image Recognition
• DeepCut: Joint Subset Partition and Labeling for Multi Person Pose Estimation
• Fully Convolutional Networks for Semantic Segmentation
• Microsoft COCO: Common Objects in Context

Related

• VISUALCENT: Visual Human Analysis using Dynamic Centroid Representation
• Keypoint Communities
• Rethinking Keypoint Representations: Modeling Keypoints and Poses as Objects for Multi-Person Human Pose Estimation
• Bottom-Up Human Pose Estimation by Ranking Heatmap-Guided Adaptive Keypoint Estimates
• ProbPose: A Probabilistic Approach to 2D Human Pose Estimation