2024/12/05 by Ahmad Darkhalil, Darkhalil, Ahmad, Rhodri Guerrier +5 · 1 voice · 2 citations
Psychology · Engineering · Computer Science · #Educational Games and Gamification #Human Motion and Animation #Augmented Reality Applications
paper · pdf · doi:10.48550/arxiv.2412.04592
We introduce EgoPoints, a benchmark for point tracking in egocentric videos. We annotate 4.7K challenging tracks in egocentric sequences. Compared to the popular TAP-Vid-DAVIS evaluation benchmark, we include 9x more points that go out-of-view and 59x more points that require re-identification (ReID) after returning to view. To measure the performance of models on these challenging points, we introduce evaluation metrics that specifically monitor tracking performance on points in-view, out-of-view, and points that require re-identification. We then propose a pipeline to create semi-real sequences, with automatic ground truth. We generate 11K such sequences by combining dynamic Kubric objects with scene points from EPIC Fields. When fine-tuning point tracking methods on these sequences and evaluating on our annotated EgoPoints sequences, we improve CoTracker across all metrics, including the tracking accuracy δ^⋆avg by 2.7 percentage points and accuracy on ReID sequences (ReIDδavg) by 2.4 points. We also improve δ^⋆avg and ReIDδavg of PIPs++ by 0.3 and 2.8 respectively.