2025/11/25 by A. V. Lemeshko, Lemeshko, Andrey, B. M. Gabdullin +9
Computer Science · Engineering · #3D Shape Modeling and Analysis #Advanced Neural Network Applications #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Robotics and Sensor-Based Localization
paper · pdf · doi:10.48550/arxiv.2511.20253
openalex publication_date 2025/11/25 · openalex created_date 2025/11/28 · openalex updated_date 2026/07/28
3D object detection is fundamental for spatial understanding. Real-world environments demand models capable of recognizing diverse, previously unseen objects, which remains a major limitation of closed-set methods. Existing open-vocabulary 3D detectors relax annotation requirements but still depend on training scenes, either as point clouds or images. We take this a step further by introducing Zoo3D, the first training-free 3D object detection framework. Our method constructs 3D bounding boxes via graph clustering of 2D instance masks, then assigns semantic labels using a novel open-vocabulary module with best-view selection and view-consensus mask generation. Zoo3D operates in two modes: the zero-shot Zoo3D0, which requires no training at all, and the self-supervised Zoo3D1, which refines 3D box prediction by training a class-agnostic detector on Zoo3D0-generated pseudo labels. Furthermore, we extend Zoo3D beyond point clouds to work directly with posed and even unposed images. Across ScanNet200 and ARKitScenes benchmarks, both Zoo3D0 and Zoo3D1 achieve state-of-the-art results in open-vocabulary 3D object detection. Remarkably, our zero-shot Zoo3D0 outperforms all existing self-supervised methods, hence demonstrating the power and adaptability of training-free, off-the-shelf approaches for real-world 3D understanding. Code is available at https://github.com/col14m/zoo3d .