Few to Big: Prototype Expansion Network via Diffusion Learner for Point Cloud Few-shot Semantic Segmentation

2025/09/16 by Zhao, Qianguang, Yan Zhou, Wang, Dongli +5
Computer Science · Earth and Planetary Sciences · Engineering · #3D Shape Modeling and Analysis #3D Surveying and Cultural Heritage #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Image Processing and 3D Reconstruction

paper · pdf · doi:10.48550/arxiv.2509.12878

openalex publication_date 2025/09/16 · openalex created_date 2025/10/18 · openalex updated_date 2026/08/01

Abstract

Few-shot 3D point cloud semantic segmentation aims to segment novel categories using a minimal number of annotated support samples. However, prototypes derived from the limited non-structural point cloud support set are often misaligned and have a small capacity, hindering effective gen eralization to novel categories. This stems from two core issues: i) the prototype possess limited representational capacity fails to cover the full intra-class diversity of a novel category, and ii) the prototypes suffer from misalignment with the query space due to the inter-set inconsistency between support and query sets. To address these issues, our work focuses on leveraging the few support samples to construct a well-aligned big-capacity prototype. Motivated by the powerful generative capabilities of diffusion models, we re-purpose its pre-trained conditional encoder to provide rich feature components for prototype ex pansion. Subsequently, a push-pull force aligns this expanded prototype towards the query feature space. Under this setup, we introduce the Prototype Expansion Network (PENet), a framework that constructs aligned big-capacity prototypes from two complementary feature sources. Specifically, PENet employs a dual-stream learner architecture: it retains a conventional fully supervised Intrinsic Learner (IL) to distill representative features, while introducing a novel Diffusion Learner (DL) to provide rich generalizable features. The resulting dual prototypes are then processed by a Prototype Assimilation Module (PAM), which adopts a push-pull attention block to align the prototypes with the query space. Furthermore, a Prototype Calibration Mechanism (PCM) regularizes the final big-capacity prototype to prevent semantic drift. Extensive experiments on the S3DIS and ScanNet datasets demonstrate that PENet outperforms state-of-the-art methods across various few-shot settings.

Citations

• Enhancing Video-Language Representations With Structural Spatio-Temporal Alignment
• No Time to Train: Empowering Non-Parametric Networks for Few-shot 3D Scene Segmentation
• Rethinking Few-shot 3D Point Cloud Semantic Segmentation
• Point Cloud Pre-training with Diffusion Models
• Boosting Few-shot 3D Point Cloud Segmentation via Query-Guided Enhancement
• Few-Shot 3D Point Cloud Semantic Segmentation via Stratified Class-Specific Attention Based Transformer Network
• Your Diffusion Model is Secretly a Zero-Shot Classifier
• Erasing Concepts from Diffusion Models
• Adding Conditional Control to Text-to-Image Diffusion Models
• Adding Conditional Control to Text-to-Image Diffusion Models
• Scalable Diffusion Models with Transformers
• Point-E: A System for Generating 3D Point Clouds from Complex Prompts
• LION: Latent Point Diffusion Models for 3D Shape Generation
• Visual Prompting via Image Inpainting
• Bidirectional Feature Globalization for Few-shot Semantic Segmentation of 3D Point Cloud Scenes
• Stratified Transformer for 3D Point Cloud Segmentation
• Objects in Semantic Topology
• Improved Denoising Diffusion Probabilistic Models
• Few-shot 3D Point Cloud Semantic Segmentation
• Denoising Diffusion Probabilistic Models
• SemanticPOSS: A Point Cloud Dataset with Large Quantity of Dynamic Instances
• Meta-Transfer Learning through Hard Tasks
• Dynamic Graph CNN for Learning on Point Clouds
• Dynamic Graph CNN for Learning on Point Clouds
• Prototypical Networks for Few-shot Learning
• ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes
• Matching Networks for One Shot Learning

Related

• EPSegFZ: Efficient Point Cloud Semantic Segmentation for Few- and Zero-Shot Scenarios with Language Guidance
• Rethinking Few-shot 3D Point Cloud Semantic Segmentation
• Query-aware Hub Prototype Learning for Few-Shot 3D Point Cloud Semantic Segmentation
• Open-world Point Cloud Semantic Segmentation: A Human-in-the-loop Framework
• Unleashing the Potential of the Diffusion Model in Few-shot Semantic Segmentation