HDC: Hierarchical Distillation for Multi-level Noisy Consistency in Semi-Supervised Fetal Ultrasound Segmentation

2025/04/14 by Tran Lan Trinh Le, Le, Tran Quoc Khanh, Nguyen Tuan Vu +13 · 2 citations
Computer Science · Medicine · #Advanced Neural Network Applications #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Fetal and Pediatric Neurological Disorders

paper · pdf · doi:10.48550/arxiv.2504.09876

openalex publication_date 2025/04/14 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Transvaginal ultrasound is a critical imaging modality for evaluating cervical anatomy and detecting physiological changes. However, accurate segmentation of cervical structures remains challenging due to low contrast, shadow artifacts, and indistinct boundaries. While convolutional neural networks (CNNs) have demonstrated efficacy in medical image segmentation, their reliance on large-scale annotated datasets presents a significant limitation in clinical ultrasound imaging. Semi-supervised learning (SSL) offers a potential solution by utilizing unlabeled data, yet existing teacher-student frameworks often encounter confirmation bias and high computational costs. In this paper, a novel semi-supervised segmentation framework, called HDC, is proposed incorporating adaptive consistency learning with a single-teacher architecture. The framework introduces a hierarchical distillation mechanism with two objectives: Correlation Guidance Loss for aligning feature representations and Mutual Information Loss for stabilizing noisy student learning. The proposed approach reduces model complexity while enhancing generalization. Experiments on fetal ultrasound datasets, FUGC and PSFH, demonstrate competitive performance with reduced computational overhead compared to multi-teacher models.

Citations

• Improving Generalization in Visual Reasoning via Self-Ensemble
• Intrapartum Ultrasound Image Segmentation of Pubic Symphysis and Fetal Head Using Dual Student-Teacher Framework with CNN-ViT Collaborative Learning
• Blurry-Consistency Segmentation Framework with Selective Stacking on Differential Interference Contrast 3D Breast Cancer Spheroid
• V_kD: Improving Knowledge Distillation using Orthogonal Projections
• Dual Mean-Teacher: An Unbiased Semi-Supervised Framework for Audio-Visual Source Localization
• Semi-Supervised Semantic Segmentation using Redesigned Self-Training for White Blood Cells
• Alternate Diverse Teaching for Semi-supervised Medical Image Segmentation
• Switching Temporary Teachers for Semi-Supervised Semantic Segmentation
• Logic-induced Diagnostic Reasoning for Semi-supervised Semantic Segmentation
• Class Attention Transfer Based Knowledge Distillation
• Conflict-Based Cross-View Consistency for Semi-Supervised Semantic Segmentation
• When CNN Meet with ViT: Towards Semi-Supervised Learning for Multi-Class Medical Image Semantic Segmentation
• Knowledge Distillation from A Stronger Teacher
• Knowledge Distillation with the Reused Teacher Classifier
• Semi-Supervised Medical Image Segmentation via Cross Teaching between CNN and Transformer
• Information Theoretic Representation Distillation
• Perturbed and Strict Mean Teachers for Semi-supervised Semantic Segmentation
• FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling
• Semi-Supervised Semantic Segmentation with Cross Pseudo Supervision
• Distilling Knowledge via Knowledge Review
• Barlow Twins: Self-Supervised Learning via Redundancy Reduction
• In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning
• Knowledge Distillation: A Survey
• Semi-Supervised Semantic Segmentation with Cross-Consistency Training
• FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
• Uncertainty-aware Self-ensembling Model for Semi-supervised 3D Left Atrium Segmentation
• Deep Co-Training for Semi-Supervised Image Recognition
• Knowledge Transfer with Jacobian Matching
• Decoupled Deep Neural Network for Semi-supervised Semantic Segmentation
• Distilling the Knowledge in a Neural Network
• Rényi Divergence and Kullback-Leibler Divergence
• CoDi: Conditional Diffusion Distillation for Higher-Fidelity and Faster Image Generation

Cited by

• Graph-Theoretic Consistency for Robust and Topology-Aware Semi-Supervised Histopathology Segmentation
• Learning Disentangled Stain and Structural Representations for Semi-Supervised Histopathology Segmentation

Related

• Leveraging Information Divergence for Robust Semi-Supervised Fetal Ultrasound Image Segmentation
• Semi-supervised Cervical Segmentation on Ultrasound by A Dual Framework for Neural Networks
• Semi-supervised Learning of Fetal Anatomy from Ultrasound
• Intrapartum Ultrasound Image Segmentation of Pubic Symphysis and Fetal Head Using Dual Student-Teacher Framework with CNN-ViT Collaborative Learning
• HiCo: Hierarchical Contrastive Learning for Ultrasound Video Model Pretraining