ReMoE: Report-Guided Mixture-of-Experts for Multimodal OCT/OCTA Anomaly Detection

2026/07/31 by Zihan Nie, Qincheng Qiao, Muhao Xu +4
Computer Science · #cs.CV

paper · pdf

arxiv created 2026/07/31 · arxiv updated 2026/08/03

Abstract

Multimodal medical anomaly detection identifies samples deviating from normal patterns, where scarce abnormal cases make normality modeling from normal data practical. In retinal Optical Coherence Tomography (OCT) and OCT Angiography (OCTA) anomaly detection, existing unsupervised methods rely on visual feature distributions, reconstruction residuals, or encoder-decoder discrepancies, making anomaly scores depend on appearance-level deviations, while multimodal normality also contains semantic organization described in normal medical reports. To this end, we propose Report-Guided Mixture-of-Experts (ReMoE), which distills normal report semantics into an image-to-text prior student, builds modality-aware priors, and uses Report-Guided Modality Modulation (RMM) to modulate features through mixture-of-experts routing. Experiments on a private OCT/OCTA dataset with paired normal reports and a public OCTA500-3MM setting using a fixed normal report demonstrate state-of-the-art performance.

Citations

• AnyAD: Unified Any-Modality Anomaly Detection in Incomplete Multi-Sequence MRI
• Unlocking the Potential of Reverse Distillation for Anomaly Detection
• Vision Transformers Need Registers
• DINOv2: Learning Robust Visual Features without Supervision
• DRAEM -- A discriminatively trained reconstruction embedding for surface\n anomaly detection
• Towards Total Recall in Industrial Anomaly Detection
• Student-Teacher Feature Pyramid Matching for Anomaly Detection
• PaDiM: a Patch Distribution Modeling Framework for Anomaly Detection and\n Localization
• Contrastive Learning of Medical Visual Representations from Paired Images and Text