EM3M: An Electron Micrograph Dataset for Microstructural Segmentation and Generation

2025/08/22 by Nan wang, wang, Nan, Zhiyi Xia +17 · 1 citation
Biochemistry, Genetics and Molecular Biology · Materials Science · #Advanced Electron Microscopy Techniques and Applications #Computer Vision and Pattern Recognition (cs.CV) #Electron and X-Ray Spectroscopy Techniques #FOS: Computer and information sciences #Machine Learning in Materials Science

paper · pdf · doi:10.48550/arxiv.2508.16239

openalex publication_date 2025/08/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Quantitative microstructural characterization is fundamental to materials science, and electron micrographs (EMs) provide indispensable high-resolution insights. However, progress in deep learning-based analysis of EMs has been hampered by the scarcity of large-scale, expert-annotated public datasets. To address this issue, we introduce EM3M, a large-scale and multimodal dataset for instance-level understanding of EMs. EM3M comprises 5,091 high-quality EMs, approximately 3 million instance segmentation annotations, and image-level textual descriptions with disentangled attributes. The dataset is constructed through a rigorous multi-stage curation and validation pipeline, with comprehensive statistical analyses to ensure reliability and reproducibility. Building upon these curated image-text pairs, we further provide a text-to-image diffusion model that serves as a controllable data augmentation engine, demonstrating that synthetic augmentation consistently improves downstream segmentation performance. To establish a systematic benchmark, we evaluate representative instance segmentation methods on EM3M. Our results reveal that conventional detection-based and query-based methods struggle with the extreme instance densities and textural complexities inherent in EMs. We additionally provide an optimized flow-based baseline to facilitate fair comparison and future research. EM3M Dataset: https://huggingface.co/datasets/UniParser/EM3M, the generative engine Generation: https://huggingface.co/UniParser/EM3M-Gen, and an online demo Segmentation demo: https://www.bohrium.com/apps/uni-aims are publicly available to support future research in automated materials analysis.

Citations

• fMRI2GES: Co-speech Gesture Reconstruction from fMRI Signal with Dual Brain Decoding Alignment
• Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
• MicroVQA: A Multimodal Reasoning Benchmark for Microscopy-Based Scientific Research
• MMSci: A Dataset for Graduate-Level Multi-Discipline Multimodal Scientific Understanding
• CellViT: Vision Transformers for Precise Cell Segmentation and Classification
• Micrograph segmentations for DDEVD
• Segment Anything
• FastInst: A Simple Query-Based Model for Real-Time Instance Segmentation
• GPT-4 Technical Report
• RTMDet: An Empirical Study of Designing Real-Time Object Detectors
• DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation
• DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation
• Mask DINO: Towards A Unified Transformer-based Framework for Object Detection and Segmentation
• High-Resolution Image Synthesis with Latent Diffusion Models
• High-Resolution Image Synthesis with Latent Diffusion Models
• Masked-attention Mask Transformer for Universal Image Segmentation
• Data privacy protection in microscopic image analysis for material data mining
• Recent advances and applications of deep learning methods in materials science
• A deep learned nanowire segmentation model using synthetic data augmentation
• LoRA: Low-Rank Adaptation of Large Language Models
• Emerging Properties in Self-Supervised Vision Transformers
• Emerging Properties in Self-Supervised Vision Transformers
• Synthetic Image Rendering Solves Annotation Problem in Deep Learning Nanoparticle Segmentation
• Denoising Diffusion Probabilistic Models
• Cascade R-CNN: High Quality Object Detection and Instance Segmentation
• Dari Barisan ke Pakatan: berubahnya dinamiks Pilihan Raya UmumKuala Lumpur 1955-2013
• MMDetection: Open MMLab Detection Toolbox and Benchmark
• YOLACT: Real-time Instance Segmentation
• Hybrid Task Cascade for Instance Segmentation
• HoVer-Net: Simultaneous Segmentation and Classification of Nuclei in\n Multi-Tissue Histology Images
• Panoptic Segmentation
• Mask R-CNN
• Aggregated Residual Transformations for Deep Neural Networks
• ImageJ for Microscopy
• Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
• Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

Cited by

• Uni-Parser Technical Report

Related

• MatSAM: Efficient Extraction of Microstructures of Materials via Visual Large Model
• EMCNet : Graph-Nets for Electron Micrographs Classification
• Deep Generative Models-Assisted Automated Labeling for Electron Microscopy Images Segmentation
• Accelerating Domain-Aware Electron Microscopy Analysis Using Deep Learning Models with Synthetic Data and Image-Wide Confidence Scoring
• Hierarchical Network Fusion for Multi-Modal Electron Micrograph Representation Learning with Foundational Large Language Models