FUNSD: A Dataset for Form Understanding in Noisy Scanned Documents

2019/05/27 by Guillaume Jaume, Jaume, Guillaume, Hazim Kemal Ekenel +3 · 58 citations
Computer Science · Mathematics · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Information Retrieval (cs.IR) #Machine Learning (cs.LG) #Machine Learning (stat.ML) #cs.CV #cs.IR #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.1905.13538

ICDAR'19 OST workshop

arxiv created 2019/10/29 · arxiv updated 2019/10/30

Abstract

We present a new dataset for form understanding in noisy scanned documents (FUNSD) that aims at extracting and structuring the textual content of forms. The dataset comprises 199 real, fully annotated, scanned forms. The documents are noisy and vary widely in appearance, making form understanding (FoUn) a challenging task. The proposed dataset can be used for various tasks, including text detection, optical character recognition, spatial layout analysis, and entity labeling/linking. To the best of our knowledge, this is the first publicly available dataset with comprehensive annotations to address FoUn task. We also present a set of baselines and introduce metrics to evaluate performance on the FUNSD dataset, which can be downloaded at https://guillaumejaume.github.io/FUNSD/.

Cited by

• StepX-Edge: An On-Device UI Vision-Language Model via Architecture-Training-Deployment Co-Design
• DocAnnot -- Accelerating the Creation of Key Information Extraction Datasets with GenAI-Powered Auto-annotation
• Accelerating End-to-End PDF to Markdown Conversion Through Assisted Generation
• Layout-Aware Text Editing for Efficient Transformation of Academic PDFs to Markdown
• KH-FUNSD: A Hierarchical and Fine-Grained Layout Analysis Dataset for Low-Resource Khmer Business Document
• DocVAL: Validated Chain-of-Thought Distillation for Grounded Document VQA
• LocateAnything3D: Vision-Language 3D Detection with Chain-of-Sight
• ARIAL: An Agentic Framework for Document VQA with Precise Answer Localization
• MGA-VQA: Secure and Interpretable Graph-Augmented Visual Question Answering with Memory-Guided Protection Against Unauthorized Knowledge Use
• OutSafe-Bench: A Benchmark for Multimodal Offensive Content Detection in Large Language Models
• Towards Resource-Efficient Multimodal Intelligence: Learned Routing among Specialized Expert Models
• IndicVisionBench: Benchmarking Cultural and Multilingual Understanding in VLMs
• ThaiOCRBench: A Task-Diverse Benchmark for Vision-Language Understanding in Thai
• Seeing Straight: Document Orientation Detection for Efficient OCR
• NVIDIA Nemotron Nano V2 VL
• SlideAgent: Hierarchical Agentic Framework for Multi-Page Visual Document Understanding
• FineVision: Open Data Is All You Need
• Document Intelligence in the Era of Large Language Models: A Survey
• FlexDoc: Parameterized Sampling for Diverse Multilingual Synthetic Documents for Training Document Understanding Models
• Hybrid OCR-LLM Framework for Enterprise-Scale Document Information Extraction Under Copy-heavy Task
• Exploring OCR-augmented Generation for Bilingual VQA
• LLM/Agent-as-Data-Analyst: A Survey
• SynDoc: A Hybrid Discriminative-Generative Framework for Enhancing Synthetic Domain-Adaptive Document Key Information Extraction
• Reading the unreadable: creating a dataset of 19th century English newspapers using image-to-text language models
• CommonForms: A Large, Diverse Dataset for Form Field Detection
• OTCR: Optimal Transmission, Compression and Representation for Multimodal Information Extraction
• MindVL: Towards Efficient and Effective Training of Multimodal Large Language Models on Ascend NPUs
• AgenticIE: An Adaptive Agent for Information Extraction from Complex Regulatory Documents
• Benchmarking Vision-Language Models on Chinese Ancient Documents: From OCR to Knowledge Reasoning
• R²-CoD: Understanding Text-Graph Complementarity in Relational Reasoning via Knowledge Co-Distillation
• Zero-shot OCR Accuracy of Low-Resourced Languages: A Comparative Analysis on Sinhala and Tamil
• Mono-InternVL-1.5: Towards Cheaper and Faster Monolithic Multimodal Large Language Models
• A Survey on MLLM-based Visually Rich Document Understanding: Methods, Challenges, and Emerging Trends
• DocPolarBERT: A Pre-trained Model for Document Understanding with Relative Polar Coordinate Encoding of Layout Structures
• Spatial ModernBERT: Spatial-Aware Transformer for Table and Key-Value Extraction in Financial Documents at Scale
• DREAM: Document Reconstruction via End-to-end Autoregressive Model
• Seeing is Believing? Mitigating OCR Hallucinations in Multimodal Large Language Models
• Robustness Evaluation of OCR-based Visual Document Understanding under Multi-Modal Adversarial Attacks
• MultiFinBen: Benchmarking Large Language Models for Multilingual and Multimodal Financial Application
• Efficient Medical VIE via Reinforcement Learning
• The OCR Quest for Generalization: Learning to recognize low-resource alphabets with model editing
• Open CaptchaWorld: A Comprehensive Web-based Platform for Testing and Benchmarking Multimodal LLM Agents
• Diagnosing and Mitigating Modality Interference in Multimodal Large Language Models
• FUDOKI: Discrete Flow-based Unified Understanding and Generation via Kinetic-Optimal Velocities
• Doc-CoB: Enhancing Document Understanding with Visual Chain-of-Boxes Reasoning
• Enhancing Large Vision-Language Models with Layout Modality for Table Question Answering on Japanese Annual Securities Reports
• FS-DAG: Few Shot Domain Adapting Graph Networks for Visually Rich Document Understanding
• Prolonged Reasoning Is Not All You Need: Certainty-Based Adaptive Routing for Efficient LLM/MLLM Reasoning
• Visual Agentic Reinforcement Fine-Tuning
• Information Extraction from Visually Rich Documents using LLM-based Organization of Documents into Independent Textual Segments
• Adaptive Markup Language Generation for Contextually-Grounded Visual Document Understanding
• LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decoding
• A Multimodal Pipeline for Clinical Data Extraction: Applying Vision-Language Models to Scans of Transfusion Reaction Reports
• AdaParse: An Adaptive Parallel PDF Parsing and Resource Scaling Engine
• DocQT: Improving Document Forgery Localization Robustness via Diverse JPEG Quantization Tables
• Relation-Rich Visual Document Generator for Visual Information Extraction
• Capybara-OMNI: An Efficient Paradigm for Building Omni-Modal Language Models
• A comparative study on textual and multimodal PII detection using large language models

Related

• Transformers and Language Models in Form Understanding: A Comprehensive Review of Scanned Document Analysis
• CommonForms: A Large, Diverse Dataset for Form Field Detection
• Data-Efficient Information Extraction from Form-Like Documents
• Form-NLU: Dataset for the Form Natural Language Understanding
• DocStruct: A Multimodal Method to Extract Hierarchy Structure in Document for General Form Understanding