DIMT25@ICDAR2025: HW-TSC's End-to-End Document Image Machine Translation System Leveraging Large Vision-Language Model

2025/04/24 by Zhanglin Wu, Wu, Zhanglin, Tengfei Song +15
Computer Science · #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Natural Language Processing Techniques

paper · pdf · doi:10.48550/arxiv.2504.17315

openalex publication_date 2025/04/24 · openalex created_date 2025/10/18 · openalex updated_date 2026/07/28

Abstract

This paper presents the technical solution proposed by Huawei Translation Service Center (HW-TSC) for the "End-to-End Document Image Machine Translation for Complex Layouts" competition at the 19th International Conference on Document Analysis and Recognition (DIMT25@ICDAR2025). Leveraging state-of-the-art open-source large vision-language model (LVLM), we introduce a training framework that combines multi-task learning with perceptual chain-of-thought to develop a comprehensive end-to-end document translation system. During the inference phase, we apply minimum Bayesian decoding and post-processing strategies to further enhance the system's translation capabilities. Our solution uniquely addresses both OCR-based and OCR-free document image translation tasks within a unified framework. This paper systematically details the training methods, inference strategies, LVLM base models, training data, experimental setups, and results, demonstrating an effective approach to document image machine translation.

Citations

• Evaluating Menu OCR and Translation: A Benchmark for Aligning Human and Automated Evaluations in Large Vision-Language Models
• Qwen2.5-VL Technical Report
• Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling
• Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization
• Choose the Final Translation from NMT and LLM hypotheses Using MBR Decoding: HW-TSC's Submission to the WMT24 General MT Shared Task
• HW-TSC's Submission to the CCMT 2024 Machine Translation Tasks
• An Empirical Study of Translation Hypothesis Ensembling with Large Language Models
• How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Composition
• Continual Pre-Training of Large Language Models: How to (re)warm your model?
• A Survey on Evaluation of Large Language Models
• A Survey on Evaluation of Large Language Models
• ChatGPT for good? On opportunities and challenges of large language models for education
• Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
• BNAI, NO-TOKEN, and MIND-UNITY: Pillars of a Systemic Revolution in Artificial Intelligence
• Perspectives and Prospects on Transformer Architecture for Cross-Modal Tasks with Language and Vision
• COMET: A Neural Framework for MT Evaluation
• A Survey on Multi-Task Learning
• A Survey on Multi-Task Learning

Related

• Improving MLLM's Document Image Machine Translation via Synchronously Self-reviewing Its OCR Proficiency
• PATIMT-Bench: A Multi-Scenario Benchmark for Position-Aware Text Image Machine Translation in Large Vision-Language Models
• Single-to-mix Modality Alignment with Multimodal Large Language Model for Document Image Machine Translation
• HunyuanOCR Technical Report
• MIT-10M: A Large Scale Parallel Corpus of Multilingual Image Translation