Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
2016/09/26 by Yonghui Wu, Wu, Yonghui, Mike Schuster +62 · 2 voices · 522 citations
Computer Science · #Natural Language Processing Techniques #Topic Modeling #Multimodal Machine Learning Applications
paper · pdf · doi:10.48550/arxiv.1609.08144
Abstract
Neural Machine Translation (NMT) is an end-to-end learning approach for automated translation, with the potential to overcome many of the weaknesses of conventional phrase-based translation systems. Unfortunately, NMT systems are known to be computationally expensive both in training and in translation inference. Also, most NMT systems have difficulty with rare words. These issues have hindered NMT's use in practical deployments and services, where both accuracy and speed are essential. In this work, we present GNMT, Google's Neural Machine Translation system, which attempts to address many of these issues. Our model consists of a deep LSTM network with 8 encoder and 8 decoder layers using attention and residual connections. To improve parallelism and therefore decrease training time, our attention mechanism connects the bottom layer of the decoder to the top layer of the encoder. To accelerate the final translation speed, we employ low-precision arithmetic during inference computations. To improve handling of rare words, we divide words into a limited set of common sub-word units ("wordpieces") for both input and output. This method provides a good balance between the flexibility of "character"-delimited models and the efficiency of "word"-delimited models, naturally handles translation of rare words, and ultimately improves the overall accuracy of the system. Our beam search technique employs a length-normalization procedure and uses a coverage penalty, which encourages generation of an output sentence that is most likely to cover all the words in the source sentence. On the WMT'14 English-to-French and English-to-German benchmarks, GNMT achieves competitive results to state-of-the-art. Using a human side-by-side evaluation on a set of isolated simple sentences, it reduces translation errors by an average of 60% compared to Google's phrase-based production system.
Cited by
- How Fast Can Reward Models Score? A Systems Study of C++ and PyTorch Inference Runtimes for RLHF
- ChineseBERT: Chinese Pretraining Enhanced by Glyph and Pinyin Information
- A Deep Reinforcement Learning Algorithm for the Vehicle Routing Problem with Stochastic Demands and Outsourcing
- Artificial Intelligence for Mathematical Reasoning: An Integrated Survey of Language Models, Neuro-symbolic Systems, and Verified Discovery
- TD-DPO: Difference-Aware Preference Optimization for Mitigating Sycophancy in Clinical Autism Intervention Dialogue
- Tokenization with Split Trees
- Stochasticity in Tokenisation Improves Robustness
- Leveraging Natural Language Processing to Unravel the Mystery of Life: A Review of NLP Approaches in Genomics, Transcriptomics, and Proteomics
- Are Statistical Methods Obsolete in the Era of Deep Learning? A Study of ODE Inverse Problems
- Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMs
- Assessing the Impact of Typological Features on Multilingual Machine Translation in the Age of Large Language Models
- Joint Optimization for Greedy Longest-match Tokenization
- BHARATI: Morphology-Aware Tokenizers for Classical Indian Languages with Subword Fertility Analysis
- Look Closer! An Adversarial Parametric Editing Framework for Hallucination Mitigation in VLMs
- TokSuite: Measuring the Impact of Tokenizer Choice on Language Model Behavior
- ReasonCD: A Multimodal Reasoning Large Model for Implicit Change-of-Interest Semantic Mining
- Building Patient Journeys in Hebrew: A Language Model for Clinical Timeline Extraction
- Tacotron: Towards End-to-End Speech Synthesis
- Efficient Continual Learning in Neural Machine Translation: A Low-Rank Adaptation Approach
- Factorization tricks for LSTM networks
- Atlas: A Dataset and Benchmark for E-commerce Clothing Product\n Categorization
- Learning to Generate Reviews and Discovering Sentiment
- Improving Post Training Neural Quantization: Layer-wise Calibration and Integer Programming
- Quasi-Recurrent Neural Networks
- ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT
- An Empirical Investigation of Multi-bridge Multilingual NMT models
- SimCLS: A Simple Framework for Contrastive Learning of Abstractive Summarization
- A Hybrid Word-Character Approach to Abstractive Summarization
- Hard-Coded Gaussian Attention for Neural Machine Translation
- Explaining Documents' Relevance to Search Queries
- Advancing Acoustic-to-Word CTC Model with Attention and Mixed-Units
- CARL: Criticality-Aware Agentic Reinforcement Learning
- On the Binding Problem in Artificial Neural Networks
- Representation Degeneration Problem in Training Natural Language Generation Models
- The brain-AI convergence: Predictive and generative world models for general-purpose computation
- Learning Energy-Based Approximate Inference Networks for Structured Applications in NLP
- Empowering Things with Intelligence: A Survey of the Progress, Challenges, and Opportunities in Artificial Intelligence of Things
- UPB at SemEval-2020 Task 12: Multilingual Offensive Language Detection on Social Media by Fine-tuning a Variety of BERT-based Models
- SGD-QA: Fast Schema-Guided Dialogue State Tracking for Unseen Services
- Layer-Wise Multi-View Learning for Neural Machine Translation
- Pretrained Transformers for Simple Question Answering over Knowledge Graphs
- Deep Learning for Environmentally Robust Speech Recognition: An Overview of Recent Developments
- What's in a Name? -- Gender Classification of Names with Character Based Machine Learning Models
- SDM: Sequential Deep Matching Model for Online Large-scale Recommender System
- Semantic Explanations of Predictions
- Implementing Neural Turing Machines
- An End-to-End Approach to Automatic Speech Assessment for Cantonese-speaking People with Aphasia
- MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers
- Syntax-Enhanced Neural Machine Translation with Syntax-Aware Word Representations
- Mono vs Multilingual Transformer-based Models: a Comparison across Several Language Tasks
- VALUE: A Multi-Task Benchmark for Video-and-Language Understanding Evaluation
- Bayesian Recurrent Neural Networks
- Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERT
- MKD: a Multi-Task Knowledge Distillation Approach for Pretrained Language Models
- Charformer: Fast Character Transformers via Gradient-based Subword Tokenization
- SQWA: Stochastic Quantized Weight Averaging for Improving the Generalization Capability of Low-Precision Deep Neural Networks
- CTRL: A Conditional Transformer Language Model for Controllable Generation
- A Deep Reinforcement Learning Chatbot
- Compositional Generalization for Primitive Substitutions
- Sub-word Level Lip Reading With Visual Attention
- Kaleido-BERT: Vision-Language Pre-training on Fashion Domain
- Translation, Sentiment and Voices: A Computational Model to Translate\n and Analyse Voices from Real-Time Video Calling
- Fine-Tuning Language Models from Human Preferences
- Creative GANs for generating poems, lyrics, and metaphors
- Enhancing Clinical Information Extraction with Transferred Contextual Embeddings
- A Lightweight Recurrent Network for Sequence Modeling
- StrucTexT: Structured Text Understanding with Multi-Modal Transformers
- Exploring question answering: metric analysis and evaluation framework for enhanced interpretability
- Students Need More Attention: BERT-based AttentionModel for Small Data with Application to AutomaticPatient Message Triage
- Hippo: Taming Hyper-parameter Optimization of Deep Learning with Stage Trees
- Modeling Past and Future for Neural Machine Translation
- On the importance of single directions for generalization
- ERNIE: Enhanced Representation through Knowledge Integration
- Neural machine translation in EFL classrooms: learners’ vocabulary improvement, immediate vocabulary retention and delayed vocabulary retention
- Document Graph for Neural Machine Translation
- PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization
- Does injecting linguistic structure into language models lead to better alignment with brain recordings?
- Neural Machine Translation and Sequence-to-sequence Models: A Tutorial
- Evaluating Subword Tokenization Techniques for Bengali: A Benchmark Study with BengaliBPE
- Segmentation Beyond Defaults: Asymmetrical Byte Pair Encoding for Optimal Machine Translation Performance
- BIG MOOD: Relating Transformers to Explicit Commonsense Knowledge
- Adversarial Attacks and Defense on Texts: A Survey
- DTS: Enhancing Large Reasoning Models via Decoding Tree Sketching
- Multilingual BERT language model for medical tasks: Evaluation on domain-specific adaptation and cross-linguality
- Modular Linear Tokenization (MLT)
- Unsupervised Machine Translation Using Monolingual Corpora Only
- Constrained Language Models Yield Few-Shot Semantic Parsers
- Prediction with a Short Memory
- Simple, Scalable Adaptation for Neural Machine Translation
- Structure-Invariant Testing for Machine Translation
- A Comprehensive Exploration on WikiSQL with Table-Aware Word\n Contextualization
- AutoML-Zero: Evolving Machine Learning Algorithms From Scratch
- Cross-Lingual Semantic Role Labeling with High-Quality Translated Training Corpus
- Language Through a Prism: A Spectral Approach for Multiscale Language Representations
- Vulnerability Under Adversarial Machine Learning: Bias or Variance?
- ReClor: A Reading Comprehension Dataset Requiring Logical Reasoning
- Portuguese Named Entity Recognition using BERT-CRF
- An Empirical Study of Mini-Batch Creation Strategies for Neural Machine Translation
- Efficient Attentions for Long Document Summarization
- VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Experts
- Attention-based sequence-to-sequence model for speech recognition: development of state-of-the-art system on LibriSpeech and its application to non-native English
- MARMOT: A Deep Learning Framework for Constructing Multimodal Representations for Vision-and-Language Tasks
- Mixed-Precision Training for NLP and Speech Recognition with OpenSeq2Seq
- Unicoder-VL: A Universal Encoder for Vision and Language by Cross-modal Pre-training
- Parallax: Sparsity-aware Data Parallel Training of Deep Neural Networks
- Interactive neural machine translation
- GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding
- Mixed Precision Training
- Semantic Parsing with Dual Learning
- Towards Neural Machine Translation with Partially Aligned Corpora
- Generative Language Modeling for Automated Theorem Proving
- A Deep Reinforced Model for Abstractive Summarization
- SAFER: A Structure-free Approach for Certified Robustness to Adversarial Word Substitutions
- FashionBERT: Text and Image Matching with Adaptive Loss for Cross-modal Retrieval
- Learning Algorithms for Active Learning
- Deep learning: new computational modelling techniques for genomics
- Stochastic Beams and Where to Find Them: The Gumbel-Top-k Trick for Sampling Sequences Without Replacement
- Selfie: Self-supervised Pretraining for Image Embedding
- Testing Machine Translation via Referential Transparency
- PairConnect: A Compute-Efficient MLP Alternative to Attention
- Interpretable Self-supervised Multi-task Learning for COVID-19 Information Retrieval and Extraction
- Later-stage Minimum Bayes-Risk Decoding for Neural Machine Translation
- Multimodal learning enables chat-based exploration of single-cell data
- Exploring Sparsity in Recurrent Neural Networks
- AdvPicker: Effectively Leveraging Unlabeled Data via Adversarial Discriminator for Cross-Lingual NER
- Accelerating Sparse Deep Neural Networks
- Learning Deep Transformer Models for Machine Translation
- Two-Timescale Voltage Control in Distribution Grids Using Deep Reinforcement Learning
- Resurrecting Submodularity for Neural Text Generation
- Bridging the Gap between Spatial and Spectral Domains: A Survey on Graph Neural Networks
- OpenNMT: Neural Machine Translation Toolkit
- An Evaluation Dataset and Strategy for Building Robust Multi-turn Response Selection Model
- A Study of Multilingual Neural Machine Translation
- LAMPRET: Layout-Aware Multimodal PreTraining for Document Understanding
- Recurrent Graph Syntax Encoder for Neural Machine Translation
- Denoising based Sequence-to-Sequence Pre-training for Text Generation
- Shallow-to-Deep Training for Neural Machine Translation
- Human-centric Metric for Accelerating Pathology Reports Annotation
- On the Significance of Question Encoder Sequence Model in the\n Out-of-Distribution Performance in Visual Question Answering
- ActBERT: Learning Global-Local Video-Text Representations
- What the [MASK]? Making Sense of Language-Specific BERT Models
- DeepZensols: Deep Natural Language Processing Framework
- Information Aggregation for Multi-Head Attention with Routing-by-Agreement
- Robust Neural Machine Translation with Doubly Adversarial Inputs
- Deep geometric knowledge distillation with graphs
- Convolutional Attention-based Seq2Seq Neural Network for End-to-End ASR
- Deep Hierarchical Classification for Category Prediction in E-commerce System
- Bayesian Attention Modules
- Slip Detection: Analysis and Calibration of Univariate Tactile Signals
- A Comprehensive Survey on Hardware-Aware Neural Architecture Search
- ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks
- An Unsupervised Autoregressive Model for Speech Representation Learning
- Pre-trained Language Model for Biomedical Question Answering
- StarCraft II: A New Challenge for Reinforcement Learning
- UIBert: Learning Generic Multimodal Representations for UI Understanding
- ST-BERT: Cross-modal Language Model Pre-training For End-to-end Spoken Language Understanding
- Efficiently Reusing Old Models Across Languages via Transfer Learning
- Self-Supervised Text-Vision Alignment for Automated Brain MRI Abnormality Detection: A Multicenter Study (ALIGN Study)
- A Computational Model of Commonsense Moral Decision Making
- Multi-Head Attention with Disagreement Regularization
- VL-BERT: Pre-training of Generic Visual-Linguistic Representations
- Generating Wikipedia by Summarizing Long Sequences
- PiSLTRc: Position-informed Sign Language Transformer with Content-aware Convolution
- Can LLMs Translate Human Instructions into a Reinforcement Learning Agent's Internal Emergent Symbolic Representation?
- Daydream: Accurately Estimating the Efficacy of Optimizations for DNN Training
- SEAL: Segment-wise Extractive-Abstractive Long-form Text Summarization
- CNN Is All You Need
- Full-Sentence Models Perform Better in Simultaneous Translation Using the Information Enhanced Decoding Strategy
- Bridging Neural Machine Translation and Bilingual Dictionaries
- Log-based Anomaly Detection Without Log Parsing
- A Survey of Domain Adaptation for Neural Machine Translation
- Pctx: Tokenizing Personalized Context for Generative Recommendation
- Stand-Alone Self-Attention in Vision Models
- A spelling correction model for end-to-end speech recognition
- Forging GEMs: Advancing Greek NLP through Quality-Based Corpus Curation
- SODBench: A Large Language Model Approach to Documenting Spreadsheet Operations
- Psycholinguistic Tripartite Graph Network for Personality Detection
- Challenges in Data-to-Document Generation
- Denoising Neural Machine Translation Training with Trusted Data and Online Data Selection
- Themis: Fair and Efficient GPU Cluster Scheduling
- Transformer-Based Low-Resource Language Translation: A Study on Standard Bengali to Sylheti
- One Big Net For Everything
- ImageBERT: Cross-modal Pre-training with Large-scale Weak-supervised Image-Text Data
- Stochastic Gradient/Mirror Descent: Minimax Optimality and Implicit Regularization
- Automatic Classification of Human Translation and Machine Translation: A Study from the Perspective of Lexical Diversity
- Device Placement Optimization with Reinforcement Learning
- Single-Queue Decoding for Neural Machine Translation
- Enhancing Pre-trained Chinese Character Representation with Word-aligned Attention
- Zero-Shot Dual Machine Translation
- Unsupervised State Representation Learning in Atari
- iCapsNets: Towards Interpretable Capsule Networks for Text Classification
- AdvAug: Robust Adversarial Augmentation for Neural Machine Translation
- The University of Sydney's Machine Translation System for WMT19
- Towards General Purpose Vision Systems
- Give your Text Representation Models some Love: the Case for Basque
- Interactive-Predictive Neural Machine Translation through Reinforcement and Imitation
- Learning a Multi-Domain Curriculum for Neural Machine Translation
- Interpretable Entity Representations through Large-Scale Typing
- CommandSans: Securing AI Agents with Surgical Precision Prompt Sanitization
- Annotating publicly-available samples and studies using interpretable modeling of unstructured metadata
- Less is More: ClipBERT for Video-and-Language Learning via Sparse Sampling
- VS-Quant: Per-vector Scaled Quantization for Accurate Low-Precision Neural Network Inference
- A Low-Resource Speech-Driven NLP Pipeline for Sinhala Dyslexia Assistance
- Cold-Start Reinforcement Learning with Softmax Policy Gradient
- Increasing LLM response trustworthiness using voting ensembles
- Translating Phrases in Neural Machine Translation
- Evaluating Embedding Frameworks for Scientific Domain
- Improving Sign Language Translation with Monolingual Data by Sign Back-Translation
- VisualBERT: A Simple and Performant Baseline for Vision and Language
- Efficient Neural Audio Synthesis
- Rearchitecting Datacenter Lifecycle for AI: A TCO-Driven Framework
- Better with Less: Small Proprietary Models Surpass Large Language Models in Financial Transaction Understanding
- Efficient Sketching and Nearest Neighbor Search Algorithms for Sparse Vector Sets
- Enabling Physical AI through Biological Principles
- How Do Source-side Monolingual Word Embeddings Impact Neural Machine Translation?
- MLPerf Training Benchmark
- Exploring the limits of Concurrency in ML Training on Google TPUs
- Attend and Diagnose: Clinical Time Series Analysis using Attention Models
- Machine Reading Comprehension: a Literature Review
- Deep Multi-Kernel Convolutional LSTM Networks and an Attention-Based\n Mechanism for Videos
- Cross-Modality Relevance for Reasoning on Language and Vision
- Combinatorial Testing for Deep Learning Systems
- Multiscale Collaborative Deep Models for Neural Machine Translation
- Understanding Reuse, Performance, and Hardware Cost of DNN Dataflows: A Data-Centric Approach Using MAESTRO
- AI Brown and AI Koditex: LLM-Generated Corpora Comparable to Traditional Corpora of English and Czech Texts
- LXMERT: Learning Cross-Modality Encoder Representations from Transformers
- Hybrid Retrieval-Generation Reinforced Agent for Medical Image Report Generation
- Learning by Abstraction: The Neural State Machine
- Learning to Detect Unacceptable Machine Translations for Downstream Tasks
- Zero-shot Knowledge Transfer via Adversarial Belief Matching
- Generation of focused drug molecule library using recurrent neural network
- Biomedical Entity Representations with Synonym Marginalization
- CodeBERT: A Pre-Trained Model for Programming and Natural Languages
- Impact of AI assistance on student agency
- Literature review on vulnerability detection using NLP technology
- A First Look at Mobile Intelligence: Architecture, Experimentation and Challenges
- All You Can Embed: Natural Language based Vehicle Retrieval with\n Spatio-Temporal Transformers
- TRANS-BLSTM: Transformer with Bidirectional LSTM for Language Understanding
- Local Contextual Attention with Hierarchical Structure for Dialogue Act Recognition
- Integer Quantization for Deep Learning Inference: Principles and Empirical Evaluation
- Controllable Text Simplification with Explicit Paraphrasing
- Hybrid Data-Model Parallel Training for Sequence-to-Sequence Recurrent Neural Network Machine Translation
- An Exploration of Word Embedding Initialization in Deep-Learning Tasks
- Dynamic Curriculum Learning for Low-Resource Neural Machine Translation
- Multimodal Learning for Hateful Memes Detection
- Parallel Blockwise Knowledge Distillation for Deep Neural Network Compression
- KVL-BERT: Knowledge Enhanced Visual-and-Linguistic BERT for Visual Commonsense Reasoning
- Semi-Autoregressive Neural Machine Translation
- Speculating LLMs' Chinese Training Data Pollution from Their Tokens
- Generating Sentences by Editing Prototypes
- Analysis of Predictive Coding Models for Phonemic Representation Learning in Small Datasets
- M4SER: Multimodal, Multirepresentation, Multitask, and Multistrategy Learning for Speech Emotion Recognition
- An overview of neural architectures for self-supervised audio representation learning from masked spectrograms
- Multilingual Neural Machine Translation for Zero-Resource Languages
- Democratizing Production-Scale Distributed Deep Learning
- Gram-CTC: Automatic Unit Selection and Target Decomposition for Sequence Labelling
- Aligned Image-Word Representations Improve Inductive Transfer Across\n Vision-Language Tasks
- Reversible Architectures for Arbitrarily Deep Residual Neural Networks
- Bit-Flip Attack: Crushing Neural Network with Progressive Bit Search
- KuBERT: Central Kurdish BERT Model and Its Application for Sentiment Analysis
- Sentiment-based Candidate Selection for NMT
- Dynamic Programming Encoding for Subword Segmentation in Neural Machine Translation
- Model Compression and Hardware Acceleration for Neural Networks: A Comprehensive Survey
- A Comparative Study of Transformer-Based Language Models on Extractive Question Answering
- Recent Trends in the Use of Deep Learning Models for Grammar Error Handling
- Enabling Multi-Source Neural Machine Translation By Concatenating Source\n Sentences In Multiple Languages
- Attention Forcing for Sequence-to-sequence Model Training
- Black-box Generation of Adversarial Text Sequences to Evade Deep Learning Classifiers
- Regularizing Deep Neural Networks by Noise: Its Interpretation and Optimization
- A Random Matrix Perspective on Mixtures of Nonlinearities for Deep Learning
- MOOCRep: A Unified Pre-trained Embedding of MOOC Entities
- Discriminative Adversarial Search for Abstractive Summarization
- Neural Text Generation with Artificial Negative Examples
- Efficient Algorithms for Device Placement of DNN Graph Operators
- XPipe: Efficient Pipeline Model Parallelism for Multi-GPU DNN Training
- Open-endedness in AI systems, cellular evolution and intellectual discussions
- InterBERT: Vision-and-Language Interaction for Multi-modal Pretraining
- CAGFuzz: Coverage-Guided Adversarial Generative Fuzzing Testing of Deep Learning Systems
- Topic Modeling with Contextualized Word Representation Clusters
- Automatic Cross-Replica Sharding of Weight Update in Data-Parallel Training
- Dynamics of Deep Neural Networks and Neural Tangent Hierarchy
- Massively Multilingual Neural Machine Translation
- HufuNet: Embedding the Left Piece as Watermark and Keeping the Right Piece for Ownership Verification in Deep Neural Networks
- Experiments with Rich Regime Training for Deep Learning
- Data Augmentation Generative Adversarial Networks
- The Natural Language Decathlon: Multitask Learning as Question Answering
- Graph-based Filtering of Out-of-Vocabulary Words for Encoder-Decoder Models
- A Study of Reinforcement Learning for Neural Machine Translation
- Neural Network Distiller: A Python Package For DNN Compression Research
- Synergy: Resource Sensitive DNN Scheduling in Multi-Tenant Clusters
- LfEdNet: A Task-based Day-ahead Load Forecasting Model for Stochastic Economic Dispatch
- Corpora Generation for Grammatical Error Correction
- Hunyuan-MT Technical Report
- Globally Optimal Gradient Descent for a ConvNet with Gaussian Inputs
- Multi-modal Transformer for Video Retrieval
- HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training
- Sharp Minima Can Generalize For Deep Nets
- Neural Machine Translation with Explicit Phrase Alignment
- A Runtime-Based Computational Performance Predictor for Deep Neural Network Training
- chDzDT: Word-level morphology-aware language model for Algerian social media text
- Noisy Self-Knowledge Distillation for Text Summarization
- Understanding and Enhancing the Use of Context for Machine Translation
- Modeling Fine-Grained Entity Types with Box Embeddings
- Neural network gradient-based learning of black-box function interfaces
- Independently Recurrent Neural Network (IndRNN): Building A Longer and Deeper RNN
- A Correspondence Between Random Neural Networks and Statistical Field\n Theory
- Seeing Out of tHe bOx: End-to-End Pre-training for Vision-Language Representation Learning
- Learning to Remember Rare Events
- Approximating Continuous Functions by ReLU Nets of Minimal Width
- When Does Pretraining Help? Assessing Self-Supervised Learning for Law and the CaseHOLD Dataset
- PAIR: Planning and Iterative Refinement in Pre-trained Transformers for Long Text Generation
- Axiomatic Attribution for Deep Networks
- Enhancing Robustness of Autoregressive Language Models against Orthographic Attacks via Pixel-based Approach
- FlowletFormer: Network Behavioral Semantic Aware Pre-training Model for Traffic Classification
- Sequence Model with Self-Adaptive Sliding Window for Efficient Spoken Document Segmentation
- An Optimized and Energy-Efficient Parallel Implementation of Non-Iteratively Trained Recurrent Neural Networks
- SemToken: Semantic-Aware Tokenization for Efficient Long-Context Language Modeling
- Unsupervised Learning Layers for Video Analysis
- Counterspeech for Mitigating the Influence of Media Bias: Comparing Human and LLM-Generated Responses
- In2x at WMT25 Translation Task
- Learning to Coordinate Multiple Reinforcement Learning Agents for Diverse Query Reformulation
- BERT might be Overkill: A Tiny but Effective Biomedical Entity Linker based on Residual Convolutional Neural Networks
- PAWS: Paraphrase Adversaries from Word Scrambling
- Language as a matrix product state
- DeLS-3D: Deep Localization and Segmentation with a 3D Semantic Map
- An Empirical Comparison of Syllabuses for Curriculum Learning
- A Study of BFLOAT16 for Deep Learning Training
- Generating Titles for Web Tables
- Deep Neural Network for Semantic-based Text Recognition in Images
- When Alignment Hurts: Decoupling Representational Spaces in Multilingual Models
- Wavy Transformer
- Modeling Diagnostic Label Correlation for Automatic ICD Coding
- Score-informed Neural Operator for Enhancing Ordering-based Causal Discovery
- Learning to Generate Multiple Style Transfer Outputs for an Input Sentence
- Language Models with Transformers
- Scalable Transformers for Neural Machine Translation
- SupraTok: Cross-Boundary Tokenization for Enhanced Language Model Performance
- Model Interpretability and Rationale Extraction by Input Mask Optimization
- Effective Use of Graph Convolution Network and Contextual Sub-Tree\n forCommodity News Event Extraction
- Adma: A Flexible Loss Function for Neural Networks
- Optimizing Sponsored Search Ranking Strategy by Deep Reinforcement Learning
- DTMT: A Novel Deep Transition Architecture for Neural Machine Translation
- Rethinking Dialogue State Tracking with Reasoning
- Shatter: An Efficient Transformer Encoder with Single-Headed Self-Attention and Relative Sequence Partitioning
- AI-Powered Social Bots
- Generating Representative Headlines for News Stories
- ModiPick: SLA-aware Accuracy Optimization For Mobile Deep Inference
- A Comparison of Approaches to Document-level Machine Translation
- Anomaly Detection for IoT Global Connectivity
- Phantom: A High-Performance Computational Core for Sparse Convolutional Neural Networks
- Improving Back-Translation with Uncertainty-based Confidence Estimation
- SignBERT: Pre-Training of Hand-Model-Aware Representation for Sign Language Recognition
- How LSTM Encodes Syntax: Exploring Context Vectors and Semi-Quantization on Natural Text
- Marvel: A Data-centric Compiler for DNN Operators on Spatial Accelerators
- SE3M: A model for software effort estimation using pre-trained embedding models
- A Survey on Non-Intrusive ASR Refinement: From Output-Level Correction to Full-Model Distillation
- MeanSum: A Neural Model for Unsupervised Multi-document Abstractive Summarization
- Recurrent multiple shared layers in Depth for Neural Machine Translation
- MIRA: Leveraging Multi-Intention Co-click Information in Web-scale Document Retrieval using Deep Neural Networks
- Being-ahead: Benchmarking and Exploring Accelerators for Hardware-Efficient AI Deployment
- TAG : Type Auxiliary Guiding for Code Comment Generation
- Modelling and Classifying the Components of a Literature Review
- Semantic Sentence Matching with Densely-connected Recurrent and Co-attentive Information
- Text Summarization with Pretrained Encoders
- NTT's Machine Translation Systems for WMT19 Robustness Task
- BERT-DST: Scalable End-to-End Dialogue State Tracking with Bidirectional Encoder Representations from Transformer
- An Actor-Critic Algorithm for Sequence Prediction
- PolyDL: Polyhedral Optimizations for Creation of High Performance DL primitives
- ChEmbed: Enhancing Chemical Literature Search Through Domain-Specific Text Embeddings
- AR: Auto-Repair the Synthetic Data for Neural Machine Translation
- Bridging Text and Video: A Universal Multimodal Transformer for Video-Audio Scene-Aware Dialog
- FeatureCuts: Feature Selection for Large Data by Optimizing the Cutoff
- CancerBERT: a BERT model for Extracting Breast Cancer Phenotypes from Electronic Health Records
- Neural Machine Translating from Natural Language to SPARQL
- Large-Scale Evolution of Image Classifiers
- Learning Efficient Lexically-Constrained Neural Machine Translation with External Memory
- Explaining Natural Language Processing Classifiers with Occlusion and Language Modeling
- Language Arithmetics: Towards Systematic Language Neuron Identification and Manipulation
- Dual-mode ASR: Unify and Improve Streaming ASR with Full-context Modeling
- Collaborative Training of GANs in Continuous and Discrete Spaces for Text Generation
- Phenotyping of Clinical Notes with Improved Document Classification Models Using Contextualized Neural Language Models
- Neural CRF Model for Sentence Alignment in Text Simplification
- Length Representations in Large Language Models
- "The Boating Store Had Its Best Sail Ever": Pronunciation-attentive Contextualized Pun Recognition
- BERTScore: Evaluating Text Generation with BERT
- Machine Learning With Neuromorphic Photonics
- Improving Neural Machine Translation by Bidirectional Training
- Tensorized Embedding Layers for Efficient Model Compression
- Make (Nearly) Every Neural Network Better: Generating Neural Network Ensembles by Weight Parameter Resampling
- Chunk-Based Bi-Scale Decoder for Neural Machine Translation
- How Can BERT Help Lexical Semantics Tasks?
- Robust, Deep, and Reinforcement Learning for Management of Communication\n and Power Networks
- Tree Transformer: Integrating Tree Structures into Self-Attention
- Automatic Dialogic Instruction Detection for K-12 Online One-on-one Classes
- Towards Transfer Learning for End-to-End Speech Synthesis from Deep Pre-Trained Language Models
- Understanding Natural Language Instructions for Fetching Daily Objects Using GAN-Based Multimodal Target-Source Classification
- Enhancing Hindi NER in Low Context: A Comparative study of Transformer-based models with vs. without Retrieval Augmentation
- On the Blindspots of Convolutional Networks
- Towards Safer Smart Contracts: A Sequence Learning Approach to Detecting Security Threats
- Cseq2seq: Cyclic Sequence-to-Sequence Learning
- Exploring Hyper-Parameter Optimization for Neural Machine Translation on GPU Architectures
- On Accurate Evaluation of GANs for Language Generation
- Jointly Optimizing Diversity and Relevance in Neural Response Generation
- Taylor's law for Human Linguistic Sequences
- Connecting the Dots Between MLE and RL for Sequence Prediction
- Liputan6: A Large-scale Indonesian Dataset for Text Summarization
- Deep Recurrent Neural Network for Protein Function Prediction from Sequence
- STN4DST: A Scalable Dialogue State Tracking based on Slot Tagging Navigation
- Cross-Lingual BERT Transformation for Zero-Shot Dependency Parsing
- Deep Neural Machine Translation with Linear Associative Unit
- Data Augmentation for Low-Resource Named Entity Recognition Using Backtranslation
- Improving Pre-Trained Multilingual Models with Vocabulary Expansion
- Simplifying Neural Machine Translation with Addition-Subtraction Twin-Gated Recurrent Networks
- Machine Translation between Vietnamese and English: an Empirical Study
- Multilingual Constituency Parsing with Self-Attention and Pre-Training
- The Cross-Lingual Arabic Information REtrieval (CLAIRE) System
- Dialogue Summarization with Supporting Utterance Flow Modeling and Fact Regularization
- UzBERT: pretraining a BERT model for Uzbek
- Improved Speech Representations with Multi-Target Autoregressive Predictive Coding
- Multilingual AMR-to-Text Generation
- Teaching Temporal Logics to Neural Networks
- AdsGNN: Behavior-Graph Augmented Relevance Modeling in Sponsored Search
- Foundations of Vector Retrieval
- Neural Information Retrieval: A Literature Review
- Hierarchical Multitask Learning for CTC-based Speech Recognition
- BoostingBERT:Integrating Multi-Class Boosting into BERT for NLP Tasks
- Autocorrect in the Process of Translation -- Multi-task Learning Improves Dialogue Machine Translation
- Improving Readability for Automatic Speech Recognition Transcription
- They, Them, Theirs: Rewriting with Gender-Neutral English
- Recurrent Neural Network-based Model for Accelerated Trajectory Analysis\n in AIMD Simulations
- Incorporating BERT into Neural Machine Translation
- Multi-layer Representation Fusion for Neural Machine Translation
- Residual Continual Learning
- Uncertainty-Aware Semantic Augmentation for Neural Machine Translation
- Focus Group on Artificial Intelligence for Health
- BERT Meets Chinese Word Segmentation
- Reference Language based Unsupervised Neural Machine Translation
- ATOM: Commit Message Generation Based on Abstract Syntax Tree and Hybrid Ranking
- EPNAS: Efficient Progressive Neural Architecture Search
- Unified Language Model Pre-training for Natural Language Understanding\n and Generation
- Impact of Code Context and Prompting Strategies on Automated Unit Test Generation with Modern General-Purpose Large Language Models
- Exploiting Deep Representations for Neural Machine Translation
- Dual Inference for Improving Language Understanding and Generation
- Seed LiveInterpret 2.0: End-to-end Simultaneous Speech-to-speech Translation with Your Voice
- Artificial Quantum Neural Network: quantum neurons, logical elements and tests of convolutional nets
- Towards a resource for multilingual lexicons: an MT assisted and human-in-the-loop multilingual parallel corpus with multi-word expression annotation
- Boosting Active Learning for Speech Recognition with Noisy Pseudo-labeled Samples
- Seed-X: Building Strong Multilingual Translation LLM with 7B Parameters
- Step-Audio 2 Technical Report
- Fast Structured Decoding for Sequence Models
- Do Transformer Attention Heads Provide Transparency in Abstractive Summarization?
- Convolutional Sequence to Sequence Learning
- DziriBERT: a Pre-trained Language Model for the Algerian Dialect
- Leveraging Personal Navigation Assistant Systems Using Automated Social Media Traffic Reporting
- A Topic Guided Pointer-Generator Model for Generating Natural Language Code Summaries
- Search Spaces for Neural Model Training
- Neural Machine Translation for Multilingual Grapheme-to-Phoneme\n Conversion
- Composed Variational Natural Language Generation for Few-shot Intents
- On the Downstream Performance of Compressed Word Embeddings
- BlockBPE: Parallel BPE Tokenization
- Superbloom: Bloom filter meets Transformer
- Semantic Tagging with Deep Residual Networks
- Incremental Adaptation of NMT for Professional Post-editors: A User Study
- Has Machine Translation Achieved Human Parity? A Case for Document-level Evaluation
- Human Languages in Source Code: Auto-Translation for Localized\n Instruction
- Search to Distill: Pearls are Everywhere but not the Eyes
- Towards Robust Neural Machine Translation
- UM-IU@LING at SemEval-2019 Task 6: Identifying Offensive Tweets Using BERT and SVMs
- Deceptive Deletions for Protecting Withdrawn Posts on Social Platforms
- A Latent Morphology Model for Open-Vocabulary Neural Machine Translation
- Federated Learning of N-gram Language Models
- Recurrent Neural Network from Adder's Perspective: Carry-lookahead RNN
- Accelerating Distributed SGD for Linear Regression using Iterative Pre-Conditioning
- Efficient Estimation of Influence of a Training Instance
- How to Fine-Tune BERT for Text Classification?
- Estimate and Replace: A Novel Approach to Integrating Deep Neural Networks with Existing Applications
- Latent-Variable Non-Autoregressive Neural Machine Translation with Deterministic Inference Using a Delta Posterior
- Recurrent Neural Networks (RNNs): A gentle Introduction and Overview
- TauRieL: Targeting Traveling Salesman Problem with a deep reinforcement learning inspired architecture
- Weighted Transformer Network for Machine Translation
- Laplacian Networks: Bounding Indicator Function Smoothness for Neural Network Robustness
- Positioning yourself in the maze of Neural Text Generation: A Task-Agnostic Survey
- Introducing Aspects of Creativity in Automatic Poetry Generation
- Bag-of-Words as Target for Neural Machine Translation
- Improving Truthfulness of Headline Generation
- Improving Human Text Comprehension through Semi-Markov CRF-based Neural Section Title Generation
- Pre-Trained Models: Past, Present and Future
- SkillSpan: Hard and Soft Skill Extraction from English Job Postings
- Zero-shot Dependency Parsing with Pre-trained Multilingual Sentence Representations
- GroupBERT: Enhanced Transformer Architecture with Efficient Grouped Structures
- Dual Supervised Learning for Natural Language Understanding and Generation
- Positional Attention-based Frame Identification with BERT: A Deep Learning Approach to Target Disambiguation and Semantic Frame Selection
- Bertrand-DR: Improving Text-to-SQL using a Discriminative Re-ranker
- BERT as a Teacher: Contextual Embeddings for Sequence-Level Reward
- Going Wider: Recurrent Neural Network With Parallel Cells
- Revisiting Language Encoding in Learning Multilingual Representations
- Entity Projection via Machine Translation for Cross-Lingual NER
- Empirical Evaluation of Deep Learning Model Compression Techniques on the WaveNet Vocoder
- Aggregated Residual Transformations for Deep Neural Networks
- Neural Sequence Model Training via α-divergence Minimization
- Multimodal Dialogue State Tracking By QA Approach with Data Augmentation
- An Empirical Accuracy Law for Sequential Machine Translation: the Case\n of Google Translate
- Infusing Disease Knowledge into BERT for Health Question Answering, Medical Inference and Disease Name Recognition
- Visually grounded emotion regulation via diffusion models and user-driven reappraisal
- Jointly Encoding Word Confusion Network and Dialogue Context with BERT for Spoken Language Understanding
- Margin-Based Regularization and Selective Sampling in Deep Neural Networks
- Advancing CTC-CRF Based End-to-End Speech Recognition with Wordpieces and Conformers
- Hybrid Self-Attention Network for Machine Translation
- StructuralLM: Structural Pre-training for Form Understanding
- Learning to Expand: Reinforced Pseudo-relevance Feedback Selection for Information-seeking Conversations
- Unsupervised Multimodal Neural Machine Translation with Pseudo Visual Pivoting
- ViDove: A Translation Agent System with Multimodal Context and Memory-Augmented Reasoning
- DCN+: Mixed Objective and Deep Residual Coattention for Question Answering
- Generating Symbolic Reasoning Problems with Transformer GANs
- Learning to Evaluate Translation Beyond English: BLEURT Submissions to the WMT Metrics 2020 Shared Task
- Google Neural Machine Translation [wikipedia]
- Google Translate [wikipedia]
- Greg Corrado [wikipedia]
- Long short-term memory [wikipedia]
- Attention Is All You Need [wikipedia]
- Neural machine translation [wikipedia]
- Seq2seq [wikipedia]
- Transformer (deep learning) [wikipedia]
- Mixture of experts [wikipedia]
- Deep learning [wikipedia]
Discussions
Related