Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
2016/09/26 by Yonghui Wu, Wu, Yonghui, Mike Schuster +62 · 2 voices · 5,676 citations
Computer Science · #Artificial intelligence #Artificial neural network #Bridging (networking) #Computer science #Encoder #Example-based machine translation #Inference #Machine translation #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Natural language processing #Phrase #Sentence #Topic Modeling #Transfer-based machine translation #Translation (biology) #Word (group theory) #cs.AI #cs.CL #cs.LG
paper · pdf · doi:10.48550/arxiv.1609.08144
published in arXiv (Cornell University) (Cornell University)
openalex publication_date 2016/09/26 · openalex created_date 2016/10/07 · arxiv created 2016/10/08 · arxiv updated 2016/10/11 · openalex updated_date 2026/07/28
Abstract
Neural Machine Translation (NMT) is an end-to-end learning approach for automated translation, with the potential to overcome many of the weaknesses of conventional phrase-based translation systems. Unfortunately, NMT systems are known to be computationally expensive both in training and in translation inference. Also, most NMT systems have difficulty with rare words. These issues have hindered NMT's use in practical deployments and services, where both accuracy and speed are essential. In this work, we present GNMT, Google's Neural Machine Translation system, which attempts to address many of these issues. Our model consists of a deep LSTM network with 8 encoder and 8 decoder layers using attention and residual connections. To improve parallelism and therefore decrease training time, our attention mechanism connects the bottom layer of the decoder to the top layer of the encoder. To accelerate the final translation speed, we employ low-precision arithmetic during inference computations. To improve handling of rare words, we divide words into a limited set of common sub-word units ("wordpieces") for both input and output. This method provides a good balance between the flexibility of "character"-delimited models and the efficiency of "word"-delimited models, naturally handles translation of rare words, and ultimately improves the overall accuracy of the system. Our beam search technique employs a length-normalization procedure and uses a coverage penalty, which encourages generation of an output sentence that is most likely to cover all the words in the source sentence. On the WMT'14 English-to-French and English-to-German benchmarks, GNMT achieves competitive results to state-of-the-art. Using a human side-by-side evaluation on a set of isolated simple sentences, it reduces translation errors by an average of 60% compared to Google's phrase-based production system.
Cited by
- How Fast Can Reward Models Score? A Systems Study of C++ and PyTorch Inference Runtimes for RLHF
- ChineseBERT: Chinese Pretraining Enhanced by Glyph and Pinyin Information
- A Deep Reinforcement Learning Algorithm for the Vehicle Routing Problem with Stochastic Demands and Outsourcing
- Artificial Intelligence for Mathematical Reasoning: An Integrated Survey of Language Models, Neuro-symbolic Systems, and Verified Discovery
- TD-DPO: Difference-Aware Preference Optimization for Mitigating Sycophancy in Clinical Autism Intervention Dialogue
- Tokenization with Split Trees
- Stochasticity in Tokenisation Improves Robustness
- Leveraging Natural Language Processing to Unravel the Mystery of Life: A Review of NLP Approaches in Genomics, Transcriptomics, and Proteomics
- Are Statistical Methods Obsolete in the Era of Deep Learning? A Study of ODE Inverse Problems
- Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMs
- Assessing the Impact of Typological Features on Multilingual Machine Translation in the Age of Large Language Models
- Joint Optimization for Greedy Longest-match Tokenization
- BHARATI: Morphology-Aware Tokenizers for Classical Indian Languages with Subword Fertility Analysis
- Look Closer! An Adversarial Parametric Editing Framework for Hallucination Mitigation in VLMs
- TokSuite: Measuring the Impact of Tokenizer Choice on Language Model Behavior
- ReasonCD: A Multimodal Reasoning Large Model for Implicit Change-of-Interest Semantic Mining
- Building Patient Journeys in Hebrew: A Language Model for Clinical Timeline Extraction
- Tacotron: Towards End-to-End Speech Synthesis
- Efficient Continual Learning in Neural Machine Translation: A Low-Rank Adaptation Approach
- Factorization tricks for LSTM networks
- Atlas: A Dataset and Benchmark for E-commerce Clothing Product\n Categorization
- Learning to Generate Reviews and Discovering Sentiment
- Improving Post Training Neural Quantization: Layer-wise Calibration and Integer Programming
- Quasi-Recurrent Neural Networks
- ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT
- An Empirical Investigation of Multi-bridge Multilingual NMT models
- SimCLS: A Simple Framework for Contrastive Learning of Abstractive Summarization
- A Hybrid Word-Character Approach to Abstractive Summarization
- Hard-Coded Gaussian Attention for Neural Machine Translation
- Explaining Documents' Relevance to Search Queries
- Advancing Acoustic-to-Word CTC Model with Attention and Mixed-Units
- CARL: Criticality-Aware Agentic Reinforcement Learning
- On the Binding Problem in Artificial Neural Networks
- Representation Degeneration Problem in Training Natural Language Generation Models
- The brain-AI convergence: Predictive and generative world models for general-purpose computation
- Learning Energy-Based Approximate Inference Networks for Structured Applications in NLP
- Empowering Things with Intelligence: A Survey of the Progress, Challenges, and Opportunities in Artificial Intelligence of Things
- UPB at SemEval-2020 Task 12: Multilingual Offensive Language Detection on Social Media by Fine-tuning a Variety of BERT-based Models
- SGD-QA: Fast Schema-Guided Dialogue State Tracking for Unseen Services
- Layer-Wise Multi-View Learning for Neural Machine Translation
- Pretrained Transformers for Simple Question Answering over Knowledge Graphs
- Deep Learning for Environmentally Robust Speech Recognition: An Overview of Recent Developments
- What's in a Name? -- Gender Classification of Names with Character Based Machine Learning Models
- SDM: Sequential Deep Matching Model for Online Large-scale Recommender System
- Semantic Explanations of Predictions
- Implementing Neural Turing Machines
- An End-to-End Approach to Automatic Speech Assessment for Cantonese-speaking People with Aphasia
- MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers
- Syntax-Enhanced Neural Machine Translation with Syntax-Aware Word Representations
- Mono vs Multilingual Transformer-based Models: a Comparison across Several Language Tasks
- VALUE: A Multi-Task Benchmark for Video-and-Language Understanding Evaluation
- Bayesian Recurrent Neural Networks
- Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERT
- MKD: a Multi-Task Knowledge Distillation Approach for Pretrained Language Models
- Charformer: Fast Character Transformers via Gradient-based Subword Tokenization
- SQWA: Stochastic Quantized Weight Averaging for Improving the Generalization Capability of Low-Precision Deep Neural Networks
- CTRL: A Conditional Transformer Language Model for Controllable Generation
- A Deep Reinforcement Learning Chatbot
- Compositional Generalization for Primitive Substitutions
- Sub-word Level Lip Reading With Visual Attention
- Kaleido-BERT: Vision-Language Pre-training on Fashion Domain
- Translation, Sentiment and Voices: A Computational Model to Translate and Analyse Voices from Real-Time Video Calling
- Fine-Tuning Language Models from Human Preferences
- Creative GANs for generating poems, lyrics, and metaphors
- Enhancing Clinical Information Extraction with Transferred Contextual Embeddings
- A Lightweight Recurrent Network for Sequence Modeling
- StrucTexT: Structured Text Understanding with Multi-Modal Transformers
- Exploring question answering: metric analysis and evaluation framework for enhanced interpretability
- Students Need More Attention: BERT-based AttentionModel for Small Data with Application to AutomaticPatient Message Triage
- Hippo: Taming Hyper-parameter Optimization of Deep Learning with Stage Trees
- Modeling Past and Future for Neural Machine Translation
- On the importance of single directions for generalization
- ERNIE: Enhanced Representation through Knowledge Integration
- Neural machine translation in EFL classrooms: learners’ vocabulary improvement, immediate vocabulary retention and delayed vocabulary retention
- Document Graph for Neural Machine Translation
- PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization
- Does injecting linguistic structure into language models lead to better alignment with brain recordings?
- Neural Machine Translation and Sequence-to-sequence Models: A Tutorial
- Evaluating Subword Tokenization Techniques for Bengali: A Benchmark Study with BengaliBPE
- Segmentation Beyond Defaults: Asymmetrical Byte Pair Encoding for Optimal Machine Translation Performance
- BIG MOOD: Relating Transformers to Explicit Commonsense Knowledge
- Adversarial Attacks and Defense on Texts: A Survey
- DTS: Enhancing Large Reasoning Models via Decoding Tree Sketching
- Multilingual BERT language model for medical tasks: Evaluation on domain-specific adaptation and cross-linguality
- Modular Linear Tokenization (MLT)
- Unsupervised Machine Translation Using Monolingual Corpora Only
- Constrained Language Models Yield Few-Shot Semantic Parsers
- Prediction with a Short Memory
- Simple, Scalable Adaptation for Neural Machine Translation
- Structure-Invariant Testing for Machine Translation
- A Comprehensive Exploration on WikiSQL with Table-Aware Word\n Contextualization
- AutoML-Zero: Evolving Machine Learning Algorithms From Scratch
- Cross-Lingual Semantic Role Labeling with High-Quality Translated Training Corpus
- Language Through a Prism: A Spectral Approach for Multiscale Language Representations
- Vulnerability Under Adversarial Machine Learning: Bias or Variance?
- ReClor: A Reading Comprehension Dataset Requiring Logical Reasoning
- Portuguese Named Entity Recognition using BERT-CRF
- An Empirical Study of Mini-Batch Creation Strategies for Neural Machine Translation
- Efficient Attentions for Long Document Summarization
- VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Experts
- Attention-based sequence-to-sequence model for speech recognition: development of state-of-the-art system on LibriSpeech and its application to non-native English
- MARMOT: A Deep Learning Framework for Constructing Multimodal Representations for Vision-and-Language Tasks
- Mixed-Precision Training for NLP and Speech Recognition with OpenSeq2Seq
- Unicoder-VL: A Universal Encoder for Vision and Language by Cross-modal Pre-training
- Parallax: Sparsity-aware Data Parallel Training of Deep Neural Networks
- Interactive neural machine translation
- GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding
- Mixed Precision Training
- Semantic Parsing with Dual Learning
- Towards Neural Machine Translation with Partially Aligned Corpora
- Generative Language Modeling for Automated Theorem Proving
- A Deep Reinforced Model for Abstractive Summarization
- SAFER: A Structure-free Approach for Certified Robustness to Adversarial Word Substitutions
- FashionBERT: Text and Image Matching with Adaptive Loss for Cross-modal Retrieval
- Learning Algorithms for Active Learning
- Deep learning: new computational modelling techniques for genomics
- Stochastic Beams and Where to Find Them: The Gumbel-Top-k Trick for Sampling Sequences Without Replacement
- Selfie: Self-supervised Pretraining for Image Embedding
- Testing Machine Translation via Referential Transparency
- PairConnect: A Compute-Efficient MLP Alternative to Attention
- Interpretable Self-supervised Multi-task Learning for COVID-19 Information Retrieval and Extraction
- Later-stage Minimum Bayes-Risk Decoding for Neural Machine Translation
- Multimodal learning enables chat-based exploration of single-cell data
- Exploring Sparsity in Recurrent Neural Networks
- AdvPicker: Effectively Leveraging Unlabeled Data via Adversarial Discriminator for Cross-Lingual NER
- Accelerating Sparse Deep Neural Networks
- Learning Deep Transformer Models for Machine Translation
- Two-Timescale Voltage Control in Distribution Grids Using Deep Reinforcement Learning
- Resurrecting Submodularity for Neural Text Generation
- Bridging the Gap between Spatial and Spectral Domains: A Survey on Graph Neural Networks
- OpenNMT: Neural Machine Translation Toolkit
- An Evaluation Dataset and Strategy for Building Robust Multi-turn Response Selection Model
- A Study of Multilingual Neural Machine Translation
- LAMPRET: Layout-Aware Multimodal PreTraining for Document Understanding
- Recurrent Graph Syntax Encoder for Neural Machine Translation
- Denoising based Sequence-to-Sequence Pre-training for Text Generation
- Shallow-to-Deep Training for Neural Machine Translation
- Human-centric Metric for Accelerating Pathology Reports Annotation
- On the Significance of Question Encoder Sequence Model in the Out-of-Distribution Performance in Visual Question Answering
- ActBERT: Learning Global-Local Video-Text Representations
- What the [MASK]? Making Sense of Language-Specific BERT Models
- DeepZensols: Deep Natural Language Processing Framework
- Information Aggregation for Multi-Head Attention with Routing-by-Agreement
- Robust Neural Machine Translation with Doubly Adversarial Inputs
- Deep geometric knowledge distillation with graphs
- Convolutional Attention-based Seq2Seq Neural Network for End-to-End ASR
- Deep Hierarchical Classification for Category Prediction in E-commerce System
- Bayesian Attention Modules
- Slip Detection: Analysis and Calibration of Univariate Tactile Signals
- A Comprehensive Survey on Hardware-Aware Neural Architecture Search
- ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks
- An Unsupervised Autoregressive Model for Speech Representation Learning
- Pre-trained Language Model for Biomedical Question Answering
- StarCraft II: A New Challenge for Reinforcement Learning
- UIBert: Learning Generic Multimodal Representations for UI Understanding
- ST-BERT: Cross-modal Language Model Pre-training For End-to-end Spoken Language Understanding
- Efficiently Reusing Old Models Across Languages via Transfer Learning
- Self-Supervised Text-Vision Alignment for Automated Brain MRI Abnormality Detection: A Multicenter Study (ALIGN Study)
- A Computational Model of Commonsense Moral Decision Making
- Multi-Head Attention with Disagreement Regularization
- VL-BERT: Pre-training of Generic Visual-Linguistic Representations
- Generating Wikipedia by Summarizing Long Sequences
- PiSLTRc: Position-informed Sign Language Transformer with Content-aware Convolution
- Can LLMs Translate Human Instructions into a Reinforcement Learning Agent's Internal Emergent Symbolic Representation?
- Daydream: Accurately Estimating the Efficacy of Optimizations for DNN Training
- SEAL: Segment-wise Extractive-Abstractive Long-form Text Summarization
- CNN Is All You Need
- Full-Sentence Models Perform Better in Simultaneous Translation Using the Information Enhanced Decoding Strategy
- Bridging Neural Machine Translation and Bilingual Dictionaries
- Log-based Anomaly Detection Without Log Parsing
- A Survey of Domain Adaptation for Neural Machine Translation
- Pctx: Tokenizing Personalized Context for Generative Recommendation
- Stand-Alone Self-Attention in Vision Models
- A spelling correction model for end-to-end speech recognition
- Forging GEMs: Advancing Greek NLP through Quality-Based Corpus Curation
- SODBench: A Large Language Model Approach to Documenting Spreadsheet Operations
- Psycholinguistic Tripartite Graph Network for Personality Detection
- Challenges in Data-to-Document Generation
- Denoising Neural Machine Translation Training with Trusted Data and Online Data Selection
- Themis: Fair and Efficient GPU Cluster Scheduling
- Transformer-Based Low-Resource Language Translation: A Study on Standard Bengali to Sylheti
- One Big Net For Everything
- ImageBERT: Cross-modal Pre-training with Large-scale Weak-supervised Image-Text Data
- Stochastic Gradient/Mirror Descent: Minimax Optimality and Implicit Regularization
- Automatic Classification of Human Translation and Machine Translation: A Study from the Perspective of Lexical Diversity
- Device Placement Optimization with Reinforcement Learning
- Single-Queue Decoding for Neural Machine Translation
- Enhancing Pre-trained Chinese Character Representation with Word-aligned Attention
- Zero-Shot Dual Machine Translation
- Unsupervised State Representation Learning in Atari
- iCapsNets: Towards Interpretable Capsule Networks for Text Classification
- AdvAug: Robust Adversarial Augmentation for Neural Machine Translation
- The University of Sydney's Machine Translation System for WMT19
- Towards General Purpose Vision Systems
- Give your Text Representation Models some Love: the Case for Basque
- Interactive-Predictive Neural Machine Translation through Reinforcement and Imitation
- Learning a Multi-Domain Curriculum for Neural Machine Translation
- Interpretable Entity Representations through Large-Scale Typing
- CommandSans: Securing AI Agents with Surgical Precision Prompt Sanitization
- Annotating publicly-available samples and studies using interpretable modeling of unstructured metadata
- Less is More: ClipBERT for Video-and-Language Learning via Sparse Sampling
- VS-Quant: Per-vector Scaled Quantization for Accurate Low-Precision Neural Network Inference
- A Low-Resource Speech-Driven NLP Pipeline for Sinhala Dyslexia Assistance
- Cold-Start Reinforcement Learning with Softmax Policy Gradient
- Increasing LLM response trustworthiness using voting ensembles
- Translating Phrases in Neural Machine Translation
- Evaluating Embedding Frameworks for Scientific Domain
- Improving Sign Language Translation with Monolingual Data by Sign Back-Translation
- VisualBERT: A Simple and Performant Baseline for Vision and Language
- Efficient Neural Audio Synthesis
- Rearchitecting Datacenter Lifecycle for AI: A TCO-Driven Framework
- Better with Less: Small Proprietary Models Surpass Large Language Models in Financial Transaction Understanding
- Efficient Sketching and Nearest Neighbor Search Algorithms for Sparse Vector Sets
- Enabling Physical AI through Biological Principles
- How Do Source-side Monolingual Word Embeddings Impact Neural Machine Translation?
- MLPerf Training Benchmark
- Exploring the limits of Concurrency in ML Training on Google TPUs
- Attend and Diagnose: Clinical Time Series Analysis using Attention Models
- Machine Reading Comprehension: a Literature Review
- Deep Multi-Kernel Convolutional LSTM Networks and an Attention-Based Mechanism for Videos
- Cross-Modality Relevance for Reasoning on Language and Vision
- Combinatorial Testing for Deep Learning Systems
- Multiscale Collaborative Deep Models for Neural Machine Translation
- Understanding Reuse, Performance, and Hardware Cost of DNN Dataflows: A Data-Centric Approach Using MAESTRO
- AI Brown and AI Koditex: LLM-Generated Corpora Comparable to Traditional Corpora of English and Czech Texts
- LXMERT: Learning Cross-Modality Encoder Representations from Transformers
- Hybrid Retrieval-Generation Reinforced Agent for Medical Image Report Generation
- Learning by Abstraction: The Neural State Machine
- Learning to Detect Unacceptable Machine Translations for Downstream Tasks
- Zero-shot Knowledge Transfer via Adversarial Belief Matching
- Generation of focused drug molecule library using recurrent neural network
- Biomedical Entity Representations with Synonym Marginalization
- CodeBERT: A Pre-Trained Model for Programming and Natural Languages
- Impact of AI assistance on student agency
- Literature review on vulnerability detection using NLP technology
- A First Look at Mobile Intelligence: Architecture, Experimentation and Challenges
- All You Can Embed: Natural Language based Vehicle Retrieval with Spatio-Temporal Transformers
- TRANS-BLSTM: Transformer with Bidirectional LSTM for Language Understanding
- Local Contextual Attention with Hierarchical Structure for Dialogue Act Recognition
- Integer Quantization for Deep Learning Inference: Principles and Empirical Evaluation
- Controllable Text Simplification with Explicit Paraphrasing
- Hybrid Data-Model Parallel Training for Sequence-to-Sequence Recurrent Neural Network Machine Translation
- An Exploration of Word Embedding Initialization in Deep-Learning Tasks
- Dynamic Curriculum Learning for Low-Resource Neural Machine Translation
- Multimodal Learning for Hateful Memes Detection
- Parallel Blockwise Knowledge Distillation for Deep Neural Network Compression
- KVL-BERT: Knowledge Enhanced Visual-and-Linguistic BERT for Visual Commonsense Reasoning
- Semi-Autoregressive Neural Machine Translation
- Speculating LLMs' Chinese Training Data Pollution from Their Tokens
- Generating Sentences by Editing Prototypes
- Analysis of Predictive Coding Models for Phonemic Representation Learning in Small Datasets
- M4SER: Multimodal, Multirepresentation, Multitask, and Multistrategy Learning for Speech Emotion Recognition
- An overview of neural architectures for self-supervised audio representation learning from masked spectrograms
- Multilingual Neural Machine Translation for Zero-Resource Languages
- Democratizing Production-Scale Distributed Deep Learning
- Gram-CTC: Automatic Unit Selection and Target Decomposition for Sequence Labelling
- Aligned Image-Word Representations Improve Inductive Transfer Across Vision-Language Tasks
- Reversible Architectures for Arbitrarily Deep Residual Neural Networks
- Bit-Flip Attack: Crushing Neural Network with Progressive Bit Search
- KuBERT: Central Kurdish BERT Model and Its Application for Sentiment Analysis
- Sentiment-based Candidate Selection for NMT
- Dynamic Programming Encoding for Subword Segmentation in Neural Machine Translation
- Model Compression and Hardware Acceleration for Neural Networks: A Comprehensive Survey
- A Comparative Study of Transformer-Based Language Models on Extractive Question Answering
- Recent Trends in the Use of Deep Learning Models for Grammar Error Handling
- Enabling Multi-Source Neural Machine Translation By Concatenating Source Sentences In Multiple Languages
- Attention Forcing for Sequence-to-sequence Model Training
- Black-box Generation of Adversarial Text Sequences to Evade Deep Learning Classifiers
- Regularizing Deep Neural Networks by Noise: Its Interpretation and Optimization
- A Random Matrix Perspective on Mixtures of Nonlinearities for Deep Learning
- MOOCRep: A Unified Pre-trained Embedding of MOOC Entities
- Discriminative Adversarial Search for Abstractive Summarization
- Neural Text Generation with Artificial Negative Examples
- Efficient Algorithms for Device Placement of DNN Graph Operators
- XPipe: Efficient Pipeline Model Parallelism for Multi-GPU DNN Training
- Open-endedness in AI systems, cellular evolution and intellectual discussions
- InterBERT: Vision-and-Language Interaction for Multi-modal Pretraining
- CAGFuzz: Coverage-Guided Adversarial Generative Fuzzing Testing of Deep Learning Systems
- Topic Modeling with Contextualized Word Representation Clusters
- Automatic Cross-Replica Sharding of Weight Update in Data-Parallel Training
- Dynamics of Deep Neural Networks and Neural Tangent Hierarchy
- Massively Multilingual Neural Machine Translation
- HufuNet: Embedding the Left Piece as Watermark and Keeping the Right Piece for Ownership Verification in Deep Neural Networks
- Experiments with Rich Regime Training for Deep Learning
- Data Augmentation Generative Adversarial Networks
- The Natural Language Decathlon: Multitask Learning as Question Answering
- Graph-based Filtering of Out-of-Vocabulary Words for Encoder-Decoder Models
- A Study of Reinforcement Learning for Neural Machine Translation
- Neural Network Distiller: A Python Package For DNN Compression Research
- Synergy: Resource Sensitive DNN Scheduling in Multi-Tenant Clusters
- LfEdNet: A Task-based Day-ahead Load Forecasting Model for Stochastic Economic Dispatch
- Corpora Generation for Grammatical Error Correction
- Hunyuan-MT Technical Report
- Globally Optimal Gradient Descent for a ConvNet with Gaussian Inputs
- Multi-modal Transformer for Video Retrieval
- HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training
- Sharp Minima Can Generalize For Deep Nets
- Neural Machine Translation with Explicit Phrase Alignment
- A Runtime-Based Computational Performance Predictor for Deep Neural Network Training
- chDzDT: Word-level morphology-aware language model for Algerian social media text
- Noisy Self-Knowledge Distillation for Text Summarization
- Understanding and Enhancing the Use of Context for Machine Translation
- Modeling Fine-Grained Entity Types with Box Embeddings
- Neural network gradient-based learning of black-box function interfaces
- Independently Recurrent Neural Network (IndRNN): Building A Longer and Deeper RNN
- A Correspondence Between Random Neural Networks and Statistical Field Theory
- Seeing Out of tHe bOx: End-to-End Pre-training for Vision-Language Representation Learning
- Learning to Remember Rare Events
- Approximating Continuous Functions by ReLU Nets of Minimal Width
- When Does Pretraining Help? Assessing Self-Supervised Learning for Law and the CaseHOLD Dataset
- PAIR: Planning and Iterative Refinement in Pre-trained Transformers for Long Text Generation
- Axiomatic Attribution for Deep Networks
- Enhancing Robustness of Autoregressive Language Models against Orthographic Attacks via Pixel-based Approach
- FlowletFormer: Network Behavioral Semantic Aware Pre-training Model for Traffic Classification
- Sequence Model with Self-Adaptive Sliding Window for Efficient Spoken Document Segmentation
- An Optimized and Energy-Efficient Parallel Implementation of Non-Iteratively Trained Recurrent Neural Networks
- SemToken: Semantic-Aware Tokenization for Efficient Long-Context Language Modeling
- Unsupervised Learning Layers for Video Analysis
- Counterspeech for Mitigating the Influence of Media Bias: Comparing Human and LLM-Generated Responses
- In2x at WMT25 Translation Task
- Learning to Coordinate Multiple Reinforcement Learning Agents for Diverse Query Reformulation
- BERT might be Overkill: A Tiny but Effective Biomedical Entity Linker based on Residual Convolutional Neural Networks
- PAWS: Paraphrase Adversaries from Word Scrambling
- Language as a matrix product state
- DeLS-3D: Deep Localization and Segmentation with a 3D Semantic Map
- An Empirical Comparison of Syllabuses for Curriculum Learning
- A Study of BFLOAT16 for Deep Learning Training
- Generating Titles for Web Tables
- Deep Neural Network for Semantic-based Text Recognition in Images
- When Alignment Hurts: Decoupling Representational Spaces in Multilingual Models
- Wavy Transformer
- Modeling Diagnostic Label Correlation for Automatic ICD Coding
- Score-informed Neural Operator for Enhancing Ordering-based Causal Discovery
- Learning to Generate Multiple Style Transfer Outputs for an Input Sentence
- Language Models with Transformers
- Scalable Transformers for Neural Machine Translation
- SupraTok: Cross-Boundary Tokenization for Enhanced Language Model Performance
- Model Interpretability and Rationale Extraction by Input Mask Optimization
- Effective Use of Graph Convolution Network and Contextual Sub-Tree forCommodity News Event Extraction
- Adma: A Flexible Loss Function for Neural Networks
- Optimizing Sponsored Search Ranking Strategy by Deep Reinforcement Learning
- DTMT: A Novel Deep Transition Architecture for Neural Machine Translation
- Rethinking Dialogue State Tracking with Reasoning
- Shatter: An Efficient Transformer Encoder with Single-Headed Self-Attention and Relative Sequence Partitioning
- AI-Powered Social Bots
- Generating Representative Headlines for News Stories
- ModiPick: SLA-aware Accuracy Optimization For Mobile Deep Inference
- A Comparison of Approaches to Document-level Machine Translation
- Anomaly Detection for IoT Global Connectivity
- Phantom: A High-Performance Computational Core for Sparse Convolutional Neural Networks
- Improving Back-Translation with Uncertainty-based Confidence Estimation
- SignBERT: Pre-Training of Hand-Model-Aware Representation for Sign Language Recognition
- How LSTM Encodes Syntax: Exploring Context Vectors and Semi-Quantization on Natural Text
- Marvel: A Data-centric Compiler for DNN Operators on Spatial Accelerators
- SE3M: A model for software effort estimation using pre-trained embedding models
- A Survey on Non-Intrusive ASR Refinement: From Output-Level Correction to Full-Model Distillation
- MeanSum: A Neural Model for Unsupervised Multi-document Abstractive Summarization
- Recurrent multiple shared layers in Depth for Neural Machine Translation
- MIRA: Leveraging Multi-Intention Co-click Information in Web-scale Document Retrieval using Deep Neural Networks
- Being-ahead: Benchmarking and Exploring Accelerators for Hardware-Efficient AI Deployment
- TAG : Type Auxiliary Guiding for Code Comment Generation
- Modelling and Classifying the Components of a Literature Review
- Semantic Sentence Matching with Densely-connected Recurrent and Co-attentive Information
- Text Summarization with Pretrained Encoders
- NTT's Machine Translation Systems for WMT19 Robustness Task
- BERT-DST: Scalable End-to-End Dialogue State Tracking with Bidirectional Encoder Representations from Transformer
- An Actor-Critic Algorithm for Sequence Prediction
- PolyDL: Polyhedral Optimizations for Creation of High Performance DL primitives
- ChEmbed: Enhancing Chemical Literature Search Through Domain-Specific Text Embeddings
- AR: Auto-Repair the Synthetic Data for Neural Machine Translation
- Bridging Text and Video: A Universal Multimodal Transformer for Video-Audio Scene-Aware Dialog
- FeatureCuts: Feature Selection for Large Data by Optimizing the Cutoff
- CancerBERT: a BERT model for Extracting Breast Cancer Phenotypes from Electronic Health Records
- Neural Machine Translating from Natural Language to SPARQL
- Large-Scale Evolution of Image Classifiers
- Learning Efficient Lexically-Constrained Neural Machine Translation with External Memory
- Explaining Natural Language Processing Classifiers with Occlusion and Language Modeling
- Language Arithmetics: Towards Systematic Language Neuron Identification and Manipulation
- Dual-mode ASR: Unify and Improve Streaming ASR with Full-context Modeling
- Collaborative Training of GANs in Continuous and Discrete Spaces for Text Generation
- Phenotyping of Clinical Notes with Improved Document Classification Models Using Contextualized Neural Language Models
- Neural CRF Model for Sentence Alignment in Text Simplification
- Length Representations in Large Language Models
- "The Boating Store Had Its Best Sail Ever": Pronunciation-attentive Contextualized Pun Recognition
- BERTScore: Evaluating Text Generation with BERT
- Machine Learning With Neuromorphic Photonics
- Improving Neural Machine Translation by Bidirectional Training
- Tensorized Embedding Layers for Efficient Model Compression
- Make (Nearly) Every Neural Network Better: Generating Neural Network Ensembles by Weight Parameter Resampling
- Chunk-Based Bi-Scale Decoder for Neural Machine Translation
- How Can BERT Help Lexical Semantics Tasks?
- Robust, Deep, and Reinforcement Learning for Management of Communication and Power Networks
- Tree Transformer: Integrating Tree Structures into Self-Attention
- Automatic Dialogic Instruction Detection for K-12 Online One-on-one Classes
- Towards Transfer Learning for End-to-End Speech Synthesis from Deep Pre-Trained Language Models
- Understanding Natural Language Instructions for Fetching Daily Objects Using GAN-Based Multimodal Target-Source Classification
- Enhancing Hindi NER in Low Context: A Comparative study of Transformer-based models with vs. without Retrieval Augmentation
- On the Blindspots of Convolutional Networks
- Towards Safer Smart Contracts: A Sequence Learning Approach to Detecting Security Threats
- Cseq2seq: Cyclic Sequence-to-Sequence Learning
- Exploring Hyper-Parameter Optimization for Neural Machine Translation on GPU Architectures
- On Accurate Evaluation of GANs for Language Generation
- Jointly Optimizing Diversity and Relevance in Neural Response Generation
- Taylor's law for Human Linguistic Sequences
- Connecting the Dots Between MLE and RL for Sequence Prediction
- Liputan6: A Large-scale Indonesian Dataset for Text Summarization
- Deep Recurrent Neural Network for Protein Function Prediction from Sequence
- STN4DST: A Scalable Dialogue State Tracking based on Slot Tagging Navigation
- Cross-Lingual BERT Transformation for Zero-Shot Dependency Parsing
- Deep Neural Machine Translation with Linear Associative Unit
- Data Augmentation for Low-Resource Named Entity Recognition Using Backtranslation
- Improving Pre-Trained Multilingual Models with Vocabulary Expansion
- Simplifying Neural Machine Translation with Addition-Subtraction Twin-Gated Recurrent Networks
- Machine Translation between Vietnamese and English: an Empirical Study
- Multilingual Constituency Parsing with Self-Attention and Pre-Training
- The Cross-Lingual Arabic Information REtrieval (CLAIRE) System
- Dialogue Summarization with Supporting Utterance Flow Modeling and Fact Regularization
- UzBERT: pretraining a BERT model for Uzbek
- Improved Speech Representations with Multi-Target Autoregressive Predictive Coding
- Multilingual AMR-to-Text Generation
- Teaching Temporal Logics to Neural Networks
- AdsGNN: Behavior-Graph Augmented Relevance Modeling in Sponsored Search
- Foundations of Vector Retrieval
- Neural Information Retrieval: A Literature Review
- Hierarchical Multitask Learning for CTC-based Speech Recognition
- BoostingBERT:Integrating Multi-Class Boosting into BERT for NLP Tasks
- Autocorrect in the Process of Translation -- Multi-task Learning Improves Dialogue Machine Translation
- Improving Readability for Automatic Speech Recognition Transcription
- They, Them, Theirs: Rewriting with Gender-Neutral English
- Recurrent Neural Network-based Model for Accelerated Trajectory Analysis in AIMD Simulations
- Incorporating BERT into Neural Machine Translation
- Multi-layer Representation Fusion for Neural Machine Translation
- Residual Continual Learning
- Uncertainty-Aware Semantic Augmentation for Neural Machine Translation
- Focus Group on Artificial Intelligence for Health
- BERT Meets Chinese Word Segmentation
- Reference Language based Unsupervised Neural Machine Translation
- ATOM: Commit Message Generation Based on Abstract Syntax Tree and Hybrid Ranking
- EPNAS: Efficient Progressive Neural Architecture Search
- Unified Language Model Pre-training for Natural Language Understanding and Generation
- Impact of Code Context and Prompting Strategies on Automated Unit Test Generation with Modern General-Purpose Large Language Models
- Exploiting Deep Representations for Neural Machine Translation
- Dual Inference for Improving Language Understanding and Generation
- Seed LiveInterpret 2.0: End-to-end Simultaneous Speech-to-speech Translation with Your Voice
- Artificial Quantum Neural Network: quantum neurons, logical elements and tests of convolutional nets
- Towards a resource for multilingual lexicons: an MT assisted and human-in-the-loop multilingual parallel corpus with multi-word expression annotation
- Boosting Active Learning for Speech Recognition with Noisy Pseudo-labeled Samples
- Seed-X: Building Strong Multilingual Translation LLM with 7B Parameters
- Step-Audio 2 Technical Report
- Fast Structured Decoding for Sequence Models
- Do Transformer Attention Heads Provide Transparency in Abstractive Summarization?
- Pre-trained Language Models in Biomedical Domain: A Systematic Survey
- Convolutional Sequence to Sequence Learning
- DziriBERT: a Pre-trained Language Model for the Algerian Dialect
- Leveraging Personal Navigation Assistant Systems Using Automated Social Media Traffic Reporting
- A Topic Guided Pointer-Generator Model for Generating Natural Language Code Summaries
- Search Spaces for Neural Model Training
- Neural Machine Translation for Multilingual Grapheme-to-Phoneme Conversion
- Composed Variational Natural Language Generation for Few-shot Intents
- Learning to Decipher Hate Symbols
- Transferring Monolingual Model to Low-Resource Language: The Case of Tigrinya
- On the Downstream Performance of Compressed Word Embeddings
- BlockBPE: Parallel BPE Tokenization
- Superbloom: Bloom filter meets Transformer
- Semantic Tagging with Deep Residual Networks
- Incremental Adaptation of NMT for Professional Post-editors: A User Study
- Has Machine Translation Achieved Human Parity? A Case for Document-level Evaluation
- Human Languages in Source Code: Auto-Translation for Localized Instruction
- Deep Learning: Our Miraculous Year 1990-1991
- Search to Distill: Pearls are Everywhere but not the Eyes
- Towards Robust Neural Machine Translation
- UM-IU@LING at SemEval-2019 Task 6: Identifying Offensive Tweets Using BERT and SVMs
- Deceptive Deletions for Protecting Withdrawn Posts on Social Platforms
- A Latent Morphology Model for Open-Vocabulary Neural Machine Translation
- Federated Learning of N-gram Language Models
- Recurrent Neural Network from Adder's Perspective: Carry-lookahead RNN
- Accelerating Distributed SGD for Linear Regression using Iterative Pre-Conditioning
- Efficient Estimation of Influence of a Training Instance
- How to Fine-Tune BERT for Text Classification?
- Estimate and Replace: A Novel Approach to Integrating Deep Neural Networks with Existing Applications
- Latent-Variable Non-Autoregressive Neural Machine Translation with Deterministic Inference Using a Delta Posterior
- Recurrent Neural Networks (RNNs): A gentle Introduction and Overview
- TauRieL: Targeting Traveling Salesman Problem with a deep reinforcement learning inspired architecture
- Weighted Transformer Network for Machine Translation
- Laplacian Networks: Bounding Indicator Function Smoothness for Neural Network Robustness
- Positioning yourself in the maze of Neural Text Generation: A Task-Agnostic Survey
- Introducing Aspects of Creativity in Automatic Poetry Generation
- Bag-of-Words as Target for Neural Machine Translation
- Improving Truthfulness of Headline Generation
- Improving Human Text Comprehension through Semi-Markov CRF-based Neural Section Title Generation
- Pre-Trained Models: Past, Present and Future
- SkillSpan: Hard and Soft Skill Extraction from English Job Postings
- Zero-shot Dependency Parsing with Pre-trained Multilingual Sentence Representations
- GroupBERT: Enhanced Transformer Architecture with Efficient Grouped Structures
- Dual Supervised Learning for Natural Language Understanding and Generation
- Positional Attention-based Frame Identification with BERT: A Deep Learning Approach to Target Disambiguation and Semantic Frame Selection
- Bertrand-DR: Improving Text-to-SQL using a Discriminative Re-ranker
- BERT as a Teacher: Contextual Embeddings for Sequence-Level Reward
- Going Wider: Recurrent Neural Network With Parallel Cells
- Exploring Pre-training with Alignments for RNN Transducer based End-to-End Speech Recognition
- Revisiting Language Encoding in Learning Multilingual Representations
- Entity Projection via Machine Translation for Cross-Lingual NER
- Empirical Evaluation of Deep Learning Model Compression Techniques on the WaveNet Vocoder
- Aggregated Residual Transformations for Deep Neural Networks
- Neural Sequence Model Training via α-divergence Minimization
- Multimodal Dialogue State Tracking By QA Approach with Data Augmentation
- An Empirical Accuracy Law for Sequential Machine Translation: the Case\n of Google Translate
- Infusing Disease Knowledge into BERT for Health Question Answering, Medical Inference and Disease Name Recognition
- Visually grounded emotion regulation via diffusion models and user-driven reappraisal
- Jointly Encoding Word Confusion Network and Dialogue Context with BERT for Spoken Language Understanding
- Margin-Based Regularization and Selective Sampling in Deep Neural Networks
- Advancing CTC-CRF Based End-to-End Speech Recognition with Wordpieces and Conformers
- Hybrid Self-Attention Network for Machine Translation
- StructuralLM: Structural Pre-training for Form Understanding
- Learning to Expand: Reinforced Pseudo-relevance Feedback Selection for Information-seeking Conversations
- Unsupervised Multimodal Neural Machine Translation with Pseudo Visual Pivoting
- ViDove: A Translation Agent System with Multimodal Context and Memory-Augmented Reasoning
- DCN+: Mixed Objective and Deep Residual Coattention for Question Answering
- Generating Symbolic Reasoning Problems with Transformer GANs
- Learning to Evaluate Translation Beyond English: BLEURT Submissions to the WMT Metrics 2020 Shared Task
- Residual Tree Aggregation of Layers for Neural Machine Translation
- Improving Text Generation with Student-Forcing Optimal Transport
- Neural Program Synthesis with Priority Queue Training
- Adaptive Dense-to-Sparse Paradigm for Pruning Online Recommendation System with Non-Stationary Data
- Noisin: Unbiased Regularization for Recurrent Neural Networks
- Stochastic Mirror Descent on Overparameterized Nonlinear Models: Convergence, Implicit Regularization, and Generalization
- Langevin Cooling for Domain Translation
- Story Generation from Sequence of Independent Short Descriptions
- Multi-node Bert-pretraining: Cost-efficient Approach
- MedicalBERT: enhancing biomedical natural language processing using pretrained BERT-based model
- Speeding up Deep Model Training by Sharing Weights and Then Unsharing
- Adversarial Attacks on Deep Learning Models in Natural Language Processing: A Survey
- Single-Read Reconstruction for DNA Data Storage Using Transformers
- Pixel-BERT: Aligning Image Pixels with Text by Deep Multi-Modal Transformers
- Understanding and Improving Encoder Layer Fusion in Sequence-to-Sequence Learning
- Blackbox Dataset Inference for LLM
- OmniDraft: A Cross-vocabulary, Online Adaptive Drafter for On-device Speculative Decoding
- Lexically Cohesive Neural Machine Translation with Copy Mechanism
- On Characterizing the Capacity of Neural Networks using Algebraic Topology
- Bootstrapped Q-learning with Context Relevant Observation Pruning to Generalize in Text-based Games
- Automatic Post-Editing for Vietnamese
- A Sketch-Based Neural Model for Generating Commit Messages from Diffs
- UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training
- Understanding Transformers for Bot Detection in Twitter
- Self-Attentive Sequential Recommendation
- IndoBERTweet: A Pretrained Language Model for Indonesian Twitter with Effective Domain-Specific Vocabulary Initialization
- Robust Reading Comprehension with Linguistic Constraints via Posterior Regularization
- Transaction Categorization with Relational Deep Learning in QuickBooks
- Natural Question Generation with Reinforcement Learning Based Graph-to-Sequence Model
- Minimizing the Bag-of-Ngrams Difference for Non-Autoregressive Neural Machine Translation
- Subword Language Model for Query Auto-Completion
- The Detection of Distributional Discrepancy for Text Generation
- A Distributional Approach to Controlled Text Generation
- Understand customer reviews with less data and in short time: pretrained language representation and active learning
- Characterizing the Deep Neural Networks Inference Performance of Mobile Applications
- Hyperpruning: Efficient Search through Pruned Variants of Recurrent Neural Networks Leveraging Lyapunov Spectrum
- Assuring the Machine Learning Lifecycle: Desiderata, Methods, and Challenges
- Stay On-Topic: Generating Context-specific Fake Restaurant Reviews
- Enriching Non-Autoregressive Transformer with Syntactic and SemanticStructures for Neural Machine Translation
- Training Deeper Neural Machine Translation Models with Transparent Attention
- MergeDistill: Merging Pre-trained Language Models using Distillation
- OmniNet: A unified architecture for multi-modal multi-task learning
- Incorporating Domain Knowledge into Materials Tokenization
- Multi-lingual Functional Evaluation for Large Language Models
- Communicating Smartly in Molecular Communication Environments: Neural Networks in the Internet of Bio-Nano Things
- Learning Architectures from an Extended Search Space for Language Modeling
- Curvature Enhanced Data Augmentation for Regression
- Using Linguistic Typology to Enrich Multilingual Lexicons: the Case of Lexical Gaps in Kinship
- Compute Trends Across Three Eras of Machine Learning
- A cost-benefit analysis of cross-lingual transfer methods
- DRAGNN: A Transition-based Framework for Dynamically Connected Neural Networks
- LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding
- Terminal Brain Damage: Exposing the Graceless Degradation in Deep Neural Networks Under Hardware Fault Attacks
- An empirical analysis of phrase-based and neural machine translation
- Distributional Discrepancy: A Metric for Unconditional Text Generation
- Benchmarking TPU, GPU, and CPU Platforms for Deep Learning
- An Efficient Graph Convolutional Network Technique for the Travelling Salesman Problem
- A Deep Reinforcement Learning Chatbot (Short Version)
- ML Based Lineage in Databases
- Direct speech-to-speech translation with a sequence-to-sequence model
- Semi-Supervised Learning with Data Augmentation for End-to-End ASR
- Cross-lingual Pre-training Based Transfer for Zero-shot Neural Machine Translation
- Memory Slices: A Modular Building Block for Scalable, Intelligent Memory Systems
- Large Language Models -- the Future of Fundamental Physics?
- Mixed Precision Training With 8-bit Floating Point
- Attaining the Unattainable? Reassessing Claims of Human Parity in Neural Machine Translation
- Exploring Neural Transducers for End-to-End Speech Recognition
- The Mathematical Foundations of Manifold Learning
- A Convolutional Encoder Model for Neural Machine Translation
- Understanding LLM-Centric Challenges for Deep Learning Frameworks: An Empirical Analysis
- Subspace Approximation for Approximate Nearest Neighbor Search in NLP
- Aspect Term Extraction with History Attention and Selective Transformation
- Neural Machine Translation: A Review and Survey
- Hierarchical Transformer Network for Utterance-level Emotion Recognition
- Measuring prominence of scientific work in online news as a proxy for impact
- Investigation of BERT Model on Biomedical Relation Extraction Based on Revised Fine-tuning Mechanism
- Synchronous Bidirectional Inference for Neural Sequence Generation
- The Helsinki Neural Machine Translation System
- Utilizing Language Relatedness to improve Machine Translation: A Case Study on Languages of the Indian Subcontinent
- Global-aware Beam Search for Neural Abstractive Summarization
- Wat zei je? Detecting Out-of-Distribution Translations with Variational Transformers
- Compressing LSTM Networks by Matrix Product Operators
- K-TanH: Efficient TanH For Deep Learning
- The Impact of Preprocessing on Arabic-English Statistical and Neural Machine Translation
- LIMIT-BERT : Linguistic Informed Multi-Task BERT
- Automated proof synthesis for propositional logic with deep neural networks
- An Evaluation of Neural Machine Translation Models on Historical Spelling Normalization
- Improving the Performance of Online Neural Transducer Models
- A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Models
- From Zero to Hero: On the Limitations of Zero-Shot Cross-Lingual Transfer with Multilingual Transformers
- Regressive Ensemble for Machine Translation Quality Evaluation
- Universal Neural Machine Translation for Extremely Low Resource Languages
- Why Neural Machine Translation Prefers Empty Outputs
- Breaking the Beam Search Curse: A Study of (Re-)Scoring Methods and Stopping Criteria for Neural Machine Translation
- Searching for TrioNet: Combining Convolution with Local and Global Self-Attention
- On the quantization of recurrent neural networks
- Accelerated Reinforcement Learning for Sentence Generation by Vocabulary Prediction
- Sentence-wise Smooth Regularization for Sequence to Sequence Learning
- VIVO: Visual Vocabulary Pre-Training for Novel Object Captioning
- A Massively Multilingual Analysis of Cross-linguality in Shared Embedding Space
- Passport-aware Normalization for Deep Model Protection
- Detecting Hallucinated Content in Conditional Neural Sequence Generation
- Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation
- Approximate Distribution Matching for Sequence-to-Sequence Learning
- Multi-Task Learning and Adapted Knowledge Models for Emotion-Cause Extraction
- MANet: Multimodal Attention Network based Point- View fusion for 3D Shape Recognition
- Recent Trends in Deep Learning Based Natural Language Processing
- Learning from Learning Machines: Optimisation, Rules, and Social Norms
- One-shot and few-shot learning of word embeddings
- Artificial Intelligence : from Research to Application ; the Upper-Rhine Artificial Intelligence Symposium (UR-AI 2019)
- Auditing Data Provenance in Text-Generation Models
- Uniform-in-Time Weak Error Analysis for Stochastic Gradient Descent Algorithms via Diffusion Approximation
- RLTIR: Activity-based Interactive Person Identification based on Reinforcement Learning Tree
- DAPPLE: A Pipelined Data Parallel Approach for Training Large Models
- Language Tags Matter for Zero-Shot Neural Machine Translation
- A little goes a long way: Improving toxic language classification despite data scarcity
- Faster, Simpler and More Accurate Hybrid ASR Systems Using Wordpieces
- Reinforced Generative Adversarial Network for Abstractive Text Summarization
- Iterative Dual Domain Adaptation for Neural Machine Translation
- EmotionX-KU: BERT-Max based Contextual Emotion Classifier
- Rate Distortion For Model Compression: From Theory To Practice
- On the Dimensionality of Word Embedding
- On the Geometry of Adversarial Examples
- End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architectures
- Teaching Machines to Converse
- Backpropagation through time and the brain
- A systematic comparison of grapheme-based vs. phoneme-based label units for encoder-decoder-attention models
- Event Detection as Question Answering with Entity Information
- Demystifying the MLPerf Benchmark Suite
- On the Effectiveness of Mitigating Data Poisoning Attacks with Gradient Shaping
- Investigating Label Bias in Beam Search for Open-ended Text Generation
- End-to-end Speech Recognition with Word-based RNN Language Models
- Vector-Vector-Matrix Architecture: A Novel Hardware-Aware Framework for Low-Latency Inference in NLP Applications
- Towards Preemptive Detection of Depression and Anxiety in Twitter
- Adversarial Ranking for Language Generation
- Non-asymptotic Excess Risk Bounds for Classification with Deep Convolutional Neural Networks
- Speech-language Pre-training for End-to-end Spoken Language Understanding
- Just ASK: Building an Architecture for Extensible Self-Service Spoken Language Understanding
- Learning to Represent Bilingual Dictionaries
- Morphological and Language-Agnostic Word Segmentation for NMT
- TokAlign: Efficient Vocabulary Adaptation via Token Alignment
- A Semi-Supervised Approach for Low-Resourced Text Generation
- Steering Output Style and Topic in Neural Response Generation
- Adv-BERT: BERT is not robust on misspellings! Generating nature adversarial samples on BERT
- FastSeq: Make Sequence Generation Faster
- KR-BERT: A Small-Scale Korean-Specific Language Model
- Search Engine Guided Non-Parametric Neural Machine Translation
- Learning Light-Weight Translation Models from Deep Transformer
- Knowledge Enhanced Contextual Word Representations
- Learning to Generate Corrective Patches using Neural Machine Translation
- Self-supervised Latent Space Optimization with Nebula Variational Coding
- A Practical Guide to Studying Emergent Communication through Grounded Language Games
- An Encoder-Decoder Framework Translating Natural Language to Database Queries
- Paraphrase Generation with Deep Reinforcement Learning
- Learning to Write Notes in Electronic Health Records
- Memory-Efficient FastText: A Comprehensive Approach Using Double-Array Trie Structures and Mark-Compact Memory Management
- Neural Mention Detection
- Duluth at SemEval-2020 Task 7: Using Surprise as a Key to Unlock Humorous Headlines
- Learning to Reformulate the Queries on the WEB
- Retrieving Sequential Information for Non-Autoregressive Neural Machine Translation
- Why not be Versatile? Applications of the SGNMT Decoder for Machine Translation
- A Multilingual Modeling Method for Span-Extraction Reading Comprehension
- Investigating Backtranslation in Neural Machine Translation
- PMF-CEC: Phoneme-augmented Multimodal Fusion for Context-aware ASR Error Correction with Error-specific Selective Decoding
- Adversarial Subword Regularization for Robust Neural Machine Translation
- DICT-MLM: Improved Multilingual Pre-Training using Bilingual Dictionaries
- Architecture for a multilingual Wikipedia
- Adding A Filter Based on The Discriminator to Improve Unconditional Text Generation
- Evaluating Syntactic Properties of Seq2seq Output with a Broad Coverage HPSG: A Case Study on Machine Translation
- Learning to Represent Image and Text with Denotation Graph
- Improving Neural Machine Translation with Conditional Sequence Generative Adversarial Nets
- A Call for Prudent Choice of Subword Merge Operations in Neural Machine Translation
- Placeto: Learning Generalizable Device Placement Algorithms for Distributed Machine Learning
- Towards Fully Automated Manga Translation
- Question Type Guided Attention in Visual Question Answering
- Learning from Imperfect Annotations
- A crossover code for high-dimensional composition
- Non-Autoregressive Neural Machine Translation
- Sensor Transformation Attention Networks
- Time to Take Emoji Seriously: They Vastly Improve Casual Conversational Models
- Towards one-shot learning for rare-word translation with external experts
- Greedy Search with Probabilistic N-gram Matching for Neural Machine Translation
- In-training Matrix Factorization for Parameter-frugal Neural Machine Translation
- Streaming Object Detection for 3-D Point Clouds
- CapWAP: Captioning with a Purpose
- Token-level Adaptive Training for Neural Machine Translation
- Scaling Laws for Neural Machine Translation
- Who Reasons in the Large Language Models?
- GWLAN: General Word-Level AutocompletioN for Computer-Aided Translation
- Robust Deep Sensing Through Transfer Learning in Cognitive Radio
- Exploring Fluent Query Reformulations with Text-to-Text Transformers and Reinforcement Learning
- Identifying and Exploiting Structures for Reliable Deep Learning
- Fast Interleaved Bidirectional Sequence Generation
- Token-Level Ensemble Distillation for Grapheme-to-Phoneme Conversion
- Feedforward and Recurrent Neural Networks Backward Propagation and Hessian in Matrix Form
- Noisy Text Data: Achilles' Heel of popular transformer based NLP models
- Improving Biomedical Pretrained Language Models with Knowledge
- Pseudo Siamese Network for Few-shot Intent Generation
- Masked ELMo: An evolution of ELMo towards fully contextual RNN language models
- Joint Training for Neural Machine Translation Models with Monolingual Data
- Sample Efficient Adaptive Text-to-Speech
- LAnoBERT: System Log Anomaly Detection based on BERT Masked Language Model
- Evaluating Amharic Machine Translation
- SS-Auto: A Single-Shot, Automatic Structured Weight Pruning Framework of DNNs with Ultra-High Efficiency
- Towards Unsupervised Language Understanding and Generation by Joint Dual Learning
- Depth Growing for Neural Machine Translation
- Adversarial Self-Supervised Data-Free Distillation for Text Classification
- Upcycle Your OCR: Reusing OCRs for Post-OCR Text Correction in Romanised Sanskrit
- Style Obfuscation by Invariance
- Identity Crisis: Memorization and Generalization under Extreme Overparameterization
- Fast Sequence Generation with Multi-Agent Reinforcement Learning
- Neural Machine Translation with Adequacy-Oriented Learning
- Multimodal Joint Attribute Prediction and Value Extraction for E-commerce Product
- S2ORC: The Semantic Scholar Open Research Corpus
- Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges
- Latent Topic Conversational Models
- Machine Translation of Low-Resource Spoken Dialects: Strategies for Normalizing Swiss German
- Low-Resource Language Modelling of South African Languages
- To Beam Or Not To Beam: That is a Question of Cooperation for Language GANs
- Scalable Methods for 8-bit Training of Neural Networks
- Multi-branch Attentive Transformer
- SubCharacter Chinese-English Neural Machine Translation with Wubi encoding
- Reinforcement Learning Based Graph-to-Sequence Model for Natural Question Generation
- Cross-Modal Alignment with Mixture Experts Neural Network for Intral-City Retail Recommendation
- Text Analysis in Adversarial Settings: Does Deception Leave a Stylistic Trace?
- Self-Attentional Models Application in Task-Oriented Dialogue Generation Systems
- InfiFPO: Implicit Model Fusion via Preference Optimization in Large Language Models
- High-Performance Deep Learning via a Single Building Block
- A Feasible Framework for Arbitrary-Shaped Scene Text Recognition
- Dynamic Past and Future for Neural Machine Translation
- Learning Representations by Stochastic Meta-Gradient Descent in Neural Networks
- Grid Search Hyperparameter Benchmarking of BERT, ALBERT, and LongFormer on DuoRC
- Video Caption Dataset for Describing Human Actions in Japanese
- A Selective Overview of Deep Learning
- Synchronous Bidirectional Neural Machine Translation
- TBD: Benchmarking and Analyzing Deep Neural Network Training
- NameRec*: Highly Accurate and Fine-grained Person Name Recognition
- GDPRShield: AI-Powered GDPR Support for Software Developers in Small and Medium-Sized Enterprises
- Neural Language Generation: Formulation, Methods, and Evaluation
- Calculating Question Similarity is Enough: A New Method for KBQA Tasks
- Task-adaptive Pre-training of Language Models with Word Embedding Regularization
- Assessing Demographic Bias in Named Entity Recognition
- Residual Error: a New Performance Measure for Adversarial Robustness
- Algorithm to Compilation Co-design: An Integrated View of Neural Network Sparsity
- CellCLIP -- Learning Perturbation Effects in Cell Painting via Text-Guided Contrastive Learning
- Unsupervised Neural Machine Translation with SMT as Posterior Regularization
- Sequence-Level Training for Non-Autoregressive Neural Machine Translation
- Automatic Post-Editing for Machine Translation
- StructBERT: Incorporating Language Structures into Pre-training for Deep Language Understanding
- Low-Shot Classification: A Comparison of Classical and Deep Transfer Machine Learning Approaches
- An Empirical Study of Efficient ASR Rescoring with Transformers
- Cochain: Balancing Insufficient and Excessive Collaboration in LLM Agent Workflows
- Data-Free Knowledge Distillation with Soft Targeted Transfer Set Synthesis
- BERT-based Chinese Text Classification for Emergency Domain with a Novel Loss Function
- Is Neural Machine Translation Ready for Deployment? A Case Study on 30 Translation Directions
- TAB-PO: Preference Optimization with a Token-Level Adaptive Barrier for Token-Critical Structured Generation
- Why Literary Translators should embrace Translation Technology
- The Riddle of (Literary) Machine Translation Quality
- Reward Optimization for Neural Machine Translation with Learned Metrics
- Exploring the importance of context and embeddings in neural NER models for task-oriented dialogue systems
- Modeling Global and Local Node Contexts for Text Generation from Knowledge Graphs
- Breaking the Memory Wall for AI Chip with a New Dimension
- Phases of learning dynamics in artificial neural networks: with or without mislabeled data
- Capsule-Transformer for Neural Machine Translation
- Multilingual Machine Translation with Quantum Encoder Decoder Attention-based Convolutional Variational Circuits
- Style Pooling: Automatic Text Style Obfuscation for Improved Classification Fairness
- Bi-Decoder Augmented Network for Neural Machine Translation
- GLOW : Global Weighted Self-Attention Network for Web Search
- Importance-Aware Learning for Neural Headline Editing
- Vanishing Nodes: Another Phenomenon That Makes Training Deep Neural Networks Difficult
- Deep Independently Recurrent Neural Network (IndRNN)
- Stronger Transformers for Neural Multi-Hop Question Generation
- Adversarial Neural Machine Translation
- TiSpell: A Semi-Masked Methodology for Tibetan Spelling Correction covering Multi-Level Error with Data Augmentation
- Scale MLPerf-0.6 models on Google TPU-v3 Pods
- Data Augmentation for Abstractive Query-Focused Multi-Document Summarization
- Shareable Representations for Search Query Understanding
- Sequential Modelling with Applications to Music Recommendation, Fact-Checking, and Speed Reading
- CaMDN: Enhancing Cache Efficiency for Multi-tenant DNNs on Integrated NPUs
- Sharing Attention Weights for Fast Transformer
- Multi-object Tracking via End-to-end Tracklet Searching and Ranking
- Leveraging Grammar and Reinforcement Learning for Neural Program Synthesis
- Phrase-Based Attentions
- Deploying a BERT-based Query-Title Relevance Classifier in a Production System: a View from the Trenches
- Hard but Robust, Easy but Sensitive: How Encoder and Decoder Perform in Neural Machine Translation
- QiMeng-Xpiler: Transcompiling Tensor Programs for Deep Learning Systems with a Neural-Symbolic Approach
- AI agents may be worth the hype but not the resources (yet): An initial exploration of machine translation quality and costs in three language pairs in the legal and news domains
- Machine Translation in Pronunciation Space
- Enhancing Machine Translation with Dependency-Aware Self-Attention
- Deep Reinforcement Learning: An Overview
- Calibration of Encoder Decoder Models for Neural Machine Translation
- Token-free Models for Sarcasm Detection
- Guiding Teacher Forcing with Seer Forcing for Neural Machine Translation
- VIOLET : End-to-End Video-Language Transformers with Masked Visual-token Modeling
- Training LSTM Networks With Resistive Cross-Point Devices
- E2E-VLP: End-to-End Vision-Language Pre-training Enhanced by Visual Learning
- Neutron: An Implementation of the Transformer Translation Model and its Variants
- Can the Transformer Be Used as a Drop-in Replacement for RNNs in Text-Generating GANs?
- Multilingual Word Embeddings using Multigraphs
- Bayesian Sparsification of Recurrent Neural Networks
- Learning to Copy for Automatic Post-Editing
- When does pretraining help?
- PERL: Pivot-based Domain Adaptation for Pre-trained Deep Contextualized Embedding Models
- Unsupervised Multi-modal Neural Machine Translation
- Reweighted Proximal Pruning for Large-Scale Language Representation
- Extracting UMLS Concepts from Medical Text Using General and Domain-Specific Deep Learning Models
- Conversational Response Re-ranking Based on Event Causality and Role Factored Tensor Event Embedding
- PolyScientist: Automatic Loop Transformations Combined with Microkernels for Optimization of Deep Learning Primitives
- Simplifying Sentences with Sequence to Sequence Models
- Plumber: Diagnosing and Removing Performance Bottlenecks in Machine Learning Data Pipelines
- Cutting-off Redundant Repeating Generations for Neural Abstractive Summarization
- Adversarial Learning for Chinese NER from Crowd Annotations
- Understanding Learning Dynamics for Neural Machine Translation
- Alleviating the Burden of Labeling: Sentence Generation by Attention Branch Encoder-Decoder Network
- M6: A Chinese Multimodal Pretrainer
- Neural information retrieval: at the end of the early years
- Low-Precision Batch-Normalized Activations
- Towards Fully Bilingual Deep Language Modeling
- Translators as Invisible Teachers of AI: Copyright, Translation Memory, and the Political Economy of Linguistic Data
- AI and the Research-Education Environment of Physics
- CLaS-Bench: A Cross-Lingual Alignment and Steering Benchmark
- Spatial Speech Translation: Translating Across Space With Binaural Hearables
- Simplified End-to-End MMI Training and Voting for ASR
- Simplify-then-Translate: Automatic Preprocessing for Black-Box Machine Translation
- A review on the long short-term memory model
- Accelerating Large Scale Knowledge Distillation via Dynamic Importance Sampling
- Neural Symbolic Machines: Learning Semantic Parsers on Freebase with Weak Supervision
- Physics-informed linear regression is competitive with two Machine Learning methods in residential building MPC
- Beam Search with Bidirectional Strategies for Neural Response Generation
- Image to Language Understanding: Captioning approach
- Sketching Transformed Matrices with Applications to Natural Language Processing
- Match-Tensor: a Deep Relevance Model for Search
- On the Diversity of Analogy Making in Large Language Models
- Word, Subword or Character? An Empirical Study of Granularity in Chinese-English NMT
- Reference Network for Neural Machine Translation
- Consensus Measures for Unstructured Biomedical Text Annotations
- DyTTP: Trajectory Prediction with Normalization-Free Transformers
- EVIL: Exploiting Software via Natural Language
- Synthetic and Natural Noise Both Break Neural Machine Translation
- Tree of Thoughts as a Classical Heuristic Search Problem: Formal Foundations and Design Patterns
- Text2GraphQuery-Bench: A Text to Graph Query Benchmark
- Pre-Training BERT on Arabic Tweets: Practical Considerations
- PhenoTagger: a hybrid method for phenotype concept recognition using human phenotype ontology
- Multi-Head Multi-Layer Attention to Deep Language Representations for Grammatical Error Detection
- End-to-End Speech Translation with Knowledge Distillation
- A Hybrid Convolutional Variational Autoencoder for Text Generation
- Sequence-to-Sequence Models Can Directly Translate Foreign Speech
- Neural Persistence: A Complexity Measure for Deep Neural Networks Using Algebraic Topology
- Neural Rating Regression with Abstractive Tips Generation for Recommendation
- Automated Architecture Design for Deep Neural Networks
- Multi-Modal Detection of Alzheimer's Disease from Speech and Text
- Modeling Source Syntax for Neural Machine Translation
- Backretrieval: An Image-Pivoted Evaluation Metric for Cross-Lingual Text Representations Without Parallel Corpora
- ReWE: Regressing Word Embeddings for Regularization of Neural Machine Translation Systems
- What do Neural Machine Translation Models Learn about Morphology?
- Recommending Themes for Ad Creative Design via Visual-Linguistic Representations
- Mitigating Edge Machine Learning Inference Bottlenecks: An Empirical Study on Accelerating Google Edge Models
- Improving the Serving Performance of Multi-LoRA Large Language Models via Efficient LoRA and KV Cache Management
- Attention-Based End-to-End Speech Recognition on Voice Search
- SYSTRAN Purely Neural MT Engines for WMT2017
- TTPP: Temporal Transformer with Progressive Prediction for Efficient Action Anticipation
- Aligning the Pretraining and Finetuning Objectives of Language Models
- Gmst:An Unbiased Stratified Statistic and a Fast Gradient Optimization Algorithm Based on It
- Deep Ensembles on a Fixed Memory Budget: One Wide Network or Several Thinner Ones?
- Large-Scale Discrete Fourier Transform on TPUs
- On Multilingual Training of Neural Dependency Parsers
- Local minima in training of neural networks
- Tell Me Why You Feel That Way: Processing Compositional Dependency for Tree-LSTM Aspect Sentiment Triplet Extraction (TASTE)
- Neural Machine Translation: A Review of Methods, Resources, and Tools
- Deep Learning for Scene Classification: A Survey
- A Survey of Orthographic Information in Machine Translation
- Weak Supervision helps Emergence of Word-Object Alignment and improves Vision-Language Tasks
- DeepTrans: Deep Reasoning Translation via Reinforcement Learning
- The Quantum LLM: Modeling Semantic Spaces with Quantum Principles
- Alleviating the Fear of Losing Alignment in LLM Fine-tuning
- Statistically guided deep learning
- On the rate of convergence of an over-parametrized deep neural network regression estimate learned by gradient descent
- Google Neural Machine Translation [wikipedia]
- Google Translate [wikipedia]
- Greg Corrado [wikipedia]
- Long short-term memory [wikipedia]
- Attention Is All You Need [wikipedia]
- Neural machine translation [wikipedia]
- Seq2seq [wikipedia]
- Transformer (deep learning) [wikipedia]
- Mixture of experts [wikipedia]
- Deep learning [wikipedia]
Discussions
Related