Syntactically Informed Text Compression with Recurrent Neural Networks

2016/08/08 by Cox, David · 1 citation
#Computation and Language (cs.CL) #E.4 #FOS: Computer and information sciences #Information Theory (cs.IT) #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.1608.02893

Abstract

We present a self-contained system for constructing natural language models for use in text compression. Our system improves upon previous neural network based models by utilizing recent advances in syntactic parsing -- Google's SyntaxNet -- to augment character-level recurrent neural networks. RNNs have proven exceptional in modeling sequence data such as text, as their architecture allows for modeling of long-term contextual information.

Cited by

• Test-Time Steering for Lossless Text Compression via Weighted Product of Experts

Related

• Learning Directly from Grammar Compressed Text
• Compression of Deep Learning Models for Text: A Survey
• From Imitation to Prediction, Data Compression vs Recurrent Neural\n Networks for Natural Language Processing
• Character n-gram Embeddings to Improve RNN Language Models
• A Token-wise CNN-based Method for Sentence Compression