2017/08/01 by Jan A. Botha, Botha, Jan A., Emily Pitler +13 · 1 voice
#cs.CL #cs.NE
paper · pdf · doi:10.48550/arxiv.1708.00214
We show that small and shallow feed-forward neural networks can achieve near state-of-the-art results on a range of unstructured and structured language processing tasks while being considerably cheaper in memory and computational requirements than deep recurrent models. Motivated by resource-constrained environments like mobile phones, we showcase simple techniques for obtaining such small neural network models, and investigate different tradeoffs when deciding how to allocate a small memory budget.