2018/07/23 by Mark Collier, Joeran Beel, Collier, Mark +1 · 1 voice · 9 citations
Computer Science · Engineering · Mathematics · #Algorithm #Artificial intelligence #Artificial neural network #Auxiliary memory #Class (philosophy) #Computation #Computer hardware #Computer science #Domain Adaptation and Few-Shot Learning #Ferroelectric and Negative Capacitance Devices #Initialization #Machine Learning and Algorithms #Machine learning #Programming language #Scheme (mathematics) #Turing #Turing machine #cs.LG #stat.ML
paper · pdf · doi:10.48550/arxiv.1807.08518
published in arXiv (Cornell University) (Cornell University)
openalex publication_date 2018/07/23 · arxiv created 2018/07/26 · arxiv updated 2018/08/21 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05
Neural Turing Machines (NTMs) are an instance of Memory Augmented Neural Networks, a new class of recurrent neural networks which decouple computation from memory by introducing an external memory unit. NTMs have demonstrated superior performance over Long Short-Term Memory Cells in several sequence learning tasks. A number of open source implementations of NTMs exist but are unstable during training and/or fail to replicate the reported performance of NTMs. This paper presents the details of our successful implementation of a NTM. Our implementation learns to solve three sequential learning tasks from the original NTM paper. We find that the choice of memory contents initialization scheme is crucial in successfully implementing a NTM. Networks with memory contents initialized to small constant values converge on average 2 times faster than the next best memory contents initialization scheme.