Properties of Sparse Distributed Representations and their Application to Hierarchical Temporal Memory

2015/03/25 by Subutai Ahmad, Jeff Hawkins, Ahmad, Subutai +1 · 2 voices · 1 citation
Biochemistry, Genetics and Molecular Biology · Computer Science · Engineering · Neuroscience · #Advanced Memory and Neural Computing #Artificial Intelligence (cs.AI) #FOS: Biological sciences #FOS: Computer and information sciences #Gene Regulatory Network Analysis #Neural Networks and Applications #Neural dynamics and brain function #Neurons and Cognition (q-bio.NC) #cs.AI #q-bio.NC

paper · pdf · doi:10.48550/arxiv.1503.07469

openalex publication_date 2015/03/25 · arxiv published 2015/03/25 · arxiv updated 2015/03/25 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Empirical evidence demonstrates that every region of the neocortex represents information using sparse activity patterns. This paper examines Sparse Distributed Representations (SDRs), the primary information representation strategy in Hierarchical Temporal Memory (HTM) systems and the neocortex. We derive a number of properties that are core to scaling, robustness, and generalization. We use the theory to provide practical guidelines and illustrate the power of SDRs as the basis of HTM. Our goal is to help create a unified mathematical and practical framework for SDRs as it relates to cortical function.

Cited by

• Deviant Learning Algorithm: Learning Sparse Mismatch Representations through Time and Space

Discussions

• Numenta published their first paper [hn, 3 points, 0 comments]
• Properties of Sparse Distributed Representations and Their Application to HTM [hn, 2 points, 0 comments]

Related

• Encoding Data for HTM Systems
• Toward a Universal Cortical Algorithm: Examining Hierarchical Temporal Memory in Light of Frontal Cortical Function
• How do neurons operate on sparse distributed representations? A mathematical theory of sparsity, neurons and active dendrites
• Encoding Reality: Prediction-Assisted Cortical Learning Algorithm in Hierarchical Temporal Memory
• A Mathematical Formalization of Hierarchical Temporal Memory's Spatial Pooler