Scalable Machines with Intrinsic Higher Mental-State Dynamics

2026/03/13 by Ahsan Adeel, M. Bilal · 1 voice
Computer Science · Engineering · Neuroscience · #Advanced Memory and Neural Computing #Ferroelectric and Negative Capacitance Devices #Neural dynamics and brain function #cs.AI #cs.LG

paper · pdf · doi:10.48550/arxiv.2603.13453

openalex publication_date 2026/03/13 · arxiv published 2026/03/13 · arxiv updated 2026/03/13 · openalex created_date 2026/03/18 · openalex updated_date 2026/07/28

Abstract

Drawing on recent breakthroughs in cellular neurobiology and detailed biophysical modeling linking neocortical pyramidal neurons to distinct mental-state regimes, this work introduces a mathematically grounded formulation showing how models (e.g., Transformers) can implement computational principles underlying awake imaginative thought to pre-select relevant information before attention is applied via triadic modulation loops among queries (Q), keys (K), and values (V).~Scalability experiments on ImageNet-1K, benchmarked against a standard Vision Transformer (ViT), demonstrate significantly faster learning with reduced computational demand (fewer heads, layers, and tokens), consistent with our prior findings in reinforcement learning and language modeling. The approach operates at approximately O(N) complexity with respect to the number of input tokens N.

Citations

• Beyond Attention: Toward Machines with Intrinsic Higher Mental States
• An Overlooked Role of Context-Sensitive Dendrites
• VMamba: Visual State Space Model
• DeepSeek LLM: Scaling Open-Source Language Models with Longtermism
• Two-compartment neuronal spiking model expressing brain-state specific apical-amplification, -isolation and -drive regimes
• Leveraging dendritic properties to advance machine learning and neuro-inspired computing
• Cooperation Is All You Need
• Unlocking the potential of two-point cells for energy-efficient and resilient training of deep nets
• Single-phase deep learning in cortico-cortical networks
• BNAI, NO-TOKEN, and MIND-UNITY: Pillars of a Systemic Revolution in Artificial Intelligence
• The Sensory Neuron as a Transformer: Permutation-Invariant Neural Networks for Reinforcement Learning
• Training data-efficient image transformers & distillation through attention
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Local plasticity rules can learn deep representations using self-supervised contrastive predictions
• Cellular Mechanisms of Conscious Processing
• Dendritic cortical microcircuits approximate the backpropagation\n algorithm
• Attention Is All You Need
• Towards deep learning with segregated dendrites
• Active Properties of Neocortical Pyramidal Neuron Dendrites
• A cellular mechanism for cortical associations: an organizing principle for the cerebral cortex
• Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
• Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

Discussions

• Important stuff. Meanwhile, Ahsan Adeel and his team at Stirling are actually *buidling* bio-inspired, energy efficient LLMs but, sadly, no one is paying attention to them (yet). ieeexplore.ieee.org/ [bsky, 2 points, 0 comments]