MoA-Structured Decode Attention DNF Derivation, KV-Cache Accumulation, GQA/MQA, and OpenACC Kernel

2026/07/21 by Lenore Mulin, Gaetan Hains
#cs.LG #cs.AI

paper · pdf

Abstract

We derive four memory-optimal inference artifacts for transformer attention using the Mathematics of Arrays (MoA), each following directly from the forward-pass Denotational Normal Form (DNF) of with the query-row index fixed to the current decode step. The artifacts are: (1)~a single-query decode DNF in which the ψ-reduction eliminates the K^\top buffer algebraically, achieving (d_k + nd_k+ nd_v+ d_v)×4 B Dynamic Random Access Memory (DRAM) traffic result numerically verified to ‖err‖_≤2×10^-7; (2)~a C/OpenACC Graphics Processing Unit (GPU) kernel with Operational Normal Form (ONF) stride arithmetic and hardware-coalesced memory access, verified to ‖err‖_∞=0 (exact IEEE-754 floating-point arithmetic); (3)~a multi-step KV-cache with O(d_k+d_v) per-step append via MoA concatenation #; and (4)~Grouped-Query Attention (GQA) and Multi-Query Attention (MQA) derived via ψ-selection, achieving a proven \frac h_q h_kv reduction in KV traffic. All programs are verified against PyTorch scaled_dot_product_attention.

Citations

• FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
• Attention Is All You Need

Related

• Compressed Convolutional Attention: Efficient Attention in a Compressed Latent Space
• Reducing Transformer Key-Value Cache Size with Cross-Layer Attention
• GQLA: Group-Query Latent Attention for Hardware-Adaptive Large Language Model Decoding
• Fast Transformer Decoding: One Write-Head is All You Need
• GTA: Grouped-head latenT Attention