GLIMMER: generalized late-interaction memory reranker

2023/06/17 by Michiel de Jong, Yury Zemlyanskiy, de Jong, Michiel +9 · 2 citations
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Natural Language Processing Techniques #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2306.10231

openalex publication_date 2023/06/17 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Memory-augmentation is a powerful approach for efficiently incorporating external information into language models, but leads to reduced performance relative to retrieving text. Recent work introduced LUMEN, a memory-retrieval hybrid that partially pre-computes memory and updates memory representations on the fly with a smaller live encoder. We propose GLIMMER, which improves on this approach through 1) exploiting free access to the powerful memory representations by applying a shallow reranker on top of memory to drastically improve retrieval quality at low cost, and 2) incorporating multi-task training to learn a general and higher quality memory and live encoder. GLIMMER achieves strong gains in performance at faster speeds compared to LUMEN and FiD on the KILT benchmark of knowledge-intensive tasks.

Cited by

• Learning What to Remember: Adaptive Probabilistic Memory Retention for Memory-Efficient Language Models
• Beyond RAG vs. Long-Context: Learning Distraction-Aware Retrieval for Efficient Knowledge Grounding

Related

• Pre-computed memory or on-the-fly encoding? A hybrid approach to retrieval augmentation makes the most of your compute
• LightMem: Lightweight and Efficient Memory-Augmented Generation
• Robust Implementation of Retrieval-Augmented Generation on Edge-based Computing-in-Memory Architectures
• MEMORY-VQ: Compression for Tractable Internet-Scale Memory
• Memory Decoder: A Pretrained, Plug-and-Play Memory for Large Language Models