2025/02/20 by Zhichao Xu, Fengran Mo, Xu, Zhichao +13 · 1 voice · 14 citations
Computer Science · #Advanced Database Systems and Queries #FOS: Computer and information sciences #Information Retrieval (cs.IR) #Semantic Web and Ontologies #cs.IR
paper · pdf · doi:10.48550/arxiv.2502.14822
openalex publication_date 2025/02/20 · arxiv published 2025/02/20 · openalex created_date 2025/10/10 · arxiv updated 2026/03/15 · openalex updated_date 2026/07/28
The period from 2019 to the present marks one of the most significant paradigm shifts in information retrieval (IR) and natural language processing (NLP), culminating in the emergence of powerful large language models (LLMs) from 2022 onward. Methods based on pretrained encoder-only architectures (e.g., BERT) as well as decoder-only generative LLMs have outperformed many earlier approaches, demonstrating particularly strong performance in zero-shot scenarios and complex reasoning tasks. This survey examines the evolution of model architectures in IR, with a focus on two key aspects: backbone models for feature extraction and end-to-end system architectures for relevance estimation. To maintain analytical clarity, we deliberately separate architectural design from training methodologies, enabling a focused examination of structural innovations in IR systems. We trace the progression from traditional term-based retrieval models to modern neural approaches, highlighting the transformative impact of transformer-based architectures and subsequent LLM developments. The survey concludes with a forward-looking discussion of open challenges and emerging research directions, including architectural optimization for efficiency and scalability, robust handling of multimodal and multilingual data, and adaptation to novel application domains such as autonomous search agents, which may represent the next paradigm in IR.