Semantic Search At LinkedIn

2026/02/07 by Fedor Borisyuk, Sriram Vasudevan, Muchen Wu +71 · 1 voice
Computer Science · #cs.AI #cs.IR #cs.LG

paper · pdf · doi:10.48550/arxiv.2602.07309

Abstract

Semantic search with large language models (LLMs) enables retrieval by meaning rather than keyword overlap, but scaling it requires major inference efficiency advances. We present LinkedIn's LLM-based semantic search framework for AI Job Search and AI People Search, combining an LLM relevance judge, embedding-based retrieval, and a compact Small Language Model trained via multi-teacher distillation to jointly optimize relevance and engagement. A prefill-oriented inference architecture co-designed with model pruning, context compression, and text-embedding hybrid interactions boosts ranking throughput by over 75x under a fixed latency constraint while preserving near-teacher-level NDCG, enabling one of the first production LLM-based ranking systems with efficiency comparable to traditional approaches and delivering significant gains in quality and user engagement.

Citations

• Scaling Up Efficient Small Language Models Serving and Deployment for Semantic Job Search
• From Reasoning LLMs to BERT: A Two-Stage Distillation Framework for Search Relevance
• Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models
• Understanding R1-Zero-Like Training: A Critical Perspective
• Liger Kernel: Efficient Triton Kernels for LLM Training
• HLLM: Enhancing Sequential Recommendations via Hierarchical Large Language Models for Item and User Modeling
• jina-embeddings-v3: Multilingual Embeddings With Task LoRA
• NV-Retriever: Improving text embedding models with effective hard-negative mining
• LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders
• OSSCAR: One-Shot Structured Pruning in Vision and Language Models with Combinatorial Optimization
• A Two-Stage Adaptation of Large Language Models for Text Ranking
• ReLLa: Retrieval-enhanced Large Language Models for Lifelong Sequential Behavior Comprehension in Recommendation
• Could Small Language Models Serve as Recommenders? Towards Data-centric Cold-start Recommendations
• A Survey on Large Language Models for Recommendation
• Large Language Models are Zero-Shot Rankers for Recommender Systems
• Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillation
• Understanding BERT Rankers Under Distillation
• On Calibration of Modern Neural Networks
• Optimal Auction Design

Discussions

• #cs.IR #cs.AI #cs.LG Origin | Interest | Match [bsky, 0 points, 0 comments]

Related

• Scaling Up Efficient Small Language Models Serving and Deployment for Semantic Job Search
• Large Scale Retrieval for the LinkedIn Feed using Causal Language Models
• Deep Natural Language Processing for LinkedIn Search
• MixLM: High-Throughput and Effective LLM Ranking via Text-Embedding Mix-Interaction
• Enhancing Cloud-Based Large Language Model Processing with Elasticsearch and Transformer Models