LogiCoL: Logically-Informed Contrastive Learning for Set-based Dense Retrieval

2025/05/26 by Yanzhen Shen, Shen, Yanzhen, Sihao Chen +9
Computer Science · #Advanced Image and Video Retrieval Techniques #Artificial Intelligence (cs.AI) #Conjunction (astronomy) #Consistency (knowledge bases) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Image Retrieval and Classification Techniques #Information Retrieval (cs.IR) #Logical consequence #Relation (database) #Set (abstract data type) #Task (project management) #Textual entailment #Training set

paper · pdf · doi:10.48550/arxiv.2505.19588

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2025/05/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Abstract

While significant progress has been made with dual- and bi-encoder dense retrievers, they often struggle on queries with logical connectives, a use case that is often overlooked yet important in downstream applications. Current dense retrievers struggle with such queries, such that the retrieved results do not respect the logical constraints implied in the queries. To address this challenge, we introduce LogiCoL, a logically-informed contrastive learning objective for dense retrievers. LogiCoL builds upon in-batch supervised contrastive learning, and learns dense retrievers to respect the subset and mutually-exclusive set relation between query results via two sets of soft constraints expressed via t-norm in the learning objective. We evaluate the effectiveness of LogiCoL on the task of entity retrieval, where the model is expected to retrieve a set of entities in Wikipedia that satisfy the implicit logical constraints in the query. We show that models trained with LogiCoL yield improvement both in terms of retrieval performance and logical consistency in the results. We provide detailed analysis and insights to uncover why queries with logical connectives are challenging for dense retrievers and why LogiCoL is most effective.

Citations

• Constructing Set-Compositional and Negated Representations for First-Stage Ranking
• SetBERT: Enhancing Retrieval Performance for Boolean Logic and Set Operation Queries
• MoDE: CLIP Data Experts via Clustering
• Chain-of-Factors Paper-Reviewer Matching
• Towards General Text Embeddings with Multi-stage Contrastive Learning
• QUEST: A Retrieval Dataset of Entity-Seeking Queries with Implicit Set Operations
• NevIR: Negation in Neural Information Retrieval
• Text Embeddings by Weakly-Supervised Contrastive Pre-training
• Global Contrastive Batch Sampling via Optimization on Sample Permutations
• Unsupervised Dense Information Retrieval with Contrastive Learning
• Large Dual Encoders Are Generalizable Retrievers
• SimCSE: Simple Contrastive Learning of Sentence Embeddings
• SimCSE: Simple Contrastive Learning of Sentence Embeddings
• Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Sampling
• Joint Constrained Learning for Event-Event Relation Extraction
• Contrastive Learning with Hard Negative Samples
• KILT: a Benchmark for Knowledge Intensive Language Tasks
• Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval
• Dense Passage Retrieval for Open-Domain Question Answering
• A Logic-Driven Framework for Consistency of Neural Models
• Augmenting Neural Networks with First-order Logic
• Representation Learning with Contrastive Predictive Coding
• Reading Wikipedia to Answer Open-Domain Questions
• MS MARCO: A Human Generated MAchine Reading COmprehension Dataset
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Related

• Conventional Contrastive Learning Often Falls Short: Improving Dense Retrieval with Cross-Encoder Listwise Distillation and Synthetic Data
• Enhancing Retrieval Systems with Inference-Time Logical Reasoning
• LED: Lexicon-Enlightened Dense Retriever for Large-Scale Retrieval
• Teaching Dense Retrieval Models to Specialize with Listwise Distillation and LLM Data Augmentation
• How to Train Your DRAGON: Diverse Augmentation Towards Generalizable Dense Retrieval