Auto prompting without training labels: An LLM cascade for product quality assessment in e-commerce catalogs

2025/10/27 by Satyadharma, Soham, Sheikholeslami, Fatemeh, Kaul, Swati +2
#Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2510.23941

Abstract

We introduce a novel, training free cascade for auto-prompting Large Language Models (LLMs) to assess product quality in e-commerce. Our system requires no training labels or model fine-tuning, instead automatically generating and refining prompts for evaluating attribute quality across tens of thousands of product category-attribute pairs. Starting from a seed of human-crafted prompts, the cascade progressively optimizes instructions to meet catalog-specific requirements. This approach bridges the gap between general language understanding and domain-specific knowledge at scale in complex industrial catalogs. Our extensive empirical evaluations shows the auto-prompt cascade improves precision and recall by 8-10% over traditional chain-of-thought prompting. Notably, it achieves these gains while reducing domain expert effort from 5.1 hours to 3 minutes per attribute - a 99% reduction. Additionally, the cascade generalizes effectively across five languages and multiple quality assessment tasks, consistently maintaining performance gains.

Citations

• A Sequential Optimal Learning Approach to Automated Prompt Engineering in Large Language Models
• Learning variant product relationship and variation attributes from e-commerce website structures
• Fine-Tuning and Prompt Optimization: Two Great Steps that Work Better Together
• ARES: Alternating Reinforcement Learning and Supervised Fine-Tuning for Enhanced Multi-Modal Chain-of-Thought Reasoning Through Diverse AI Feedback
• Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programs
• Leveraging Large Language Models for Enhanced NLP Task Performance through Knowledge Distillation and Optimized Training Strategies
• PRewrite: Prompt Rewriting with Reinforcement Learning
• Chain of Thought with Explicit Evidence Reasoning for Few-shot Relation Extraction
• Large Language Models for Generative Recommendation: A Survey and Visionary Discussions
• Knowledge Graph Completion Models are Few-shot Learners: An Empirical Study of Relation Labeling in E-commerce with LLMs
• Large Language Models Are Human-Level Prompt Engineers
• RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning
• Large Language Models are Zero-Shot Reasoners
• Self-Consistency Improves Chain of Thought Reasoning in Language Models
• Automatic Validation of Textual Attribute Values in E-commerce Catalog by Learning with Limited Labeled Data
• Language Models are Few-Shot Learners
• EvoPrompt: Connecting LLMs with Evolutionary Algorithms Yields Powerful Prompt Optimizers

Related

• Beyond Exact Match: How Evaluation Methodology Dominates Model Choice in LLM-Based Product Attribute Extraction
• Examples as the Prompt: A Scalable Approach for Efficient LLM Adaptation in E-Commerce
• LLM-Ensemble: Optimal Large Language Model Ensemble Method for E-commerce Product Attribute Value Extraction
• Leveraging Large Language Models for Enhanced Product Descriptions in eCommerce
• PRAISE: Enhancing Product Descriptions with LLM-Driven Structured Insights