Zero-Shot Recommendations with Pre-Trained Large Language Models for Multimodal Nudging

2023/09/02 by Rachel Harrison, Harrison, Rachel M., Anton Dereventsov +3 · 2 citations
Computer Science · Social Sciences · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Computational and Text Analysis Methods #FOS: Computer and information sciences #Information Retrieval (cs.IR) #Machine Learning (cs.LG) #Multimedia (cs.MM) #Multimodal Machine Learning Applications #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2309.01026

openalex publication_date 2023/09/02 · openalex created_date 2023/09/08 · openalex updated_date 2026/07/28

Abstract

We present a method for zero-shot recommendation of multimodal non-stationary content that leverages recent advancements in the field of generative AI. We propose rendering inputs of different modalities as textual descriptions and to utilize pre-trained LLMs to obtain their numerical representations by computing semantic embeddings. Once unified representations of all content items are obtained, the recommendation can be performed by computing an appropriate similarity metric between them without any additional learning. We demonstrate our approach on a synthetic multimodal nudging environment, where the inputs consist of tabular, textual, and visual data.

Cited by

• A Survey on Generative Recommendation: Data, Model, and Tasks
• Driver Assistant: Persuading Drivers to Adjust Secondary Tasks Using Large Language Models

Related

• Zero-Shot Recommendation as Language Modeling
• Zero-Shot Next-Item Recommendation using Large Pretrained Language Models
• Multimodal Knowledge Alignment with Reinforcement Learning
• Multimodal Large Language Models with Adaptive Preference Optimization for Sequential Recommendation
• Train Once, Deploy Anywhere: Matryoshka Representation Learning for Multimodal Recommendation