Uplifting Lower-Income Data: Strategies for Socioeconomic Perspective Shifts in Large Multi-modal Models

2024/07/02 by Joan Nwatu, Oana Ignat, Nwatu, Joan +3
Social Sciences · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Computer Vision and Pattern Recognition (cs.CV) #Computers and Society (cs.CY) #E-Government and Public Services #FOS: Computer and information sciences #I.2.7 #I.2.8 #K.4

paper · pdf · doi:10.48550/arxiv.2407.02623

openalex publication_date 2024/07/02 · openalex created_date 2024/07/06 · openalex updated_date 2026/07/28

Abstract

Recent work has demonstrated that the unequal representation of cultures and socioeconomic groups in training data leads to biased Large Multi-modal (LMM) models. To improve LMM model performance on underrepresented data, we propose and evaluate several prompting strategies using non-English, geographic, and socioeconomic attributes. We show that these geographic and socioeconomic integrated prompts favor retrieving topic appearances commonly found in data from low-income households across different countries leading to improved LMM model performance on lower-income data. Our analyses identify and highlight contexts where these strategies yield the most improvements.

Related

• GeoReg: Weight-Constrained Few-Shot Regression for Socio-Economic Estimation using LLM
• Fairness in LLM-Generated Surveys
• Pre-Trained Language Models Represent Some Geographic Populations Better Than Others
• Understanding Intrinsic Socioeconomic Biases in Large Language Models
• Beyond Demographics: Fine-tuning Large Language Models to Predict Individuals' Subjective Text Perceptions