Mass-Producing Failures of Multimodal Systems with Language Models

2023/06/21 by Shengbang Tong, Erik Jones, Tong, Shengbang +3 · 4 citations
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Software Engineering (cs.SE) #Software Engineering Research #Software Engineering Techniques and Practices #Software System Performance and Reliability

paper · pdf · doi:10.48550/arxiv.2306.12105

openalex publication_date 2023/06/21 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Deployed multimodal systems can fail in ways that evaluators did not anticipate. In order to find these failures before deployment, we introduce MultiMon, a system that automatically identifies systematic failures -- generalizable, natural-language descriptions of patterns of model failures. To uncover systematic failures, MultiMon scrapes a corpus for examples of erroneous agreement: inputs that produce the same output, but should not. It then prompts a language model (e.g., GPT-4) to find systematic patterns of failure and describe them in natural language. We use MultiMon to find 14 systematic failures (e.g., "ignores quantifiers") of the CLIP text-encoder, each comprising hundreds of distinct inputs (e.g., "a shelf with a few/many books"). Because CLIP is the backbone for most state-of-the-art multimodal systems, these inputs produce failures in Midjourney 5.1, DALL-E, VideoFusion, and others. MultiMon can also steer towards failures relevant to specific use cases, such as self-driving cars. We see MultiMon as a step towards evaluation that autonomously explores the long tail of potential system failures. Code for MULTIMON is available at https://github.com/tsb0601/MultiMon.

Cited by

• What's in Common? Multimodal Models Hallucinate When Reasoning Across Scenes
• Uncovering the Computational Ingredients of Human-Like Representations in LLMs
• GHOST: Hallucination-Inducing Image Generation for Multimodal LLMs
• Discovering Divergent Representations between Text-to-Image Models

Related

• Can Large Multimodal Models Actively Recognize Faulty Inputs? A Systematic Evaluation Framework of Their Input Scrutiny Ability
• Benchmarking Large Multimodal Models against Common Corruptions
• When One Modality Sabotages the Others: A Diagnostic Lens on Multimodal Reasoning
• Capturing Failures of Large Language Models via Human Cognitive Biases
• Failures Are Fated, But Can Be Faded: Characterizing and Mitigating Unwanted Behaviors in Large-Scale Vision and Language Models