vix.ing · top · new · best · stats · spec

Linjie Li

  1. The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)
    2023/09/29 by Zhengyuan Yang, Yang, Zhengyuan, Linjie Li +11 · 11 voices · 71 citations
    Computer Science · #Multimodal Machine Learning Applications #Topic Modeling #Natural Language Processing Techniques
  2. GIT: A Generative Image-to-text Transformer for Vision and Language
    2022/05/27 by Jianfeng Wang, Zhengyuan Yang, Wang, Jianfeng +15 · 2 voices · 47 citations
    Computer Science · #Handwritten Text Recognition Techniques #Multimodal Machine Learning Applications #Natural Language Processing Techniques #cs.CV
  3. Segment Everything Everywhere All at Once
    2023/04/13 by Xueyan Zou, Jianwei Yang, Zou, Xueyan +15 · 1 voice · 66 citations
    Computer Science · #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Topic Modeling #cs.CV
  4. The Generative AI Paradox: "What It Can Create, It May Not Understand"
    2023/10/31 by Peter West, Ximing Lu, West, Peter +27 · 5 voices · 13 citations
    Computer Science · Social Sciences · Medicine · #cs.AI #cs.CL #cs.CV #cs.LG
  5. MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities
    2023/08/04 by Weihao Yu, Zhengyuan Yang, Yu, Weihao +13 · 184 citations
    Computer Science · #Topic Modeling #Natural Language Processing Techniques #Multimodal Machine Learning Applications
  6. UNITER: UNiversal Image-TExt Representation Learning
    2020/01/01 by Yen-Chun Chen, Linjie Li, Licheng Yu +5 · 90 citations
    Computer Science · #Advanced Image and Video Retrieval Techniques #Domain Adaptation and Few-Shot Learning #Multimodal Machine Learning Applications
  7. Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning
    2023/06/26 by Fuxiao Liu, Kevin Lin, Liu, Fuxiao +9 · 69 citations
    Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Computational Engineering #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Finance #Multimedia (cs.MM) #Multimodal Machine Learning Applications #and Science (cs.CE)
  8. MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Action
    2023/03/20 by Zhengyuan Yang, Yang, Zhengyuan, Linjie Li +16 · 60 citations
    Computer Science · #Multimodal Machine Learning Applications #Topic Modeling #Natural Language Processing Techniques
  9. GenXD: Generating Any 3D and 4D Scenes
    2024/11/04 by Yuyang Zhao, Chung-Ching Lin, Zhao, Yuyang +15 · 1 voice · 17 citations
    #cs.CV #cs.AI
  10. Generalized Decoding for Pixel, Image, and Language
    2022/12/21 by Xueyan Zou, Zou, Xueyan, Zi-Yi Dou +25 · 33 citations
    Computer Science · #Computation and Language (cs.CL) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling
  11. Less is More: ClipBERT for Video-and-Language Learning via Sparse Sampling
    2021/02/11 by Jie Lei, Lei, Jie, Linjie Li +11 · 25 citations
    Computer Science · #Computation and Language (cs.CL) #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Human Pose and Action Recognition #Multimodal Machine Learning Applications
  12. ShowUI: One Vision-Language-Action Model for GUI Visual Agent
    2024/11/26 by Kevin Qinghong Lin, Linjie Li, Lin, Kevin Qinghong +15 · 54 citations
    Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Human-Computer Interaction (cs.HC) #Multimodal Machine Learning Applications
  13. RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learning
    2025/04/24 by Zihan Wang, Wang, Zihan, Kangrui Wang +33 · 77 citations
    Computer Science · #Reinforcement Learning in Robotics #Multimodal Machine Learning Applications #Domain Adaptation and Few-Shot Learning
  14. Multimodal Foundation Models: From Specialists to General-Purpose Assistants
    2023/09/18 by Chunyuan Li, Li, Chunyuan, Zhe Gan +11 · 28 citations
    Computer Science · #Computation and Language (cs.CL) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling
  15. ReCo: Region-Controlled Text-to-Image Generation
    2022/11/23 by Zhengyuan Yang, Yang, Zhengyuan, Jianfeng Wang +19 · 19 citations
    Computer Science · #Multimodal Machine Learning Applications #Image Retrieval and Classification Techniques #Generative Adversarial Networks and Image Synthesis
  16. HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training
    2020/05/01 by Linjie Li, Li, Linjie, Yen-Chun Chen +9 · 14 citations
    Computer Science · #Computation and Language (cs.CL) #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Human Pose and Action Recognition #Machine Learning (cs.LG) #Multimodal Machine Learning Applications
  17. NUWA-XL: Diffusion over Diffusion for eXtremely Long Video Generation
    2023/03/22 by Shengming Yin, Yin, Shengming, Chenfei Wu +28 · 19 citations
    Computer Science · #Generative Adversarial Networks and Image Synthesis #Video Analysis and Summarization #Video Coding and Compression Technologies
  18. Can MLLMs Reason in Multimodality? EMMA: An Enhanced MultiModal ReAsoning Benchmark
    2025/01/09 by Yi Hao, Hao, Yunzhuo, Jiawei Gu +11 · 37 citations
    Computer Science · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Natural Language Processing Techniques #Semantic Web and Ontologies #Speech and dialogue systems
  19. GPT-4V in Wonderland: Large Multimodal Models for Zero-Shot Smartphone GUI Navigation
    2023/11/13 by An Yan, Zhengyuan Yang, Yan, An +21 · 18 citations
    Computer Science · Psychology · #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Human Pose and Action Recognition #Multimodal Machine Learning Applications #Social Robot Interaction and HRI
  20. SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvement
    2025/04/10 by Xiyao Wang, Wang, Xiyao, Zhengyuan Yang +15 · 40 citations
    Computer Science · #Computer Vision and Pattern Recognition (cs.CV) #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine Learning and Data Classification #Multimodal Machine Learning Applications
  21. MM-Vet v2: A Challenging Benchmark to Evaluate Large Multimodal Models for Integrated Capabilities
    2024/08/01 by Weihao Yu, Yu, Weihao, Zhengyuan Yang +17 · 17 citations
    Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Speech and dialogue systems
  22. Adversarial VQA: A New Benchmark for Evaluating the Robustness of VQA Models
    2021/06/01 by Linjie Li, Jie Lei, Li, Linjie +5 · 8 citations
    Computer Science · Medicine · #COVID-19 diagnosis using AI #Computation and Language (cs.CL) #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Multimodal Machine Learning Applications
  23. Relation-Aware Graph Attention Network for Visual Question Answering
    2019/03/29 by Linjie Li, Li, Linjie, Zhe Gan +5 · 7 citations
    Computer Science · #Advanced Image and Video Retrieval Techniques #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Multimodal Machine Learning Applications
  24. Motion Consistency Model: Accelerating Video Diffusion with Disentangled Motion-Appearance Distillation
    2024/06/11 by Yuanhao Zhai, Zhai, Yuanhao, Kevin Lin +15 · 11 citations
    Computer Science · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Image and Video Quality Assessment
  25. VALUE: A Multi-Task Benchmark for Video-and-Language Understanding Evaluation
    2021/06/08 by Linjie Li, Li, Linjie, Jie Lei +27 · 6 citations
    Computer Science · #Multimodal Machine Learning Applications #Topic Modeling #Domain Adaptation and Few-Shot Learning
  26. An Empirical Study of Multimodal Model Merging
    2023/04/28 by Yi-Lin Sung, Sung, Yi-Lin, Linjie Li +9 · 6 citations
    Computer Science · #Multimodal Machine Learning Applications #Topic Modeling #Domain Adaptation and Few-Shot Learning
  27. MMWorld: Towards Multi-discipline Multi-faceted World Model Evaluation in Videos
    2024/06/12 by Xuehai He, He, Xuehai, Weixi Feng +25 · 9 citations
    Computer Science · #Advanced Vision and Imaging #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Human Pose and Action Recognition #Video Surveillance and Tracking Methods
  28. MM-Narrator: Narrating Long-form Videos with Multimodal In-Context Learning
    2023/11/29 by Chaoyi Zhang, Zhang, Chaoyi, Kevin Lin +13 · 7 citations
    Computer Science · #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Music and Audio Processing #Natural Language Processing Techniques #Video Analysis and Summarization
  29. List Items One by One: A New Data Source and Learning Paradigm for Multimodal LLMs
    2024/04/25 by Yan An, Yan, An, Zhengyuan Yang +19 · 8 citations
    Arts and Humanities · Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Lexicography and Language Studies #Natural Language Processing Techniques #Semantic Web and Ontologies
  30. An Empirical Study of End-to-End Video-Language Transformers with Masked Visual Modeling
    2022/09/04 by Tsu-Jui Fu, Linjie Li, Fu, Tsu-Jui +11 · 4 citations
    Computer Science · #Multimodal Machine Learning Applications #Advanced Neural Network Applications #Domain Adaptation and Few-Shot Learning
  31. LightningDOT: Pre-training Visual-Semantic Embeddings for Real-Time Image-Text Retrieval
    2021/03/16 by Siqi Sun, Sun, Siqi, Yen-Chun Chen +9 · 3 citations
    Computer Science · #Multimodal Machine Learning Applications #Domain Adaptation and Few-Shot Learning #Advanced Image and Video Retrieval Techniques
  32. SlowFast-VGen: Slow-Fast Learning for Action-Driven Long Video Generation
    2024/10/30 by Yining Hong, Baiyu Liu, Hong, Yining +21 · 7 citations
    Biochemistry, Genetics and Molecular Biology · Computer Science · #Advanced Vision and Imaging #Artificial Intelligence (cs.AI) #Cell Image Analysis Techniques #Computation and Language (cs.CL) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Human Pose and Action Recognition #Machine Learning (cs.LG) #Robotics (cs.RO)
  33. EditRoom: LLM-parameterized Graph Diffusion for Composable 3D Room Layout Editing
    2024/10/03 by Zheng, Kaizhi, Xiaotong Chen, Xuehai He +16 · 7 citations
    Computer Science · Engineering · #3D Shape Modeling and Analysis #Artificial Intelligence (cs.AI) #Computer Graphics and Visualization Techniques #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Graph Theory and Algorithms #Graphics (cs.GR) #Human-Computer Interaction (cs.HC)
  34. MMIE: Massive Multimodal Interleaved Comprehension Benchmark for Large Vision-Language Models
    2024/10/14 by Peng Xia, Xia, Peng, Siwei Han +21 · 7 citations
    Computer Science · #Computation and Language (cs.CL) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling
  35. Idea2Img: Iterative Self-Refinement with GPT-4V(ision) for Automatic Image Design and Generation
    2023/10/12 by Zhengyuan Yang, Yang, Zhengyuan, Jianfeng Wang +11 · 4 citations
    Computer Science · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling
  36. Audio-Aware Large Language Models as Judges for Speaking Styles
    2025/06/06 by Cheng-Han Chiang, Chiang, Cheng-Han, Xiaofei Wang +18 · 10 citations
    Computer Science · Psychology · #Artificial Intelligence (cs.AI) #Audio and Speech Processing (eess.AS) #Computation and Language (cs.CL) #Emotion and Mood Recognition #FOS: Computer and information sciences #FOS: Electrical engineering #Music and Audio Processing #Speech Recognition and Synthesis #electronic engineering #information engineering
  37. MMSum: A Dataset for Multimodal Summarization and Thumbnail Generation of Videos
    2023/06/07 by Jielin Qiu, Qiu, Jielin, Jiacheng Zhu +21 · 3 citations
    Computer Science · #Video Analysis and Summarization #Natural Language Processing Techniques #Music and Audio Processing
  38. ViCrit: A Verifiable Reinforcement Learning Proxy Task for Visual Perception in VLMs
    2025/06/11 by Xiyao Wang, Zhengyuan Yang, Wang, Xiyao +22 · 11 citations
    Computer Science · #Computer Vision and Pattern Recognition (cs.CV) #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Machine Learning (cs.LG) #Multimodal Machine Learning Applications
  39. COSMO: COntrastive Streamlined MultimOdal Model with Interleaved Pre-Training
    2024/01/01 by Alex Jinpeng Wang, Wang, Alex Jinpeng, Linjie Li +13 · 3 citations
    Computer Science · #Multimodal Machine Learning Applications #Topic Modeling #Natural Language Processing Techniques
  40. Point-RFT: Improving Multimodal Reasoning with Visually Grounded Reinforcement Finetuning
    2025/05/26 by Minheng Ni, Ni, Minheng, Zhengyuan Yang +11 · 10 citations
    Computer Science · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Natural Language Processing Techniques #Semantic Web and Ontologies #Speech and dialogue systems
  41. Unfolding Spatial Cognition: Evaluating Multimodal Models on Visual Simulations
    2025/06/05 by Linjie Li, Li, Linjie, Mahtab Bigverdi +12 · 8 citations
    Computer Science · Engineering · #Computer Vision and Pattern Recognition (cs.CV) #Constraint Satisfaction and Optimization #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Spatial Cognition and Navigation
  42. VideoGUI: A Benchmark for GUI Automation from Instructional Videos
    2024/06/14 by Kevin Qinghong Lin, Lin, Kevin Qinghong, Linjie Li +13 · 5 citations
    Computer Science · #Video Analysis and Summarization
  43. Leveraging Visual Tokens for Extended Text Contexts in Multi-Modal Learning
    2024/06/04 by Alex Jinpeng Wang, Linjie Li, Wang, Alex Jinpeng +9 · 4 citations
    Computer Science · #Multimodal Machine Learning Applications #Open Education and E-Learning #Speech and dialogue systems
  44. VAGEN: Reinforcing World Model Reasoning for Multi-Turn VLM Agents
    2025/10/19 by Kangrui Wang, Pingyue Zhang, Wang, Kangrui +27 · 10 citations
    Computer Science · #Multimodal Machine Learning Applications #Domain Adaptation and Few-Shot Learning #Topic Modeling
  45. Diagnostic Benchmark and Iterative Inpainting for Layout-Guided Image Generation
    2023/04/13 by Jaemin Cho, Cho, Jaemin, Linjie Li +9 · 2 citations
    Computer Science · #Advanced Vision and Imaging #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Computer Graphics and Visualization Techniques #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Visual Attention and Saliency Detection
  46. Adaptive Human Matting for Dynamic Videos
    2023/04/12 by Chung-Ching Lin, Lin, Chung-Ching, Jiang Wang +11 · 1 citation
    Computer Science · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Image Enhancement Techniques #Visual Attention and Saliency Detection
  47. A Simple "Try Again" Can Elicit Multi-Turn LLM Reasoning
    2025/07/18 by Licheng Liu, Liu, Licheng, Zihan Wang +13 · 5 citations
    Computer Science · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Multimodal Machine Learning Applications #Reinforcement Learning in Robotics #Topic Modeling
  48. FullFront: Benchmarking MLLMs Across the Full Front-End Engineering Workflow
    2025/05/23 by Haoyu Sun, Huichen Will Wang, Sun, Haoyu +7 · 3 citations
    Engineering · #Computation and Language (cs.CL) #Drilling and Well Engineering #FOS: Computer and information sciences #Tunneling and Rock Mechanics
  49. InteractiveOmni: A Unified Omni-modal Model for Audio-Visual Multi-turn Dialogue
    2025/10/15 by Wenwen Tong, Tong, Wenwen, Guo, Hewei +46 · 4 citations
    Arts and Humanities · Computer Science · Psychology · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Language, Metaphor, and Cognition #Speech and dialogue systems #Subtitles and Audiovisual Media
  50. GLIMPSE: Do Large Vision-Language Models Truly Think With Videos or Just Glimpse at Them?
    2025/07/13 by Yiyang Zhou, Linjie Li, Zhou, Yiyang +21 · 6 citations
    Computer Science · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Multimodal Machine Learning Applications
  51. Seeing is Not Reasoning: MVPBench for Graph-based Evaluation of Multi-path Visual Physical CoT
    2025/05/30 by Zhuobai Dong, Dong, Zhuobai, Junchao Yi +12 · 2 citations
    Computer Science · #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #Data Visualization and Analytics #FOS: Computer and information sciences
  52. ThinkMorph: Emergent Properties in Multimodal Interleaved Chain-of-Thought Reasoning
    2025/10/30 by Jiawei Gu, Yi Hao, Gu, Jiawei +13 · 5 citations
    Computer Science · Psychology · #Multimodal Machine Learning Applications #Language, Metaphor, and Cognition #Action Observation and Synchronization
  53. Chain-of-Experts: Unlocking the Communication Power of Mixture-of-Experts Models
    2025/06/23 by Zihan Wang, Wang, Zihan, Rui Pan +17 · 1 voice · 1 citation
    #cs.LG #cs.CL
  54. Efficient Frame Selection for Long Videos at Test Time with Attention-Based MLLM Selectors
    2026/07/17 by Yilin Wang, Xiangxi Zheng, Dongxing Mao +6
    #cs.CV
  55. HumanCLAW: Can Vision-Language Models Act Through a Body?
    2026/07/29 by Siyao Li, Jiawei Gu, Shuai Liu +15
    Computer Science · #cs.CV #cs.RO