Think-as-You-See: Streaming Chain-of-Thought Reasoning for Large Vision-Language Models
2026/03/03 by Jialiang Zhang, Junlong Tong, Junyan Lin +4 · 1 voice
Computer Science · #cs.CV
paper · pdf · doi:10.48550/arxiv.2603.02872
arxiv published 2026/03/03 · arxiv updated 2026/03/06
Abstract
Large Vision Language Models (LVLMs) exhibit strong Chain-of-Thought (CoT) capabilities, yet most existing paradigms assume full-video availability before inference, a batch-style process misaligned with real-world video streams where information arrives sequentially. Motivated by the streaming nature of video data, we investigate two streaming reasoning paradigms for LVLMs. The first, an interleaved paradigm, alternates between receiving frames and producing partial reasoning but remains constrained by strictly ordered cache updates. To better match streaming inputs, we propose Think-as-You-See (TaYS), a unified framework enabling true concurrent reasoning. TaYS integrates parallelized CoT generation, stream-constrained training, and stream-parallel inference. It further employs temporally aligned reasoning units, streaming attention masks and positional encodings, and a dual KV-cache that decouples visual encoding from textual reasoning. We evaluate all paradigms on the Qwen2.5-VL family across representative video CoT tasks, including event dynamics analysis, causal reasoning, and thematic understanding. Experiments show that TaYS consistently outperforms both batch and interleaved baselines, improving reasoning performance while substantially reducing time-to-first-token (TTFT) and overall reasoning delay. These results demonstrate the effectiveness of data-aligned streaming reasoning in enabling efficient and responsive video understanding for LVLMs. We release our code at https://github.com/EIT-NLP/StreamingLLM/tree/main/TaYS
Citations
- StreamingCoT: A Dataset for Temporal Dynamics and Multimodal Chain-of-Thought Reasoning in Streaming VideoQA
- Revisiting Multimodal Positional Encoding in Vision-Language Models
- StreamingThinker: Large Language Models Can Think While Reading
- VisiPruner: Decoding Discontinuous Cross-Modal Dynamics for Efficient Multimodal LLMs
- StreamingVLM: Real-Time Understanding for Infinite Video Streams
- When Thinking Drifts: Evidential Grounding for Robust Video Reasoning
- StreamAgent: Towards Anticipatory Agents for Streaming Video Understanding
- ViTCoT: Video-Text Interleaved Chain-of-Thought for Boosting Video Understanding in Large Language Models
- Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
- Temporal Chain of Thought: Long-Video Understanding by Thinking in Frames
- Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontiers
- Flash-VStream: Efficient Real-Time Understanding for Long Video Streams
- Chain-of-Frames: Advancing Video Understanding in Multimodal LLMs via Frame-Aware Reasoning
- LLM as Effective Streaming Processor: Bridging Streaming-Batch Mismatches with Group Position Encoding
- Prolonged Reasoning Is Not All You Need: Certainty-Based Adaptive Routing for Efficient LLM/MLLM Reasoning
- Skywork R1V: Pioneering Multimodal Reasoning with Chain-of-Thought
- Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models
- Can Atomic Step Decomposition Enhance the Self-structured Reasoning of Multimodal Large Models?
- Streaming Video Question-Answering with In-context Video KV-Cache Retrieval
- Qwen2.5-VL Technical Report
- Efficient Reasoning with Hidden Thinking
- Streaming Video Understanding and Multi-round Interaction with Memory-enhanced Knowledge
- Dispider: Enabling Video LLMs with Active Real-Time Interaction via Disentangled Perception, Decision, and Reaction
- PruneVid: Visual Token Pruning for Efficient Video Large Language Models
- CoMT: A Novel Benchmark for Chain of Multi-modal Thought on Large Vision-Language Models
- StreamChat: Chatting with Streaming Video
- VideoEspresso: A Large-Scale Chain-of-Thought Dataset for Fine-Grained Video Reasoning via Core Frame Selection
- GPT-4o System Card
- LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understanding
- EventHallusion: Diagnosing Event Hallucinations in Video LLMs
- VideoCoT: A Video Chain-of-Thought Dataset with Active Annotation Tool
- VideoHallucer: Evaluating Intrinsic and Extrinsic Hallucinations in Large Video-Language Models
- VideoLLM-online: Online Video Large Language Model for Streaming Video
- StreamBench: Towards Benchmarking Continuous Improvement of Language Agents
- Streaming Long Video Understanding with Large Language Models
- Video-of-Thought: Step-by-Step Video Reasoning from Perception to Cognition
- MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understanding
- LongVLM: Efficient Long Video Understanding via Large Language Models
- Memory Consolidation Enables Long-Context Video Understanding
- M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation
- Video-LLaVA: Learning United Visual Representation by Alignment Before Projection
- DDCoT: Duty-Distinct Chain-of-Thought Prompting for Multimodal Reasoning in Language Models
- A survey on multimodal large language models
- Multimodal Chain-of-Thought Reasoning in Language Models
- PromptCap: Prompt-Guided Task-Aware Image Captioning
- BNAI, NO-TOKEN, and MIND-UNITY: Pillars of a Systemic Revolution in Artificial Intelligence
- RoFormer: Enhanced Transformer with Rotary Position Embedding
- Constructing inferences during narrative text comprehension.
- FrameMind: Frame-Interleaved Video Reasoning via Reinforcement Learning
Discussions
Related