Multi-Stream LLMs: Unblocking Language Models with Parallel Streams of Thoughts, Inputs and Outputs

2026/05/12 by Guinan Su, Yanwu Yang, Xueyan Li +1 · 17 voices · 1 citation
#cs.LG #cs.CL

paper · pdf

Abstract

The continued improvements in language model capability have unlocked their widespread use as drivers of autonomous agents, for example in coding or computer use applications. However, the core of these systems has not changed much since early instruction-tuned models like ChatGPT. Even advanced AI agents function on message exchange formats, successively exchanging messages with users, systems, with itself (i.e. chain-of-thought) and tools in a single stream of computation. This bottleneck to a single stream in chat models leads to a number of limitations: the agent cannot act (generate output) while reading, and in reverse, cannot react to new information while writing. Similarly, the agent cannot act while thinking and cannot think while reading or acting on information. In this work, we show that models can be unblocked by switching from instruction-tuning for sequential message formats to instruction-tuning for multiple, parallel streams of computation, splitting each role into a separate stream. Every forward pass of the language model then simultaneously reads from multiple input streams and generates tokens in multiple output streams, all of which causally depend on earlier timesteps. We argue that this data-driven change remedies a number of usability limitations as outlined above, improves model efficiency through parallelization, improves model security through better separation of concerns and can further improve model monitorability.

Citations

• Extending the Context of Pretrained LLMs by Dropping Their Positional Embeddings
• Your LLM Knows the Future: Uncovering Its Multi-Token Prediction Potential
• AI and the End of an Era
• Better & Faster Large Language Models via Multi-token Prediction
• Efficient Streaming Language Models with Attention Sinks
• Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection
• Toolformer: Language Models Can Teach Themselves to Use Tools
• FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
• Training language models to follow instructions with human feedback
• Language Models are Few-Shot Learners
• Attention Is All You Need
• Intelligence without representation
• A simplest systematics for the organization of turn-taking for conversation
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Cited by

• Native Video-Action Pretraining for Generalizable Robot Control

Discussions

• Multi-Stream LLMs: new paper on parallelizing/separating prompts, thinking, I/O [hn, 156 points, 16 comments]
• I've wondered for years why we don't have multi-stream LLMs to separate prompts, "thinking," and data to make them immune to prompt injection. paper came out 8 days ago. they're a lot faster and more [bsky, 22 points, 1 comments]
• there is! arxiv.org/abs/2605.12460 [bsky, 7 points, 0 comments]
• What if both forks send messages to each other during their work, and then merge, so that the final being has memories of being A, being B, *and* each side of the conversation between A and B? Hell, w [bsky, 4 points, 2 comments]
• Multi-Stream LLMs: new paper on parallelizing/separating prompts, thinking, I/O https://arxiv.org/abs/2605.12460 https://news.ycombinator.com/item?id=48227923 [bsky, 1 points, 0 comments]
• [27/30] 202 Upvotes, 18 Comments, 3 Posts, arXiv:2605.12460 🆕Multi-Stream LLMs: Unblocking Language Models with Parallel Streams of Thoughts, Inputs and Outputs Guinan Su, Yanwu Yang, Xueyan Li, Jona [bsky, 1 points, 1 comments]
• Between multi-stream LLMs (arxiv.org/abs/2605.12460), 1000t/s speeds (not even on the fastest hardware and not the limit of speculation) and improved token efficiency + better harnesses/tools the mode [bsky, 1 points, 1 comments]
• 3) Multi-Stream LLMs: Unblocking Language Models with Parallel Streams of Thoughts, Inputs and Outputs by Guinan Su, Yanwu Yang, Xueyan Li, Jonas Geiping arxiv.org/abs/2605.124... [bsky, 0 points, 1 comments]
• 📰 A new paper proposes parallelizing and separating prompts, thinking, and I/O for multi-stream large language models (LLMs), aiming to enhance efficiency and scalability. 🔗 https://arxiv.org/abs/26 [bsky, 0 points, 0 comments]
• Exciting developments in AI! A new paper explores multi-stream LLMs, focusing on parallelizing prompts, enhancing thinking processes, and optimizing I/O. This could revolutionize how we interact with [bsky, 0 points, 0 comments]
• Multi-Stream LLMs: new paper on parallelizing/separating prompts, thinking, I/O https://arxiv.org/abs/2605.12460 (https://news.ycombinator.com/item?id=48227923) [bsky, 0 points, 0 comments]
• Multi-Stream LLMs: new paper on parallelizing/separating prompts, thinking, I/O https:// arxiv.org/abs/2605.12460 # arxiv # llm # llms [mastodon, 0 points, 0 comments]
• [2605.12460] Multi-Stream LLMs: Unblocking Language Models with Parallel Streams of Thoughts, Inputs and Outputs — Interesting take on making LLMs less sequential by running parallel streams for reaso [bsky, 0 points, 0 comments]
• Multi-Stream LLMs: new paper on parallelizing/separating prompts, thinking, I/O https://arxiv.org/abs/2605.12460 (https://news.ycombinator.com/item?id=48227923) [bsky, 0 points, 0 comments]
• Multi-Stream LLMs: Unblocking Language Models with Parallel Streams of Thoughts, Inputs and Outputs arxiv.org/abs/2605.12460 [bsky, 0 points, 0 comments]
• Multi-Stream LLMs: new paper on parallelizing/separating prompts, thinking, I/O [bsky, 0 points, 0 comments]
• Multi-Stream LLMs: new paper on parallelizing/separating prompts, thinking, I/O https://arxiv.org/abs/2605.12460 [bsky, 0 points, 0 comments]

Related

• Can Language Models Follow Multiple Turns of Entangled Instructions?
• Self-controller: Controlling LLMs with Multi-round Step-by-step Self-awareness
• Efficient Streaming Language Models with Attention Sinks
• SysBench: Can Large Language Models Follow System Messages?
• Advances in LLMs with Focus on Reasoning, Adaptability, Efficiency and Ethics