vix.ing · top · new · best · stats · spec

Furu Wei

  1. Retentive Network: A Successor to Transformer for Large Language Models
    2023/07/17 by Yutao Sun, Li Dong, Sun, Yutao +13 · 8 voices · 108 citations
    Computer Science · #Topic Modeling #Natural Language Processing Techniques #Multimodal Machine Learning Applications
  2. The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits
    2024/02/27 by Shuming Ma, Hongyu Wang, Ma, Shuming +17 · 24 voices · 50 citations
    Computer Science · #Natural Language Processing Techniques #Topic Modeling #cs.CL #cs.LG
  3. LongNet: Scaling Transformers to 1,000,000,000 Tokens
    2023/07/05 by Jiayu Ding, Shuming Ma, Ding, Jiayu +13 · 7 voices · 28 citations
    Computer Science · #Natural Language Processing Techniques #Speech Recognition and Synthesis #Topic Modeling #cs.CL #cs.LG
  4. Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers
    2022/12/20 by Damai Dai, Dai, Damai, Yutao Sun +12 · 4 voices · 52 citations
    Computer Science · #Topic Modeling #Machine Learning and Data Classification #Advanced Neural Network Applications
  5. Language Is Not All You Need: Aligning Perception with Language Models
    2023/02/27 by Shaohan Huang, Huang, Shaohan, Li Dong +35 · 1 voice · 46 citations
    Computer Science · #Multimodal Machine Learning Applications #Topic Modeling #Natural Language Processing Techniques
  6. BitNet: Scaling 1-bit Transformers for Large Language Models
    2023/10/17 by Hongyu Wang, Shuming Ma, Wang, Hongyu +18 · 8 voices · 42 citations
    Computer Science · #Multimodal Machine Learning Applications #Speech Recognition and Synthesis #Topic Modeling #cs.CL
  7. DeepNet: Scaling Transformers to 1,000 Layers
    2022/03/01 by Hongyu Wang, Wang, Hongyu, Shuming Ma +10 · 2 voices · 18 citations
    Computer Science · Engineering · #Neural Networks and Reservoir Computing #Ferroelectric and Negative Capacitance Devices #Advanced Neural Network Applications
  8. Augmenting Language Models with Long-Term Memory
    2023/06/12 by Weizhi Wang, Li Dong, Wang, Weizhi +11 · 3 voices · 24 citations
    Computer Science · #Topic Modeling #Natural Language Processing Techniques #Multimodal Machine Learning Applications
  9. Improving Text Embeddings with Large Language Models
    2023/12/31 by Liang Wang, Wang, Liang, Nan Yang +9 · 4 voices · 62 citations
    Computer Science · #Topic Modeling #Natural Language Processing Techniques #Hate Speech and Cyberbullying Detection
  10. Kosmos-2: Grounding Multimodal Large Language Models to the World
    2023/06/26 by Zhiliang Peng, Peng, Zhiliang, Wenhui Wang +11 · 1 voice · 139 citations
    Computer Science · #cs.CL #cs.CV
  11. MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers
    2020/02/25 by Wenhui Wang, Furu Wei, Wang, Wenhui +9 · 237 citations
    Computer Science · Engineering · Materials Science · #Topic Modeling #Ferroelectric and Negative Capacitance Devices #Machine Learning in Materials Science
  12. BitNet b1.58 2B4T Technical Report
    2025/04/16 by Shuming Ma, Hongyu Wang, Ma, Shuming +14 · 10 voices · 13 citations
    Computer Science · #Advanced Neural Network Applications #Big Data and Digital Economy #Natural Language Processing Techniques #cs.CL #cs.LG
  13. WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing
    2022/07/04 by Sanyuan Chen, Chengyi Wang, Zhengyang Chen +15 · 255 citations
    Computer Science · #Speech Recognition and Synthesis #Speech and Audio Processing #Music and Audio Processing
  14. 1-bit AI Infra: Part 1.1, Fast and Lossless BitNet b1.58 Inference on CPUs
    2024/10/21 by Jinheng Wang, Wang, Jinheng, Hansong Zhou +13 · 5 voices · 3 citations
    #cs.CL
  15. Imagine while Reasoning in Space: Multimodal Visualization-of-Thought
    2025/01/13 by Chengzu Li, Li, Chengzu, Wenshan Wu +14 · 4 voices · 63 citations
    Computer Science · #Advanced Graph Neural Networks #Computation and Language (cs.CL) #Computer Vision and Pattern Recognition (cs.CV) #Data Visualization and Analytics #FOS: Computer and information sciences #Machine Learning (cs.LG) #Multimodal Machine Learning Applications #cs.CL #cs.CV #cs.LG
  16. BEATs: Audio Pre-Training with Acoustic Tokenizers
    2022/12/18 by Sanyuan Chen, Yu Wu, Chen, Sanyuan +11 · 93 citations
    Computer Science · #Music and Audio Processing #Speech and Audio Processing #Speech Recognition and Synthesis
  17. Swin Transformer V2: Scaling Up Capacity and Resolution
    2022/06/01 by Ze Liu, Han Hu, Yutong Lin +9 · 83 citations
    Computer Science · #Advanced Neural Network Applications #Domain Adaptation and Few-Shot Learning #Multimodal Machine Learning Applications
  18. Reinforcement Pre-Training
    2025/06/09 by Quennie Dong, Qingxiu Dong, Li Dong +13 · 5 voices · 14 citations
    Computer Science · #Topic Modeling #Natural Language Processing Techniques #Text Readability and Simplification
  19. LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking
    2022/04/18 by Yupan Huang, Tengchao Lv, Huang, Yupan +7 · 61 citations
    Computer Science · #Computation and Language (cs.CL) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Handwritten Text Recognition Techniques #Multimodal Machine Learning Applications #Topic Modeling
  20. You Only Cache Once: Decoder-Decoder Architectures for Language Models
    2024/05/08 by Yutao Sun, Sun, Yutao, Li Dong +15 · 2 voices · 26 citations
    Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #cs.CL
  21. Multilingual E5 Text Embeddings: A Technical Report
    2024/02/08 by Liang Wang, Nan Yang, Wang, Liang +9 · 86 citations
    Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Information Retrieval (cs.IR) #Natural Language Processing Techniques #Text Readability and Simplification #Topic Modeling
  22. Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaboration
    2023/07/11 by Zhenhailong Wang, Shaoguang Mao, Wang, Zhenhailong +9 · 55 citations
    Computer Science · Social Sciences · #Persona Design and Applications #AI in Service Interactions #Technology Use by Older Adults
  23. Fine-Tuning LLaMA for Multi-Stage Text Retrieval
    2023/10/12 by Xueguang Ma, Liang Wang, Ma, Xueguang +7 · 57 citations
    Computer Science · #Topic Modeling #Natural Language Processing Techniques #Multimodal Machine Learning Applications
  24. VL-BERT: Pre-training of Generic Visual-Linguistic Representations
    2019/08/22 by Weijie Su, Su, Weijie, Xizhou Zhu +11 · 39 citations
    Computer Science · #Computation and Language (cs.CL) #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Multimodal Machine Learning Applications #Topic Modeling
  25. Differential Transformer
    2024/10/07 by Tianzhu Ye, Ye, Tianzhu, Li Dong +11 · 7 voices
    Engineering · #Physics and Engineering Research Articles #cs.CL #cs.LG
  26. VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Experts
    2021/11/03 by Hangbo Bao, Bao, Hangbo, Wenhui Wang +9 · 35 citations
    Computer Science · #Multimodal Machine Learning Applications #Domain Adaptation and Few-Shot Learning #Topic Modeling
  27. Generative Representational Instruction Tuning
    2024/02/15 by Niklas Muennighoff, Hongjin Su, Muennighoff, Niklas +13 · 2 voices · 37 citations
    Computer Science · Psychology · #Innovative Teaching and Learning Methods #cs.AI #cs.CL #cs.LG
  28. TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models
    2021/09/21 by Minghao Li, Tengchao Lv, Li, Minghao +15 · 27 citations
    Computer Science · #Computation and Language (cs.CL) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Handwritten Text Recognition Techniques #Image Processing and 3D Reconstruction #Natural Language Processing Techniques
  29. Synthetic Data (Almost) from Scratch: Generalized Instruction Tuning for Language Models
    2024/02/20 by Haoran Li, Li, Haoran, Qingxiu Dong +40 · 1 voice · 15 citations
    Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Speech Recognition and Synthesis #Topic Modeling #cs.CL
  30. BERT Loses Patience: Fast and Robust Inference with Early Exit
    2020/06/07 by Wangchunshu Zhou, Zhou, Wangchunshu, Canwen Xu +9 · 22 citations
    Computer Science · #Topic Modeling #Natural Language Processing Techniques #Domain Adaptation and Few-Shot Learning
  31. BEiT v2: Masked Image Modeling with Vector-Quantized Visual Tokenizers
    2022/08/12 by Zhiliang Peng, Peng, Zhiliang, Dong Li +7 · 27 citations
    Computer Science · #Advanced Neural Network Applications #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis
  32. SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing
    2021/10/14 by Junyi Ao, Ao, Junyi, Rui Wang +25 · 24 citations
    Computer Science · #Audio and Speech Processing (eess.AS) #Computation and Language (cs.CL) #FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Natural Language Processing Techniques #Sound (cs.SD) #Speech Recognition and Synthesis #Speech and Audio Processing #electronic engineering #information engineering
  33. Unified Language Model Pre-training for Natural Language Understanding\n and Generation
    2019/05/08 by Li Dong, Dong, Li, Nan Yang +15 · 20 citations
    Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling
  34. Mixture of LoRA Experts
    2024/04/21 by Xun Wu, Wu, Xun, Shaohan Huang +3 · 39 citations
    Computer Science · #Anomaly Detection Techniques and Applications #Computation and Language (cs.CL) #Context-Aware Activity Recognition Systems #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and Data Classification #Multimedia (cs.MM)
  35. Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Tasks
    2022/08/22 by Wenhui Wang, Wang, Wenhui, Hangbo Bao +19 · 27 citations
    Computer Science · #Multimodal Machine Learning Applications #Advanced Image and Video Retrieval Techniques #Domain Adaptation and Few-Shot Learning
  36. In-context Autoencoder for Context Compression in a Large Language Model
    2023/07/13 by Tao Ge, Jing Hu, Ge, Tao +8 · 30 citations
    Computer Science · #Speech Recognition and Synthesis #Topic Modeling #Machine Learning in Healthcare
  37. MathScale: Scaling Instruction Tuning for Mathematical Reasoning
    2024/03/05 by Zhengyang Tang, Tang, Zhengyang, Xingxing Zhang +5 · 34 citations
    Computer Science · Social Sciences · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Intelligent Tutoring Systems and Adaptive Learning #Machine Learning (cs.LG) #Mathematics Education and Teaching Techniques
  38. Query2doc: Query Expansion with Large Language Models
    2023/03/14 by Liang Wang, Wang, Liang, Nan Yang +3 · 24 citations
    Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Information Retrieval (cs.IR) #Natural Language Processing Techniques #Topic Modeling #Web Data Mining and Analysis
  39. Adapting Large Language Models to Domains via Reading Comprehension
    2023/09/18 by Daixuan Cheng, Shaohan Huang, Cheng, Daixuan +3 · 27 citations
    Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Topic Modeling
  40. TextDiffuser: Diffusion Models as Text Painters
    2023/05/18 by Jingye Chen, Chen, Jingye, Yupan Huang +9 · 23 citations
    Computer Science · #Generative Adversarial Networks and Image Synthesis #Image Retrieval and Classification Techniques #Handwritten Text Recognition Techniques
  41. Speak Foreign Languages with Your Own Voice: Cross-Lingual Neural Codec Language Modeling
    2023/03/07 by Ziqiang Zhang, Zhang, Ziqiang, Long Zhou +23 · 23 citations
    Computer Science · #Artificial Intelligence (cs.AI) #Audio and Speech Processing (eess.AS) #Computation and Language (cs.CL) #FOS: Computer and information sciences #FOS: Electrical engineering #Natural Language Processing Techniques #Sound (cs.SD) #Speech Recognition and Synthesis #Topic Modeling #electronic engineering #information engineering
  42. InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training
    2020/07/15 by Zewen Chi, Li Dong, Chi, Zewen +16 · 21 citations
    Computer Science · #Topic Modeling #Natural Language Processing Techniques #Multimodal Machine Learning Applications
  43. Reasoning with Exploration: An Entropy Perspective
    2025/06/17 by Shaohan Huang, Cheng, Daixuan, Huang, Shaohan +9 · 108 citations
    Computer Science · #Natural Language Processing Techniques #Topic Modeling #Multimodal Machine Learning Applications
  44. Mind's Eye of LLMs: Visualization-of-Thought Elicits Spatial Reasoning in Large Language Models
    2024/04/04 by Wenshan Wu, Wu, Wenshan, Shaoguang Mao +11 · 29 citations
    Computer Science · Social Sciences · #Semantic Web and Ontologies #Geographic Information Systems Studies #Constraint Satisfaction and Optimization
  45. MoRA: High-Rank Updating for Parameter-Efficient Fine-Tuning
    2024/05/20 by Ting Jiang, Jiang, Ting, Shaohan Huang +20 · 2 voices · 3 citations
    Computer Science · Engineering · #Advanced Data Compression Techniques #Advanced SAR Imaging Techniques #Blind Source Separation Techniques #cs.CL #cs.LG
  46. Not All Languages Are Created Equal in LLMs: Improving Multilingual Capability by Cross-Lingual-Thought Prompting
    2023/05/11 by Haoyang Huang, Tianyi Tang, Huang, Haoyang +11 · 18 citations
    Computer Science · #Topic Modeling #Natural Language Processing Techniques #Multimodal Machine Learning Applications
  47. WavLLM: Towards Robust and Adaptive Speech Large Language Model
    2024/03/31 by Shujie Hu, Hu, Shujie, Long Zhou +20 · 24 citations
    Computer Science · #Natural Language Processing Techniques #Speech Recognition and Synthesis #Topic Modeling
  48. Kosmos-G: Generating Images in Context with Multimodal Large Language Models
    2023/10/04 by Xichen Pan, Pan, Xichen, Li Dong +9 · 20 citations
    Social Sciences · Computer Science · #Computational and Text Analysis Methods #Multimodal Machine Learning Applications #Topic Modeling
  49. StableMoE: Stable Routing Strategy for Mixture of Experts
    2022/04/18 by Damai Dai, Dong Li, Dai, Damai +11 · 15 citations
    Computer Science · Physics and Astronomy · #Advanced Graph Neural Networks #Complex Network Analysis Techniques #Computation and Language (cs.CL) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG)
  50. LLM as a Mastermind: A Survey of Strategic Reasoning with Large Language Models
    2024/04/01 by Yadong Zhang, Zhang, Yadong, Shaoguang Mao +17 · 23 citations
    Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques
  51. Autoregressive Speech Synthesis without Vector Quantization
    2024/07/11 by Lingwei Meng, Long Zhou, Meng, Lingwei +21 · 23 citations
    Computer Science · #Speech Recognition and Synthesis #Speech and dialogue systems
  52. A Length-Extrapolatable Transformer
    2022/12/20 by Yutao Sun, Sun, Yutao, Dong Li +14 · 13 citations
    Computer Science · #Natural Language Processing Techniques #Topic Modeling #Speech Recognition and Synthesis
  53. LayoutXLM: Multimodal Pre-training for Multilingual Visually-rich Document Understanding
    2021/04/18 by Yiheng Xu, Tengchao Lv, Xu, Yiheng +13 · 10 citations
    Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling
  54. WavMark: Watermarking for Audio Generation
    2023/08/24 by Guangyu Chen, Chen, Guangyu, Yu Wu +9 · 15 citations
    Computer Science · #Advanced Steganography and Watermarking Techniques #Music and Audio Processing #Digital Media Forensic Detection
  55. Learning to Rank in Generative Retrieval
    2023/06/27 by Yongqi Li, Nan Yang, Li, Yongqi +7 · 13 citations
    Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Information Retrieval (cs.IR) #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling
  56. UniSpeech-SAT: Universal Speech Representation Learning with Speaker Aware Pre-Training
    2021/10/12 by Sanyuan Chen, Yu Wu, Chen, Sanyuan +18 · 9 citations
    Computer Science · #Audio and Speech Processing (eess.AS) #Computation and Language (cs.CL) #FOS: Computer and information sciences #FOS: Electrical engineering #Music and Audio Processing #Sound (cs.SD) #Speech Recognition and Synthesis #Speech and Audio Processing #electronic engineering #information engineering
  57. Instruction Pre-Training: Language Models are Supervised Multitask Learners
    2024/06/20 by Daixuan Cheng, Cheng, Daixuan, Yuxian Gu +9 · 17 citations
    Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques
  58. Q-Sparse: All Large Language Models can be Fully Sparsely-Activated
    2024/07/15 by Hongyu Wang, Shuming Ma, Wang, Hongyu +5 · 1 voice · 4 citations
    Computer Science · #cs.CL #cs.LG
  59. LayoutReader: Pre-training of Text and Layout for Reading Order Detection
    2021/08/26 by Zilong Wang, Yiheng Xu, Wang, Zilong +7 · 8 citations
    Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Handwritten Text Recognition Techniques #Natural Language Processing Techniques #Topic Modeling
  60. CLIP Models are Few-shot Learners: Empirical Studies on VQA and Visual\n Entailment
    2022/03/14 by Haoyu Song, Song, Haoyu, Dong Li +7 · 8 citations
    Computer Science · #Advanced Image and Video Retrieval Techniques #Computation and Language (cs.CL) #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Multimodal Machine Learning Applications
  61. HIBERT: Document Level Pre-training of Hierarchical Bidirectional Transformers for Document Summarization
    2019/05/16 by Xingxing Zhang, Furu Wei, Zhang, Xingxing +3 · 6 citations
    Computer Science · #Advanced Text Analysis Techniques #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Natural Language Processing Techniques #Topic Modeling
  62. LongEmbed: Extending Embedding Models for Long Context Retrieval
    2024/04/18 by Dawei Zhu, Zhu, Dawei, Liang Wang +11 · 13 citations
    Computer Science · Social Sciences · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Geographic Information Systems Studies #Machine Learning (cs.LG) #Recommender Systems and Techniques
  63. Task-Specific Expert Pruning for Sparse Mixture-of-Experts
    2022/06/01 by Tianyu Chen, Chen, Tianyu, Shaohan Huang +13 · 8 citations
    Computer Science · Engineering · #Artificial Intelligence (cs.AI) #Distributed Sensor Networks and Detection Algorithms #FOS: Computer and information sciences #Indoor and Outdoor Localization Technologies #Machine Learning (cs.LG) #Mobile Crowdsensing and Crowdsourcing
  64. Neural Question Generation from Text: A Preliminary Study
    2017/04/06 by Qingyu Zhou, Zhou, Qingyu, Nan Yang +9 · 7 citations
    Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling
  65. Learning to Retrieve In-Context Examples for Large Language Models
    2023/07/14 by Liang Wang, Wang, Liang, Nan Yang +3 · 9 citations
    Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Information Retrieval (cs.IR) #Natural Language Processing Techniques #Topic Modeling
  66. Structured Prompting: Scaling In-Context Learning to 1,000 Examples
    2022/12/13 by Yaru Hao, Yutao Sun, Hao, Yaru +9 · 8 citations
    Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling
  67. ALYMPICS: LLM Agents Meet Game Theory -- Exploring Strategic Decision-Making with AI Agents
    2023/11/06 by Shaoguang Mao, Mao, Shaoguang, Yuzhe Cai +13 · 10 citations
    Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Computer Science and Game Theory (cs.GT) #FOS: Computer and information sciences #Topic Modeling
  68. ARLON: Boosting Diffusion Transformers with Autoregressive Models for Long Video Generation
    2024/10/27 by Zongyi Li, Li, Zongyi, Shujie Hu +16 · 14 citations
    Computer Science · #Generative Adversarial Networks and Image Synthesis #Advanced Image Processing Techniques #Image and Video Quality Assessment
  69. Data Selection via Optimal Control for Language Models
    2024/10/09 by Yuxian Gu, Li Dong, Gu, Yuxian +11 · 14 citations
    Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques
  70. PromptBERT: Improving BERT Sentence Embeddings with Prompts
    2022/01/12 by Ting Jiang, Jiang, Ting, Shaohan Huang +15 · 6 citations
    Computer Science · #Topic Modeling #Natural Language Processing Techniques #Sentiment Analysis and Opinion Mining
  71. Reaching Human-level Performance in Automatic Grammatical Error Correction: An Empirical Study
    2018/07/03 by Tao Ge, Ge, Tao, Furu Wei +3 · 9 citations
    Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Text Readability and Simplification #Topic Modeling
  72. UPRISE: Universal Prompt Retrieval for Improving Zero-Shot Evaluation
    2023/03/15 by Daixuan Cheng, Cheng, Daixuan, Shaohan Huang +17 · 7 citations
    Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling
  73. BitNet a4.8: 4-bit Activations for 1-bit LLMs
    2024/11/07 by Hongyu Wang, Shuming Ma, Wang, Hongyu +3 · 1 voice · 3 citations
    Computer Science · Engineering · #Advancements in Semiconductor Devices and Circuit Design #Semiconductor materials and devices #VLSI and Analog Circuit Testing #cs.CL #cs.LG
  74. Large Language Model for Science: A Study on P vs. NP
    2023/09/11 by Qingxiu Dong, Dong, Qingxiu, Li Dong +11 · 5 voices
    Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #cs.AI #cs.CL
  75. MiniLLM: On-Policy Distillation of Large Language Models
    2023/06/14 by Yuxian Gu, Li Dong, Gu, Yuxian +5 · 6 citations
    Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Topic Modeling
  76. Multi-Head Mixture-of-Experts
    2024/04/23 by Xun Wu, Shaohan Huang, Wu, Xun +5 · 8 citations
    Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Expert finding and Q&A systems #FOS: Computer and information sciences #Machine Learning (cs.LG) #Speech and dialogue systems
  77. Adapt-and-Distill: Developing Small, Fast and Effective Pretrained Language Models for Domains
    2021/06/25 by Yunzhi Yao, Shaohan Huang, Yao, Yunzhi +7 · 4 citations
    Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning in Healthcare #Natural Language Processing Techniques #Topic Modeling
  78. Scaling Optimal LR Across Token Horizons
    2024/09/30 by Johan Björck, Alon Benhaim, Bjorck, Johan +7 · 8 citations
    Engineering · #Advanced Measurement and Detection Methods #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Geophysical Methods and Applications #Machine Learning (cs.LG) #Welding Techniques and Residual Stresses
  79. MarkupLM: Pre-training of Text and Markup Language for Visually-rich Document Understanding
    2021/10/16 by Junlong Li, Yiheng Xu, Li, Junlong +5 · 4 citations
    Computer Science · #Handwritten Text Recognition Techniques #Video Analysis and Summarization #Multimodal Machine Learning Applications
  80. Language Models as Inductive Reasoners
    2022/12/21 by Zonglin Yang, Li Dong, Yang, Zonglin +13 · 5 citations
    Computer Science · #Natural Language Processing Techniques
  81. Document AI: Benchmarks, Models and Applications
    2021/11/16 by Lei Cui, Cui, Lei, Yiheng Xu +5 · 4 citations
    Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Handwritten Text Recognition Techniques #Image Retrieval and Classification Techniques #Text and Document Classification Technologies
  82. Controllable Natural Language Generation with Contrastive Prefixes
    2022/02/27 by Qian Jing, Qian, Jing, Dong Li +7 · 4 citations
    Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling
  83. Pre-Training to Learn in Context
    2023/05/16 by Yuxian Gu, Li Dong, Gu, Yuxian +5 · 5 citations
    Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling
  84. Think Only When You Need with Large Hybrid-Reasoning Models
    2025/05/20 by Lingjie Jiang, Jiang, Lingjie, X. J. Wu +17 · 22 citations
    Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Semantic Web and Ontologies
  85. Multimodal Latent Language Modeling with Next-Token Diffusion
    2024/12/11 by Yutao Sun, Hangbo Bao, Sun, Yutao +13 · 10 citations
    Computer Science · #Natural Language Processing Techniques #Topic Modeling #Speech and dialogue systems
  86. A Dependency-Based Neural Network for Relation Classification
    2015/07/16 by Yang Liu, Liu, Yang, Furu Wei +9 · 4 citations
    Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Natural Language Processing Techniques #Neural and Evolutionary Computing (cs.NE) #Text and Document Classification Technologies #Topic Modeling
  87. Context-DPO: Aligning Language Models for Context-Faithfulness
    2024/12/18 by Baolong Bi, Shaohan Huang, Bi, Baolong +25 · 10 citations
    Social Sciences · #Religion, Spirituality, and Psychology
  88. Chain-of-Retrieval Augmented Generation
    2025/01/24 by Liang Wang, Haonan Chen, Wang, Liang +9 · 10 citations
    Computer Science · #Advanced Image and Video Retrieval Techniques #Algorithms and Data Compression #Computation and Language (cs.CL) #FOS: Computer and information sciences #Information Retrieval (cs.IR) #Speech and dialogue systems
  89. HD-Eval: Aligning Large Language Model Evaluators Through Hierarchical Criteria Decomposition
    2024/02/24 by Yuxuan Liu, Tianchi Yang, Liu, Yuxuan +15 · 5 citations
    Computer Science · #Topic Modeling #Machine Learning in Healthcare #Explainable Artificial Intelligence (XAI)
  90. Improving Pretrained Cross-Lingual Language Models via Self-Labeled Word Alignment
    2021/06/11 by Zewen Chi, Li Dong, Chi, Zewen +11 · 3 citations
    Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling
  91. Allocating Large Vocabulary Capacity for Cross-lingual Language Model Pre-training
    2021/09/15 by Bo Zheng, Li Dong, Zheng, Bo +12 · 3 citations
    Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling
  92. Multiview Identifiers Enhanced Generative Retrieval
    2023/05/26 by Yongqi Li, Nan Yang, Li, Yongqi +7 · 4 citations
    Computer Science · #Algorithms and Data Compression #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Information Retrieval (cs.IR) #Machine Learning (cs.LG) #Natural Language Processing Techniques #Topic Modeling
  93. Generic-to-Specific Distillation of Masked Autoencoders
    2023/02/28 by Wei Huang, Zhiliang Peng, Huang, Wei +9 · 4 citations
    Computer Science · Medicine · #Advanced Neural Network Applications #Domain Adaptation and Few-Shot Learning #COVID-19 diagnosis using AI
  94. Preference Optimization for Reasoning with Pseudo Feedback
    2024/11/25 by Jiao, Fangkai, Guo, Geyang, Xingxing Zhang +7 · 8 citations
    Decision Sciences · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Multi-Criteria Decision Making
  95. Reward Reasoning Model
    2025/05/20 by Jiaxin Guo, Zewen Chi, Guo, Jiaxin +11 · 15 citations
    Social Sciences · #Computation and Language (cs.CL) #Education and Professional Development #FOS: Computer and information sciences
  96. PoSE: Efficient Context Window Extension of LLMs via Positional Skip-wise Training
    2023/09/19 by Dawei Zhu, Nan Yang, Zhu, Dawei +11 · 4 citations
    Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Natural Language Processing Techniques #Speech Recognition and Synthesis #Topic Modeling
  97. Meta Reasoning for Large Language Models
    2024/06/17 by Peizhong Gao, Gao, Peizhong, Ao Xie +11 · 5 citations
    Computer Science · #Natural Language Processing Techniques #Semantic Web and Ontologies #Topic Modeling
  98. Bitnet.cpp: Efficient Edge Inference for Ternary LLMs
    2025/02/17 by Jinheng Wang, Wang, Jinheng, Zhou, Hansong +16 · 8 citations
    Computer Science · Engineering · #Advanced Data Storage Technologies #Advanced Data and IoT Technologies #Advanced Steganography and Watermarking Techniques #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Distributed #FOS: Computer and information sciences #Machine Learning (cs.LG) #Parallel #and Cluster Computing (cs.DC)
  99. Visually-Augmented Language Modeling
    2022/05/20 by Weizhi Wang, Wang, Weizhi, Dong Li +13 · 3 citations
    Computer Science · #Computation and Language (cs.CL) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Natural Language Processing Techniques
  100. Inference with Reference: Lossless Acceleration of Large Language Models
    2023/04/10 by Nan Yang, Yang, Nan, Tao Ge +13 · 3 citations
    Computer Science · #Topic Modeling #Natural Language Processing Techniques #Speech Recognition and Synthesis
  101. MMLU-CF: A Contamination-free Multi-task Language Understanding Benchmark
    2024/12/19 by Qihao Zhao, Zhao, Qihao, Yangyu Huang +19 · 6 citations
    Computer Science · #Natural Language Processing Techniques #Topic Modeling #Speech and dialogue systems
  102. Attention Temperature Matters in Abstractive Summarization Distillation
    2021/06/07 by Shengqiang Zhang, Xingxing Zhang, Zhang, Shengqiang +5 · 2 citations
    Computer Science · #Advanced Text Analysis Techniques #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Topic Modeling
  103. Direct Preference Knowledge Distillation for Large Language Models
    2024/06/28 by Yixing Li, Li, Yixing, Yuxian Gu +9 · 4 citations
    Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Semantic Web and Ontologies #Topic Modeling
  104. Auto-ICL: In-Context Learning without Human Supervision
    2023/11/15 by Jinghan Yang, Shuming Ma, Yang, Jinghan +3 · 3 citations
    Computer Science · #AI in Service Interactions #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Context-Aware Activity Recognition Systems #FOS: Computer and information sciences #Machine Learning (cs.LG) #Topic Modeling
  105. Boosting Large Language Model for Speech Synthesis: An Empirical Study
    2023/12/30 by Hongkun Hao, Long Zhou, Hao, Hongkun +11 · 4 citations
    Computer Science · #Audio and Speech Processing (eess.AS) #Computation and Language (cs.CL) #FOS: Computer and information sciences #FOS: Electrical engineering #Natural Language Processing Techniques #Sound (cs.SD) #Speech Recognition and Synthesis #Topic Modeling #electronic engineering #information engineering
  106. On-Policy RL with Optimal Reward Baseline
    2025/05/29 by Yaru Hao, Dong Li, Hao, Yaru +9 · 9 citations
    Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Multimodal Machine Learning Applications #Reinforcement Learning in Robotics #Topic Modeling
  107. K-Level Reasoning: Establishing Higher Order Beliefs in Large Language Models for Strategic Reasoning
    2024/02/02 by Yadong Zhang, Zhang, Yadong, Shaoguang Mao +11 · 3 citations
    Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques
  108. Language Models are General-Purpose Interfaces
    2022/06/13 by Yaru Hao, Haoyu Song, Hao, Yaru +13 · 2 citations
    Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling
  109. On the Pareto Front of Multilingual Neural Machine Translation
    2023/04/06 by Liang Chen, Chen, Liang, Shuming Ma +7 · 2 citations
    Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning and Data Classification #Natural Language Processing Techniques #Topic Modeling
  110. The Era of Agentic Organization: Learning to Organize with Language Models
    2025/10/30 by Zewen Chi, Chi, Zewen, Li Dong +11 · 6 voices
    Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #cs.AI #cs.CL
  111. Neural Melody Composition from Lyrics
    2018/09/12 by Hangbo Bao, Bao, Hangbo, Shaohan Huang +13 · 1 citation
    Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Music Technology and Sound Studies #Music and Audio Processing #Speech Recognition and Synthesis
  112. Examining False Positives under Inference Scaling for Mathematical Reasoning
    2025/02/10 by Yu Wang, Nan Yang, Wang, Yu +6 · 4 citations
    Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Intelligent Tutoring Systems and Adaptive Learning
  113. Cross-Lingual Natural Language Generation via Pre-Training
    2019/09/23 by Zewen Chi, Chi, Zewen, Dong Li +9 · 1 citation
    Computer Science · #Topic Modeling #Natural Language Processing Techniques #Multimodal Machine Learning Applications
  114. DeltaLM: Encoder-Decoder Pre-training for Language Generation and Translation by Augmenting Pretrained Multilingual Encoders
    2021/06/25 by Shuming Ma, Ma, Shuming, Li Dong +14 · 2 citations
    Computer Science · #Topic Modeling #Natural Language Processing Techniques #Text Readability and Simplification
  115. CROP: Zero-shot Cross-lingual Named Entity Recognition with Multilingual Labeled Sequence Translation
    2022/10/13 by Jian Yang, Shaohan Huang, Yang, Jian +15 · 1 citation
    Computer Science · Decision Sciences · #Computation and Language (cs.CL) #Data Quality and Management #FOS: Computer and information sciences #Natural Language Processing Techniques #Topic Modeling
  116. A Survey of Knowledge-Intensive NLP with Pre-Trained Language Models
    2022/02/17 by Da Yin, Dong Li, Yin, Da +11 · 1 citation
    Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling
  117. Corrupted Image Modeling for Self-Supervised Visual Pre-Training
    2022/02/07 by Y.K. Fang, Dong Li, Fang, Yuxin +7 · 1 citation
    Biochemistry, Genetics and Molecular Biology · Computer Science · #Cell Image Analysis Techniques #Domain Adaptation and Few-Shot Learning #Advanced Neural Network Applications
  118. Supervision-Guided Codebooks for Masked Prediction in Speech Pre-training
    2022/06/21 by Chengyi Wang, Yiming Wang, Wang, Chengyi +11 · 1 citation
    Computer Science · #Audio and Speech Processing (eess.AS) #Computation and Language (cs.CL) #FOS: Computer and information sciences #FOS: Electrical engineering #Music and Audio Processing #Sound (cs.SD) #Speech Recognition and Synthesis #Speech and Audio Processing #electronic engineering #information engineering
  119. XDoc: Unified Pre-training for Cross-Format Document Understanding
    2022/10/06 by Jingye Chen, Tengchao Lv, Chen, Jingye +7 · 1 citation
    Computer Science · #Topic Modeling #Natural Language Processing Techniques #Handwritten Text Recognition Techniques
  120. Scaling and Beyond: Advancing Spatial Reasoning in MLLMs Requires New Recipes
    2025/04/21 by Huanyu Zhang, Chengzu Li, Zhang, Huanyu +19 · 5 citations
    Computer Science · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Natural Language Processing Techniques #Semantic Web and Ontologies #Speech and dialogue systems
  121. GanLM: Encoder-Decoder Pre-training with an Auxiliary Discriminator
    2022/12/20 by Jian Yang, Shuming Ma, Yang, Jian +17 · 1 citation
    Computer Science · #Natural Language Processing Techniques #Speech Recognition and Synthesis #Topic Modeling
  122. BitNet v2: Native 4-bit Activations with Hadamard Transformation for 1-bit LLMs
    2025/04/25 by Hongyu Wang, Wang, Hongyu, Shuming Ma +3 · 3 citations
    Computer Science · #Advanced Neural Network Applications #Adversarial Robustness in Machine Learning #Computation and Language (cs.CL) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG)
  123. ShifCon: Enhancing Non-Dominant Language Capabilities with a Shift-based Multilingual Contrastive Framework
    2024/10/25 by H. Zhang, Zhang, Hengyuan, Chenming Shang +14 · 2 citations
    Arts and Humanities · Social Sciences · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Multilingual Education and Policy #Second Language Learning and Teaching
  124. Black-Box On-Policy Distillation of Large Language Models
    2025/11/13 by Tianxiao Ye, Ye, Tianzhu, Dong Li +9 · 3 citations
    Computer Science · #Topic Modeling #Explainable Artificial Intelligence (XAI) #Multimodal Machine Learning Applications
  125. Learning to Ask Unanswerable Questions for Machine Reading Comprehension
    2019/06/14 by Haichao Zhu, Zhu, Haichao, Li Dong +9 · 1 citation
    Computer Science · #Topic Modeling #Natural Language Processing Techniques #Multimodal Machine Learning Applications
  126. Response Generation by Context-aware Prototype Editing
    2018/06/19 by Yu Wu, Furu Wei, Wu, Yu +9 · 1 citation
    Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Software Engineering Research #Software Testing and Debugging Techniques #Topic Modeling
  127. Democratizing Reasoning Ability: Tailored Learning from Large Language Model
    2023/10/20 by Zhaoyang Wang, Shaohan Huang, Wang, Zhaoyang +19 · 1 citation
    Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Natural Language Processing Techniques #Topic Modeling
  128. When an Image is Worth 1,024 x 1,024 Words: A Case Study in Computational Pathology
    2023/12/06 by Wenhui Wang, Wang, Wenhui, Shuming Ma +11 · 1 citation
    Computer Science · Medicine · #AI in cancer detection #Colorectal Cancer Screening and Detection #Computer Vision and Pattern Recognition (cs.CV) #Digital Imaging for Blood Diseases #FOS: Computer and information sciences
  129. WildLong: Synthesizing Realistic Long-Context Instruction Data at Scale
    2025/02/23 by Jiaxi Li, Xingxing Zhang, Li, Jiaxi +15 · 2 citations
    Computer Science · #Online Learning and Analytics
  130. Enhancing Language Model Rationality with Bi-Directional Deliberation Reasoning
    2024/07/08 by Yadong Zhang, Shaoguang Mao, Zhang, Yadong +11 · 1 citation
    Computer Science · #Topic Modeling #Natural Language Processing Techniques #Multi-Agent Systems and Negotiation
  131. RedStone: Curating General, Code, Math, and QA Data for Large Language Models
    2024/12/04 by Yaoyao Chang, Chang, Yaoyao, Lei Cui +29 · 1 citation
    Computer Science · #Natural Language Processing Techniques #Topic Modeling
  132. Model as a Game: On Numerical and Spatial Consistency for Generative Games
    2025/03/27 by Jingye Chen, Chen, Jingye, Yuzhong Zhao +13 · 1 citation
    Computer Science · Decision Sciences · Engineering · #3D Modeling in Geospatial Applications #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Multi-Agent Systems and Negotiation #Simulation Techniques and Applications
  133. LLM-as-a-Coach: Experiential Learning for Non-Verifiable Tasks
    2026/07/20 by Tianzhu Ye, Li Dong, Guanheng Chen +4 · 1 citation
    #cs.LG #cs.CL
  134. VibeVoice-ASR-BitNet Technical Report
    2026/07/23 by Songchen Xu, Ting Song, Shaohan Huang +10
    #cs.SD #cs.CL #eess.AS