Foundations of Large Language Models

2025/01/16 by Tong Xiao, Jingbo Zhu, Xiao, Tong +1 · 28 voices · 6 citations
Computer Science · #Natural Language Processing Techniques #Topic Modeling #cs.AI #cs.CL #cs.LG

paper · pdf · doi:10.48550/arxiv.2501.09223

openalex publication_date 2025/01/16 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/02

Abstract

This is a book about large language models. As indicated by the title, it primarily focuses on foundational concepts rather than comprehensive coverage of all cutting-edge technologies. The book is structured into five main chapters, each exploring a key area: pre-training, generative models, prompting, alignment, and inference. It is intended for college students, professionals, and practitioners in natural language processing and related fields, and can serve as a reference for anyone interested in large language models.

Cited by

• LLMs Can Assist with Proposal Selection at Large User Facilities
• Probing Preference Representations: A Multi-Dimensional Evaluation and Analysis Method for Reward Models
• Efficiency vs. Alignment: Investigating Safety and Fairness Risks in Parameter-Efficient Fine-Tuning of LLMs
• LSM-MS2: A Foundation Model Bridging Spectral Identification and Biological Interpretation
• SasAgent: Multi-Agent AI System for Small-Angle Scattering Data Analysis
• Learning to Shop Like Humans: A Review-driven Retrieval-Augmented Recommendation Framework with LLMs

Discussions

• Foundations of Large Language Models [hn, 219 points, 20 comments]
• Foundations of Large Language Models Tong Xiao, Jingbo Zhu This is a book about large language models... arxiv.org/abs/2501.09223 [bsky, 13 points, 1 comments]
• Nice and fresh content to understand how Large Language Models work: arxiv.org/abs/2501.09223 #LLM #NLP [bsky, 9 points, 0 comments]
• A new book on LLMs: Foundations of Large Language Models by Tong Xiao and Jingbo Zhu. arxiv.org/abs/2501.09223 Covers topics such as pretraining, scaling laws, in-context learning, COT, RLHF, etc. H [bsky, 5 points, 0 comments]
• Foundations of Large Language Models [hn, 3 points, 0 comments]
• Something I add to my when-I-am-retired-and-live-in-a-summer-house-in-italy-reading-list reading list. ~250 pages about LLMs FOR FREE! arxiv.org/pdf/2501.09223 [bsky, 1 points, 0 comments]
• arxiv.org/pdf/2501.09223 [bsky, 1 points, 0 comments]
• Good Creative Commons reference book on LLM. arxiv.org/pdf/2501.09223 [bsky, 1 points, 0 comments]
• Foundations of Large language models arxiv.org/abs/2501.09223 For those looking for a deep dive into LLM's [bsky, 1 points, 0 comments]
• Foundations of Large Language Models, by Tong Xiao, Jingbo Zhu / arXiv [This is a book about large language models. … it primarily focuses on foundational concepts rather than comprehensive coverage o [bsky, 1 points, 0 comments]
• Time to read arxiv.org/abs/2501.09223 [bsky, 1 points, 0 comments]
• Foundations of Large Language Models arxiv.org/pdf/2501.09223 [bsky, 1 points, 0 comments]
• A detailed technical FREE book on the Large Language Model Foundations @ arxiv.org/pdf/2501.09223 Made a notebookLM of the PDF @ notebooklm.google.com/notebook/fa3... #LLM [bsky, 1 points, 1 comments]
• arxiv.org/pdf/2501.09223 [bsky, 1 points, 0 comments]
• Foundations of Large Language Models https://arxiv.org/abs/2501.09223 (https://news.ycombinator.com/item?id=42799629) [bsky, 0 points, 0 comments]
• #LLM #book arxiv.org/abs/2501.09223 [bsky, 0 points, 0 comments]
• Foundations of Large Language Models arxiv.org/abs/2501.09223 [bsky, 0 points, 0 comments]
• 🤖 Fascinating insights into the building blocks of AI language understanding, shedding light on its potential and limitations. #ai Foundations of Large Language Models [bsky, 0 points, 0 comments]
• Check Out at: arxiv.org/pdf/2501.09223 [bsky, 0 points, 0 comments]
• arxiv.org/abs/2501.0... [bsky, 0 points, 0 comments]
• A really good paper: arxiv.org/abs/2501.09223 [bsky, 0 points, 0 comments]
• Full book download below 👇 https://buff.ly/3Ckonof How do you see human-AI collaboration evolving in the next 5 years? Let's discuss! #AI #MachineLearning #FutureOfWork [bsky, 0 points, 0 comments]
• Ever wondered how Large Language Models really work? This book thats available on arXiv breaks it all down, from pre-training to alignment. Perfect for anyone diving into LLMs! Check it out ⬇️ arxiv [bsky, 0 points, 0 comments]
• Foundations of Large Language Models https://arxiv.org/abs/2501.09223 (https://news.ycombinator.com/item?id=42799629) [bsky, 0 points, 0 comments]
• Foundations of Large Language Models [bsky, 0 points, 0 comments]
• Foundations of Large Language Models #HackerNews arxiv.org/abs/... [bsky, 0 points, 0 comments]
• Foundations of Large Language Models https://arxiv.org/abs/2501.09223 https://news.ycombinator.com/item?id=42799629 [bsky, 0 points, 0 comments]
• Foundations of Large Language Models https://arxiv.org/abs/2501.09223 [comments] [52 points] [bsky, 0 points, 0 comments]

Related

• History, Development, and Principles of Large Language Models-An Introductory Survey
• Formal Aspects of Language Modeling
• A Primer on Large Language Models and their Limitations
• Efficient Large Language Models: A Survey
• Decoding Large-Language Models: A Systematic Overview of Socio-Technical Impacts, Constraints, and Emerging Questions