Large Language Models Meet NLP: A Survey

2024/05/21 by Libo Qin, Qiguang Chen, Qin, Libo +15 · 25 citations
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2405.12819

openalex publication_date 2024/05/21 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

While large language models (LLMs) like ChatGPT have shown impressive capabilities in Natural Language Processing (NLP) tasks, a systematic investigation of their potential in this field remains largely unexplored. This study aims to address this gap by exploring the following questions: (1) How are LLMs currently applied to NLP tasks in the literature? (2) Have traditional NLP tasks already been solved with LLMs? (3) What is the future of the LLMs for NLP? To answer these questions, we take the first step to provide a comprehensive overview of LLMs in NLP. Specifically, we first introduce a unified taxonomy including (1) parameter-frozen paradigm and (2) parameter-tuning paradigm to offer a unified perspective for understanding the current progress of LLMs in NLP. Furthermore, we summarize the new frontiers and the corresponding challenges, aiming to inspire further groundbreaking advancements. We hope this work offers valuable insights into the potential and limitations of LLMs, while also serving as a practical guide for building effective LLMs in NLP.

Cited by

• OPOR-Bench: Evaluating Large Language Models on Online Public Opinion Report Generation
• Beware of Reasoning Overconfidence: Pitfalls in the Reasoning Process for Multi-solution Tasks
• A General Highly Accurate Online Planning Method Integrating Large Language Models into Nested Rollout Policy Adaptation for Dialogue Tasks
• LiteCache: A Query Similarity-Driven, GPU-Centric KVCache Subsystem for Efficient LLM Inference
• Relation as a Prior: A Novel Paradigm for LLM-based Document-level Relation Extraction
• SOP-Maze: Evaluating Large Language Models on Complicated Business Standard Operating Procedures
• HALO: Memory-Centric Heterogeneous Accelerator with 2.5D Integration for Low-Batch LLM Inference
• Copy-Paste to Mitigate Large Language Model Hallucinations
• Atomic Thinking of LLMs: Decoupling and Exploring Mathematical Reasoning Abilities
• QFrBLiMP: a Quebec-French Benchmark of Linguistic Minimal Pairs
• AMLA: MUL by ADD in FlashAttention Rescaling
• NorBERTo: A ModernBERT Model Trained for Portuguese with 331 Billion Tokens Corpus
• Can Large Language Models Resolve Semantic Discrepancy in Self-Destructive Subcultures? Evidence from Jirai Kei
• Turk-LettuceDetect: A Hallucination Detection Models for Turkish RAG Applications
• Teaching According to Talents! Instruction Tuning LLMs with Competence-Aware Curriculum Learning
• QCSE: A Pretrained Quantum Context-Sensitive Word Embedding for Natural Language Processing
• LLM-Driven Policy Diffusion: Enhancing Generalization in Offline Reinforcement Learning
• Pruning Weights but Not Truth: Safeguarding Truthfulness While Pruning LLMs
• Youtu-GraphRAG: Vertically Unified Agents for Graph Retrieval-Augmented Complex Reasoning
• Quantized but Deceptive? A Multi-Dimensional Truthfulness Evaluation of Quantized LLMs
• MSRS: Adaptive Multi-Subspace Representation Steering for Attribute Alignment in Large Language Models
• Rethinking Tokenization for Rich Morphology: The Dominance of Unigram over BPE and Morphological Alignment
• Latent Knowledge Scalpel: Precise and Massive Knowledge Editing for Large Language Models
• SPARQL Query Generation with LLMs: Measuring the Impact of Training Data Memorization and Knowledge Injection
• Real-World Summarization: When Evaluation Reaches Its Limits

Related

• Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond
• Large Language Models: A Survey
• A Survey of Large Language Models
• ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning
• Comparative Analysis of CHATGPT and the evolution of language models