RoBERTa: A Robustly Optimized BERT Pretraining Approach

2019/07/26 by Liu, Yinhan, Ott, Myle, Goyal, Naman +7 · 796 citations
#Computation and Language (cs.CL) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.1907.11692

Abstract

Language model pretraining has led to significant performance gains but careful comparison between different approaches is challenging. Training is computationally expensive, often done on private datasets of different sizes, and, as we will show, hyperparameter choices have significant impact on the final results. We present a replication study of BERT pretraining (Devlin et al., 2019) that carefully measures the impact of many key hyperparameters and training data size. We find that BERT was significantly undertrained, and can match or exceed the performance of every model published after it. Our best model achieves state-of-the-art results on GLUE, RACE and SQuAD. These results highlight the importance of previously overlooked design choices, and raise questions about the source of recently reported improvements. We release our models and code.

Cited by

• Fake News Classification in Urdu: A Domain Adaptation Approach for a Low-Resource Language
• FRoD: Full-Rank Efficient Fine-Tuning with Rotational Degrees for Fast Convergence
• Securing the AI Supply Chain: What Can We Learn From Developer-Reported Security Issues and Solutions of AI Projects?
• RobustMask: Certified Robustness against Adversarial Neural Ranking Attack via Randomized Masking
• Pointer-Augmented Autoregressive Generation of Patent Claims with Joint Topology and Content Decoding
• Towards Efficient Post-Training via Fourier-Driven Adapter Architectures
• BERT-based Models vs. Large Language Models for Low-Resource Named Entity Recognition: A Comparative Study on Marathi
• HiTMS: A High-Throughput Multi-Stream Linguistic Steganography Framework
• Modeling Stylistic Co-evolution in Symbolic Music Heritage Collections
• Where meaning lives: Layer-wise accessibility of psycholinguistic features in encoder and decoder language models
• Similarity All The Way Up: Multilingual Generalization in LLMs Relies on Language-Level Similarity Structures
• SourceMinds at CheckThat! 2026: NLI-Grounded Citation Auditing in a Multi-Agent Pipeline for Full Fact-Checking Article Generation
• Reading Without a Reader: Large Language Models Collapse Reading and Writing into a Single Entangled Code
• PatchWorld: Gradient-Free Optimization of Executable World Models for Agent Environments
• Reverso: Efficient Time Series Foundation Models for Zero-shot Forecasting
• TrackTeller: Temporal Multimodal 3D Grounding for Behavior-Dependent Object References
• Fixed-Budget Parameter-Efficient Training with Frozen Encoders Improves Multimodal Chest X-Ray Classification
• animal2vec and MeerKAT: A self-supervised transformer for rare-event raw audio input and a large-scale reference dataset for bioacoustics
• Foundation Model-based Evaluation of Neuropsychiatric Disorders: A Lifespan-Inclusive, Multi-Modal, and Multi-Lingual Study
• ETP-R1: Evolving Topological Planning with Reinforcement Fine-tuning for Vision-Language Navigation in Continuous Environments
• Advancing Multimodal Teacher Sentiment Analysis:The Large-Scale T-MED Dataset & The Effective AAM-TSA Model
• FaithLens: Detecting and Explaining Faithfulness Hallucination
• Large Language Models Approach Expert Pedagogical Quality in Math Tutoring but Differ in Instructional and Linguistic Profiles
• Event Extraction in Large Language Model
• A Large-Language-Model Framework for Automated Humanitarian Situation Reporting
• CienaLLM: Generative Climate-Impact Extraction from News Articles with Autoregressive LLMs
• A Comparative Study of Light-weight Language Models for PII Masking and their Deployment for Real Conversational Texts
• Easy Adaptation: An Efficient Task-Specific Knowledge Injection Method for Large Models in Resource-Constrained Environments
• Toward Ethical AI Through Bayesian Uncertainty in Neural Question Answering
• Confidence-Credibility Aware Weighted Ensembles of Small LLMs Outperform Large LLMs in Emotion Detection
• Grammar-Forced Translation of Natural Language to Temporal Logic using LLMs
• Decoding Fake Narratives in Spreading Hateful Stories: A Dual-Head RoBERTa Model with Multi-Task Learning
• ModelTables: A Corpus of Tables about Models
• TabReX : Tabular Referenceless eXplainable Evaluation
• Beyond Fast and Slow: Cognitive-Inspired Elastic Reasoning for Large Language Models
• SeBERTis: A Framework for Producing Classifiers of Security-Related Issue Reports
• TiME: Tiny Monolingual Encoders for Efficient NLP Pipelines
• C-ing Clearly: Enhanced Binary Code Explanations using C code
• TEMP: A Memory Efficient Physical-aware Tensor Partition-Mapping Framework on Wafer-scale Chips
• Explainable Ethical Assessment on Human Behaviors by Generating Conflicting Social Norms
• TorchTraceAP: A New Benchmark Dataset for Detecting Performance Anti-Patterns in Computer Vision Models
• Verifying Rumors via Stance-Aware Structural Modeling
• Improving Recursive Transformers with Mixture of LoRAs
• Fine-Tuning Causal LLMs for Text Classification: Embedding-Based vs. Instruction-Based Approaches
• SafeGen: Embedding Ethical Safeguards in Text-to-Image Generation
• Leader-driven or Leaderless: How Participation Structure Sustains Engagement and Shapes Narratives in Online Hate Communities
• Sliced ReLU attention: Quasi-linear contextual expressivity via sorting
• Multi-Intent Spoken Language Understanding: Methods, Trends, and Challenges
• SciLaD: A Large-Scale, Transparent, Reproducible Dataset for Natural Scientific Language Processing
• Explanation Bias is a Product: Revealing the Hidden Lexical and Position Preferences in Post-Hoc Feature Attribution
• Immutable Explainability: Fuzzy Logic and Blockchain for Verifiable Affective AI
• MeViS: A Multi-Modal Dataset for Referring Motion Expression Video Segmentation
• Local LLM Ensembles for Zero-shot Portuguese Named Entity Recognition
• SCOUT: A Defense Against Data Poisoning Attacks in Fine-Tuned Language Models
• Interpreto: An Explainability Library for Transformers
• Don't Throw Away Your Beams: Improving Consistency-based Uncertainties in LLMs via Beam Search
• Belief Is All You Need: Modeling Narrative Archetypes in Conspiratorial Discourse
• Adaptive Regularized Newton Method with Inexact Hessian
• The Erosion of LLM Signatures: Can We Still Distinguish Human and LLM-Generated Scientific Ideas After Iterative Paraphrasing?
• Hybrid Attribution Priors for Explainable and Robust Model Training
• gHAWK: Local and Global Structure Encoding for Scalable Training of Graph Neural Networks on Knowledge Graphs
• ValuePilot: A Two-Phase Framework for Value-Driven Decision-Making
• TROJail: Trajectory-Level Optimization for Multi-Turn Large Language Model Jailbreaks with Process Rewards
• MoCoRP: Modeling Consistent Relations between Persona and Response for Persona-based Dialogue
• FOAM: Blocked State Folding for Memory-Efficient LLM Training
• Mechanistic Interpretability of GPT-2: Lexical and Contextual Layers in Sentiment Analysis
• Patronus: Identifying and Mitigating Transferable Backdoors in Pre-trained Language Models
• Always Keep Your Promises: A Model-Agnostic Attribution Algorithm for Neural Networks
• Modeling Contextual Passage Utility for Multihop Question Answering
• Technical Report of Nomi Team in the Environmental Sound Deepfake Detection Challenge 2026
• Fine-Tuning BERT for Domain-Specific Question Answering: Toward Educational NLP Resources at University Scale
• LLMs Know More Than Words: A Genre Study with Syntax, Metaphor & Phonetics
• DAMASHA: Detecting AI in Mixed Adversarial Texts via Segmentation with Human-interpretable Attribution
• Mapping CVEs to MITRE ATT&CK Techniques: A Curated Gold-Set Classifier and the Limits of LLM-Assisted Label Expansion
• OsmT: Bridging OpenStreetMap Queries and Natural Language with Open-source Tag-aware Language Models
• E3AD: An Emotion-Aware Vision-Language-Action Model for Human-Centric End-to-End Autonomous Driving
• BERnaT: Basque Encoders for Representing Natural Textual Diversity
• Tutorial on Large Language Model-Enhanced Reinforcement Learning for Wireless Networks
• Fine-grained Narrative Classification in Biased News Articles
• Action Anticipation at a Glimpse: To What Extent Can Multimodal Cues Replace Video?
• Agent-Based Modular Learning for Multimodal Emotion Recognition in Human-Agent Systems
• Enhancing Job Matching: Occupation, Skill and Qualification Linking with the ESCO and EQF taxonomies
• ESACT: An End-to-End Sparse Accelerator for Compute-Intensive Transformers via Local Similarity
• Understanding Down Syndrome Stereotypes in LLM-Based Personas
• Story2MIDI: Emotionally Aligned Music Generation from Text
• Learned-Rule-Augmented Large Language Model Evaluators
• MAC-SLU: Multi-Intent Automotive Cabin Spoken Language Understanding Benchmark
• Scaling and context steer LLMs along the same computational path as the human brain
• DyFuLM: An Advanced Multimodal Framework for Sentiment Analysis
• MARSAD: A Multi-Functional Tool for Real-Time Social Media Analysis
• Handwritten Text Recognition for Low Resource Languages
• TempPerturb-Eval: On the Joint Effects of Internal Temperature and External Perturbations in RAG Robustness
• Open-Set Domain Adaptation Under Background Distribution Shift: Challenges and A Provably Efficient Solution
• M4-BLIP: Advancing Multi-Modal Media Manipulation Detection through Face-Enhanced Local Analysis
• Stay Unique, Stay Efficient: Preserving Model Personality in Multi-Task Merging
• Causal Invariance and Counterfactual Learning Driven Cooperative Game for Multi-Label Classification
• FastPOS: Language-Agnostic Scalable POS Tagging Framework Low-Resource Use Case
• Financial Text Classification Based On rLoRA Finetuning On Qwen3-8B model
• SatireDecoder: Visual Cascaded Decoupling for Enhancing Satirical Image Comprehension
• Simplex-Optimized Hybrid Ensemble for Large Language Model Text Detection Under Generative Distribution Drif
• Comparative Analysis of 47 Context-Based Question Answer Models Across 8 Diverse Datasets
• CodeFlowLM: Incremental Just-In-Time Defect Prediction with Pretrained Language Models and Exploratory Insights into Defect Localization
• Toward Automatic Safe Driving Instruction: A Large-Scale Vision Language Model Approach
• Tourism Question Answer System in Indian Language using Domain-Adapted Foundation Models
• Decoding the Past: Explainable Machine Learning Models for Dating Historical Texts
• Pooling Attention: Evaluating Pretrained Transformer Embeddings for Deception Classification
• BanglaSentNet: An Explainable Hybrid Deep Learning Framework for Multi-Aspect Sentiment Analysis with Cross-Domain Transfer Learning
• FedSGT: Exact Federated Unlearning via Sequential Group-based Training
• Analyzing Image Beyond Visual Aspect: Image Emotion Classification via Multiple-Affective Captioning
• Beyond Real versus Fake Towards Intent-Aware Video Analysis
• Exploring Automated Recognition of Instructional Activity and Discourse from Multimodal Classroom Data
• PEFT-Bench: A Parameter-Efficient Fine-Tuning Methods Benchmark
• Referring Video Object Segmentation with Cross-Modality Proxy Queries
• AerialMind: Towards Referring Multi-Object Tracking in UAV Scenarios
• From Words to Wisdom: Discourse Annotation and Baseline Models for Student Dialogue Understanding
• "When Data is Scarce, Prompt Smarter"... Approaches to Grammatical Error Correction in Low-Resource Settings
• Schema Matching on Graph: Iterative Graph Exploration for Efficient and Explainable Data Integration
• Efficient Covariance Estimation for Sparsified Functional Data
• A Lightweight Approach to Detection of AI-Generated Texts Using Stylometric Features
• A Multiscale Geometric Method for Capturing Relational Topic Alignment
• MultiGA: Leveraging Multi-Source Seeding in Genetic Algorithms
• Large Language Models for Sentiment Analysis to Detect Social Challenges: A Use Case with South African Languages
• Hallucinate Less by Thinking More: Aspect-Based Causal Abstention for Large Language Models
• Bridging Visual Affective Gap: Borrowing Textual Knowledge by Learning from Noisy Image-Text Pairs
• CREST: Improving Interpretability and Effectiveness of Troubleshooting at Ericsson through Criterion-Specific Trouble Report Retrieval
• Vision-Motion-Reference Alignment for Referring Multi-Object Tracking via Multi-Modal Large Language Models
• Senti-iFusion: An Integrity-centered Hierarchical Fusion Framework for Multimodal Sentiment Analysis under Uncertain Modality Missingness
• ARQUSUMM: Argument-aware Quantitative Summarization of Online Conversations
• Exploring Scientific Debt: Harnessing AI for SATD Identification in Scientific Software
• Pillar-0: A New Frontier for Radiology Foundation Models
• Stable Coresets via Posterior Sampling: Aligning Induced and Full Loss Landscapes
• PersonalizedRouter: Personalized LLM Routing via Graph-based User Preference Modeling
• When Structure Doesn't Help: LLMs Do Not Read Text-Attributed Graphs as Effectively as We Expected
• Revisiting Audio-language Pretraining for Learning General-purpose Audio Representation
• Learning from Sufficient Rationales: Analysing the Relationship Between Explanation Faithfulness and Token-level Regularisation Strategies
• ILoRA: Federated Learning with Low-Rank Adaptation for Heterogeneous Client Aggregation
• AVATAAR: Agentic Video Answering via Temporal Adaptive Alignment and Reasoning
• Standardising the NLP Workflow: A Framework for Reproducible Linguistic Analysis
• An Optimized Machine Learning Classifier for Detecting Fake Reviews Using Extracted Features
• Auden-Voice: General-Purpose Voice Encoder for Speech and Language Understanding
• Hierarchical Token Prepending: Enhancing Information Flow in Decoder-based LLM Embeddings
• Continuous Vision-Language-Action Co-Learning with Semantic-Physical Alignment for Behavioral Cloning
• Steganographic Backdoor Attacks in NLP: Ultra-Low Poisoning and Defense Evasion
• GRPO Privacy Is at Risk: A Membership Inference Attack Against Reinforcement Learning With Verifiable Rewards
• RoboTidy : A 3D Gaussian Splatting Household Tidying Benchmark for Embodied Navigation and Action
• ForgeDAN: An Evolutionary Framework for Jailbreaking Aligned Large Language Models
• Unifying points of interest taxonomies: mapping OpenStreetMap tags to the Foursquare category system
• 3DAlign-DAER: Dynamic Attention Policy and Efficient Retrieval Strategy for Fine-grained 3D-Text Alignment at Scale
• Zero-Shot Grammar Competency Estimation Using Large Language Model Generated Pseudo Labels
• BeDiscovER: The Benchmark of Discourse Understanding in the Era of Reasoning Language Models
• BIRD: Bronze Inscription Restoration and Dating
• Understanding the Complexities of Responsibly Sharing NSFW Content Online
• Reproducibility Report: Test-Time Training on Nearest Neighbors for Large Language Models
• Concept-Based Interpretability for Toxicity Detection
• ViConBERT: Context-Gloss Aligned Vietnamese Word Embedding for Polysemous and Sense-Aware Representations
• PRISM of Opinions: A Persona-Reasoned Multimodal Framework for User-centric Conversational Stance Detection
• MME-RAG: Multi-Manager-Expert Retrieval-Augmented Generation for Fine-Grained Entity Recognition in Task-Oriented Dialogues
• CURENet: Combining Unified Representations for Efficient Chronic Disease Prediction
• Large Language Models and 3D Vision for Intelligent Robotic Perception and Autonomy
• Analogical Structure, Minimal Contextual Cues and Contrastive Distractors: Input Design for Sample-Efficient Linguistic Rule Induction
• TubeRMC: Tube-conditioned Reconstruction with Mutual Constraints for Weakly-supervised Spatio-Temporal Video Grounding
• Persona-Aware Alignment Framework for Personalized Dialogue Generation
• Boosting In-Silicon Directed Evolution with Fine-Tuned Protein Language Model and Tree Search
• ACT as Human: Multimodal Large Language Model Data Annotation with Critical Thinking
• Contextual morphologically-guided tokenization for Latin encoder models
• SynClaimEval: A Framework for Evaluating the Utility of Synthetic Data in Long-Context Claim Verification
• End-to-end Contrastive Language-Speech Pretraining Model For Long-form Spoken Question Answering
• Zero-Order Sharpness-Aware Minimization
• EVADE: LLM-Based Explanation Generation and Validation for Error Detection in NLI
• CMI-MTL: Cross-Mamba interaction based multi-task learning for medical visual question answering
• Why Should the Server Do It All?: A Scalable, Versatile, and Model-Agnostic Framework for Server-Light DNN Inference over Massively Distributed Clients via Training-Free Intermediate Feature Compression
• Bot Meets Shortcut: How Can LLMs Aid in Handling Unknown Invariance OOD Scenarios?
• H-Model: Dynamic Neural Architectures for Adaptive Processing
• On the Interplay between Positional Encodings, Morphological Complexity, and Word Order Flexibility
• Estranged Predictions: Measuring Semantic Category Disruption with Masked Language Modelling
• CC30k: A Citation Contexts Dataset for Reproducibility-Oriented Sentiment Analysis
• Let's Split Up: Zero-Shot Classifier Edits for Fine-Grained Video Understanding
• SPOT: An Annotated French Corpus and Benchmark for Detecting Critical Interventions in Online Conversations
• Selecting Auxiliary Data via Neural Tangent Kernels for Low-Resource Domains
• Beyond Detection: Exploring Evidence-based Multi-Agent Debate for Misinformation Intervention and Persuasion
• Do LLMs Feel? Teaching Emotion Recognition with Prompts, Retrieval, and Curriculum Learning
• Have we reached the beginning of the end for review papers?
• HLPD: Aligning LLMs to Human Language Preference for Machine-Revised Text Detection
• QUARK: Quantization-Enabled Circuit Sharing for Transformer Acceleration by Exploiting Common Patterns in Nonlinear Operations
• Evaluating Language Model Applications for Identifying Solution-Related Content in Issue Report Discussions
• Comparing Reconstruction Attacks on Pretrained Versus Full Fine-tuned Large Language Model Embeddings on Homo Sapiens Splice Sites Genomic Data
• SugarTextNet: A Transformer-Based Framework for Detecting Sugar Dating-Related Content on Social Media with Context-Aware Focal Loss
• ELEGANCE: Efficient LLM Guidance for Audio-Visual Target Speech Extraction
• Chain-of-Thought as a Lens: Evaluating Structured Reasoning Alignment between Human Preferences and Large Language Models
• LLM-Driven Completeness and Consistency Evaluation for Cultural Heritage Data Augmentation in Cross-Modal Retrieval
• ReMoD: Rethinking Modality Contribution in Multimodal Stance Detection via Dual Reasoning
• DiagnoLLM: A Hybrid Bayesian Neural Language Framework for Interpretable Disease Diagnosis
• DRAGON: Guard LLM Unlearning in Context via Negative Detection and Reasoning
• A Metamorphic Testing Perspective on Knowledge Distillation for Language Models of Code: Does the Student Deeply Mimic the Teacher?
• Evaluating Subword Tokenization Techniques for Bengali: A Benchmark Study with BengaliBPE
• ManufactuBERT: Efficient Continual Pretraining for Manufacturing
• Search Is Not Retrieval: Decoupling Semantic Matching from Contextual Assembly in RAG
• When retrieval outperforms generation: Dense evidence retrieval for scalable fake news detection
• Ground-Truth Subgraphs for Better Training and Evaluation of Knowledge Graph Augmented LLMs
• On the Brittleness of CLIP Text Encoders
• OMPILOT: Harnessing Transformer Models for Auto Parallelization to Shared Memory Computing Paradigms
• LFC-DA: Logical Formula-Controlled Data Augmentation for Enhanced Logical Reasoning
• Multi-Object Tracking Retrieval with LLaVA-Video: A Training-Free Solution to MOT25-StAG Challenge
• Emotion Recognition in Multi-Speaker Conversations through Speaker Identification, Knowledge Distillation, and Hierarchical Fusion
• An Interdisciplinary and Cross-Task Review on Missing Data Imputation
• A Large Scale Study of AI-based Binary Function Similarity Detection Techniques for Security Researchers and Practitioners
• Data-Efficient Adaptation and a Novel Evaluation Method for Aspect-based Sentiment Analysis
• ProM3E: Probabilistic Masked MultiModal Embedding Model for Ecology
• ConMeZO: Adaptive Descent-Direction Sampling for Gradient-Free Finetuning of Large Language Models
• Optimal Singular Damage: Efficient LLM Inference in Low Storage Regimes
• ECO Decoding: Entropy-Based Control for Controllability and Fluency in Controllable Dialogue Generation
• Effective Test-Time Scaling of Discrete Diffusion through Iterative Refinement
• 3EED: Ground Everything Everywhere in 3D
• DEER: Disentangled Mixture of Experts with Instance-Adaptive Routing for Generalizable Machine-Generated Text Detection
• G2rammar: Bilingual Grammar Modeling for Enhanced Text-attributed Graph Learning
• Multi-refined Feature Enhanced Sentiment Analysis Using Contextual Instruction
• Multilingual BERT language model for medical tasks: Evaluation on domain-specific adaptation and cross-linguality
• FedAdamW: A Communication-Efficient Optimizer with Convergence and Generalization Guarantees for Federated Large Models
• From the Rock Floor to the Cloud: A Systematic Survey of State-of-the-Art NLP in Battery Life Cycle
• Cognitive Alignment in Personality Reasoning: Leveraging Prototype Theory for MBTI Inference
• FedMuon: Accelerating Federated Learning with Matrix Orthogonalization
• Exploring the Utilities of the Rationales from Large Language Models to Enhance Automated Essay Scoring
• Calibration Across Layers: Understanding Calibration Evolution in LLMs
• Elastic Architecture Search for Efficient Language Models
• Enhancing Sentiment Classification with Machine Learning and Combinatorial Fusion
• Masked Diffusion Captioning for Visual Feature Learning
• An All-Reduce Compatible Top-K Compressor for Communication-Efficient Distributed Learning
• Encoder-Decoder or Decoder-Only? Revisiting Encoder-Decoder Large Language Model
• CyberNER: A Harmonized STIX Corpus for Cybersecurity Named Entity Recognition
• On the Role of Context for Discourse Relation Classification in Scientific Writing
• Don't Let It Fade: Preserving Edits in Diffusion Language Models via Token Timestep Allocation
• RCScore: Quantifying Response Consistency in Large Language Models
• ReaKase-8B: Legal Case Retrieval via Knowledge and Reasoning Representations with LLMs
• AttnCache: Accelerating Self-Attention Inference for LLM Prefill via Attention Cache
• Diagnosing Fine-Grained Inconsistency Classification in Financial Disclosure Text
• AtmosERC: Modeling Dialogue-Level Affective Atmosphere for Emotion Recognition in Conversation
• Between Gradient and Natural Gradient: A Continuum of LoRA Initializations
• Enhancing Generative Information Extraction with Two-step Validation: A Product Attribute Use Case
• Persuasion Index: A Theory-Guided Framework for Persuasion Analysis
• Gemini Embedding 2: A Native Multimodal Embedding Model from Gemini
• CatPath‐GPT: A Mixture of Experts System for Computational Catalyst Design
• Hallucination Benchmark for Speech Foundation Models
• DP-MGTD: Privacy-Preserving Machine-Generated Text Detection via Adaptive Differentially Private Entity Sanitization
• Machine learning based screening of potential paper mill publications in cancer research: methodological and cross sectional study
• Bots into the Fediverse
• Stay Tuned: Improving Sentiment Analysis and Stance Detection Using Large Language Models
• Testing Cross-Lingual Text Comprehension In LLMs Using Next Sentence Prediction
• A Survey on Unlearning in Large Language Models
• Dynamically Weighted Momentum with Adaptive Step Sizes for Efficient Deep Network Training
• SmoothGuard: Defending Multimodal Large Language Models with Noise Perturbation and Clustering Aggregation
• Streamlining business functions in official statistical production with Machine Learning
• MERGE: Minimal Expression-Replacement GEneralization Test for Natural Language Inference
• Squrve: A Unified and Modular Framework for Complex Real-World Text-to-SQL Tasks
• COMMUNITYNOTES: A Dataset for Exploring the Helpfulness of Fact-Checking Explanations
• Utilising Large Language Models for Generating Effective Counter Arguments to Anti-Vaccine Tweets
• ReviewGuard: Enhancing Deficient Peer Review Detection via LLM-Driven Data Augmentation
• Sentiment and Volatility in Financial Markets: A Review of BERT and GARCH Applications during Geopolitical Crises
• Eigen-Value: Efficient Domain-Robust Data Valuation via Eigenvalue-Based Approach
• Beyond Higher Rank: Token-wise Input-Output Projections for Efficient Low-Rank Adaptation
• Capsule Network-Based Multimodal Fusion for Mortgage Risk Assessment from Unstructured Data Sources
• Exploration of Summarization by Generative Language Models for Automated Scoring of Long Essays
• In Generative AI We (Dis)Trust? Computational Analysis of Trust and Distrust in Reddit Discussions
• Agentic Meta-Orchestrator for Multi-task Copilots
• SALSA: Single-pass Autoregressive LLM Structured Classification
• Chitchat with AI: Understand the supply chain carbon disclosure of companies worldwide through Large Language Model
• Supervised Fine-Tuning or In-Context Learning? Evaluating LLMs for Clinical NER
• DETECT: Determining Ease and Textual Clarity of German Text Simplifications
• The Lossy Horizon: Error-Bounded Predictive Coding for Lossy Text Compression (Episode I)
• Foundation of Intelligence: Review of Math Word Problems from Human Cognition Perspective
• ArchISMiner: A Framework for Automatic Mining of Architectural Issue-Solution Pairs from Online Developer Communities
• HalleluBERT: Let every token that has meaning bear its weight
• SindBERT, the Sailor: Charting the Seas of Turkish NLP
• α-LoRA: Effective Fine-Tuning via Base Model Rescaling
• Model Merging with Functional Dual Anchors
• Large Language Models Meet Text-Attributed Graphs: A Survey of Integration Frameworks and Applications
• Bridging Language Gaps with Adaptive RAG: Improving Indonesian Language Question Answering
• Memory Constrained Dynamic Subnetwork Update for Transfer Learning
• Compress to Impress: Efficient LLM Adaptation Using a Single Gradient Step on 100 Samples
• Systematic Evaluation of Uncertainty Estimation Methods in Large Language Models
• From Masks to Worlds: A Hitchhiker's Guide to World Models
• NeoDictaBERT: Pushing the Frontier of BERT models for Hebrew
• Forging GEMs: Advancing Greek NLP through Quality-Based Corpus Curation
• Do Prompts Reshape Representations? An Empirical Study of Prompting Effects on Embeddings
• Study of Training Dynamics for Memory-Constrained Fine-Tuning
• Style Attack Disguise: When Fonts Become a Camouflage for Adversarial Intent
• Latent Space Factorization in LoRA
• COLA: Continual Learning via Autoencoder Retrieval of Adapters
• Automated HIV Screening on Dutch Electronic Health Records with Large Language Models
• Learning Noise-Resilient and Transferable Graph-Text Alignment via Dynamic Quality Assessment
• Knowledge Distillation of Uncertainty using Deep Latent Factor Model
• Difficulty-Controllable Multiple-Choice Question Generation Using Large Language Models and Direct Preference Optimization
• Selecting and Combining Large Language Models for Scalable Code Clone Detection
• VeFA: Vector-Based Feature Space Adaptation for Robust Model Fine-Tuning
• Tibetan Language and AI: A Comprehensive Survey of Resources, Methods and Challenges
• DiSRouter: Distributed Self-Routing for LLM Selections
• Analysing Knowledge Construction in Online Learning: Adapting the Interaction Analysis Model for Unstructured Large-Scale Discourse
• Evaluating LLM Story Generation through Large-scale Network Analysis of Social Structures
• IMB: An Italian Medical Benchmark for Question Answering
• Benchmarking On-Device Machine Learning on Apple Silicon with MLX
• GPTFace: Generative Pre-training of Facial-Linguistic Transformer by Span Masking and Weakly Correlated Text-image Data
• Combining Distantly Supervised Models with In Context Learning for Monolingual and Cross-Lingual Relation Extraction
• Towards Fast LLM Fine-tuning through Zeroth-Order Optimization with Projected Gradient-Aligned Perturbations
• NeuroAda: Activating Each Neuron's Potential for Parameter-Efficient Fine-Tuning
• HyperDiffusionFields (HyDiF): Diffusion-Guided Hypernetworks for Learning Implicit Molecular Neural Fields
• A Benchmark Dataset And LLMs Comparison For NFR Classification With Explainable AI
• AcademicEval: Live Long-Context LLM Benchmark
• DETree: DEtecting Human-AI Collaborative Texts via Tree-Structured Hierarchical Representation Learning
• Disparities in Multilingual LLM-Based Healthcare Q&A
• Efficient Toxicity Detection in Gaming Chats: A Comparative Study of Embeddings, Fine-Tuned Transformers and LLMs
• Explainability of Large Language Models: Opportunities and Challenges toward Generating Trustworthy Explanations
• Robustness in Text-Attributed Graph Learning: Insights, Trade-offs, and New Defenses
• HGAdapter: Hypergraph-based Adapters in Language Models for Code Summarization and Clone Detection
• Bits Leaked per Query: Information-Theoretic Bounds on Adversarial Attacks against LLMs
• Uncovering Brain-Like Hierarchical Patterns in Vision-Language Models through fMRI-Based Neural Encoding
• All You Need is One: Capsule Prompt Tuning with a Single Vector
• MOSAIC: Masked Objective with Selective Adaptation for In-domain Contrastive Learning
• Cross-Genre Authorship Attribution via LLM-Based Retrieve-and-Rerank
• Advances in Pre-trained Language Models for Domain-Specific Text Classification: A Systematic Review
• Scaling Beyond Context: A Survey of Multimodal Retrieval-Augmented Generation for Document Understanding
• Zeroth-Order Sharpness-Aware Learning with Exponential Tilting
• Direct Preference Optimization with Unobserved Preference Heterogeneity: The Necessity of Ternary Preferences
• The Cultural Mapping and Pattern Analysis (CMAP) Visualization Toolkit: Open Source Text Analysis for Qualitative and Computational Social Science
• Leveraging LLMs for Context-Aware Implicit Textual and Multimodal Hate Speech Detection
• Policy Transfer for Continuous-Time Reinforcement Learning: A (Rough) Differential Equation Approach
• Latent Topic Synthesis: Leveraging LLMs for Electoral Ad Analysis
• A Generalizable Rhetorical Strategy Annotation Model Using LLM-based Debate Simulation and Labelling
• LLMs as Scalable, General-Purpose Simulators For Evolving Digital Agent Training
• MetaBench: A Multi-task Benchmark for Assessing LLMs in Metabolomics
• AI-Powered Early Diagnosis of Mental Health Disorders from Real-World Clinical Conversations
• A Hard-Label Black-Box Evasion Attack against ML-based Malicious Traffic Detection Systems
• Detecting Early and Implicit Suicidal Ideation via Longitudinal and Information Environment Signals on Social Media
• Purifying Task Vectors in Knowledge-Aware Subspace for Model Merging
• Hierarchical Semantic Retrieval with Cobweb
• CAST: Compositional Analysis via Spectral Tracking for Understanding Transformer Layer Functions
• Signature in Code Backdoor Detection, how far are we?
• Document Intelligence in the Era of Large Language Models: A Survey
• How Sampling Affects the Detectability of Machine-written texts: A Comprehensive Study
• MedREK: Retrieval-Based Editing for Medical LLMs with Key-Aware Prompts
• Hypernetworks for Perspectivist Adaptation
• Chinese ModernBERT with Whole-Word Masking
• Dual Learning with Dynamic Knowledge Distillation and Soft Alignment for Partially Relevant Video Retrieval
• ProtoSiTex: Learning Semi-Interpretable Prototypes for Multi-label Text Classification
• When Personalization Tricks Detectors: The Feature-Inversion Trap in Machine-Generated Text Detection
• Traveling Salesman-Based Token Ordering Improves Stability in Homomorphically Encrypted Language Models
• FedMMKT:Co-Enhancing a Server Text-to-Image Model and Client Task Models in Multi-Modal Federated Learning
• Unveiling the Vulnerability of Graph-LLMs: An Interpretable Multi-Dimensional Adversarial Attack on TAGs
• On the Interplay between Human Label Variation and Model Fairness
• QDER: Query-Specific Document and Entity Representations for Multi-Vector Document Re-Ranking
• FinVet: A Collaborative Framework of RAG and External Fact-Checking Agents for Financial Misinformation Detection
• An Encoder-Integrated PhoBERT with Graph Attention for Vietnamese Token-Level Classification
• CNSocialDepress: A Chinese Social Media Dataset for Depression Risk Detection and Structured Analysis
• Fairness Metric Design Exploration in Multi-Domain Moral Sentiment Classification using Transformer-Based Models
• An Explorative Study on Distributed Computing Techniques in Training and Inference of Large Language Models
• Decoupled Multimodal Fusion for User Interest Modeling in Click-Through Rate Prediction
• Therapeutic AI and the Hidden Risks of Over-Disclosure: An Embedded AI-Literacy Framework for Mental Health Privacy
• Toward Human-Centered Readability Evaluation
• LogiNumSynth: Synthesizing Joint Logical-Numerical Reasoning Problems for Language Models
• Secret-Protected Evolution for Differentially Private Synthetic Text Generation
• Happiness is Sharing a Vocabulary: A Study of Transliteration Methods
• Image-to-Video Transfer Learning based on Image-Language Foundation Models: A Comprehensive Survey
• FactAppeal: Identifying Epistemic Factual Appeals in News Media
• Self-Supervised Representation Learning with ID-Content Modality Alignment for Sequential Recommendation
• Knowing Unknowns in an Age of Information Overload
• PENEX: AdaBoost-Inspired Neural Network Regularization
• Hybrid OCR-LLM Framework for Enterprise-Scale Document Information Extraction Under Copy-heavy Task
• CLMN: Concept based Language Models via Neural Symbolic Reasoning
• Towards Speeding up Program Repair with Non-Autoregressive Model
• Framing Unionization on Facebook: Communication around Representation Elections in the United States
• Classifier-Augmented Generation for Structured Workflow Prediction
• Text2Token: Unsupervised Text Representation Learning with Token Target Prediction
• HUME: Measuring the Human-Model Performance Gap in Text Embedding Tasks
• Don't Throw Away Your Pretrained Model
• Can We Reliably Rank Model Performance across Domains without Labeled Data?
• Entropy Meets Importance: A Unified Head Importance-Entropy Score for Stable and Efficient Transformer Pruning
• CLARity: Reasoning Consistency Alone Can Teach Reinforced Experts
• Exploring Cross-Lingual Knowledge Transfer via Transliteration-Based MLM Fine-Tuning for Critically Low-resource Chakma Language
• Fall into a Pit, Gain in a Wit: Cognitive-Guided Harmful Meme Detection via Misjudgment Risk Pattern Retrieval
• Hierarchical Indexing with Knowledge Enrichment for Multilingual Video Corpus Retrieval
• One Sentence, Two Embeddings: Contrastive Learning of Explicit and Implicit Semantic Representations
• LTCA: Long-range Temporal Context Attention for Referring Video Object Segmentation
• Kelp: A Streaming Safeguard for Large Models via Latent Dynamics-Guided Risk Detection
• AILoRA: Function-Aware Asymmetric Initialization for Low-Rank Adaptation of Large Language Models
• Towards Human-Like Grading: A Unified LLM-Enhanced Framework for Subjective Question Evaluation
• Textual Entailment and Token Probability as Bias Evaluation Metrics
• t-SNE Exaggerates Clusters, Provably
• Causality Guided Representation Learning for Cross-Style Hate Speech Detection
• AutoQual: An LLM Agent for Automated Discovery of Interpretable Features for Review Quality Assessment
• Struc-EMB: The Potential of Structure-Aware Encoding in Language Embeddings
• Investigating Counterclaims in Causality Extraction from Text
• Two-Stage Voting for Robust and Efficient Suicide Risk Detection on Social Media
• Unlocking 3D Affordance Segmentation with 2D Semantic Knowledge
• Dual-granularity Sinkhorn Distillation for Enhanced Learning from Long-tailed Noisy Data
• SliceFine: The Universal Winning-Slice Hypothesis for Pretrained Networks
• Multi-Task Pre-Finetuning of Lightweight Transformer Encoders for Text Classification and NER
• Label Semantics for Robust Hyperspectral Image Classification
• Reasoning for Hierarchical Text Classification: The Case of Patents
• Iterative design of a NAND hybrid riboswitch by deep batch Bayesian optimization
• Auto-Stega: An Agent-Driven System for Lifelong Strategy Evolution in LLM-Based Text Steganography
• Controllable Stylistic Text Generation with Train-Time Attribute-Regularized Diffusion
• Towards Label-Free Biological Reasoning Synthetic Dataset Creation via Uncertainty Filtering
• MSF-SER: Enriching Acoustic Modeling with Multi-Granularity Semantics for Speech Emotion Recognition
• Membership Inference Attacks on Tokenizers of Large Language Models
• Data Provenance Auditing of Fine-Tuned Large Language Models with a Text-Preserving Technique
• The New Quant: A Survey of Large Language Models in Financial Prediction and Trading
• Decoding Partial Differential Equations: Cross-Modal Adaptation of Decoder-only Models to PDEs
• Gamma Mixture Modeling for Cosine Similarity in Small Language Models
• Are BabyLMs Deaf to Gricean Maxims? A Pragmatic Evaluation of Sample-efficient Language Models
• A Low-Resource Speech-Driven NLP Pipeline for Sinhala Dyslexia Assistance
• Evaluating LLMs for Demographic-Targeted Social Bias Detection: A Comprehensive Benchmark Study
• Exploring OCR-augmented Generation for Bilingual VQA
• Contrastive Learning Using Graph Embeddings for Domain Adaptation of Language Models in the Process Industry
• Residualized Similarity for Faithfully Explainable Authorship Verification
• GRACE: Generative Representation Learning via Contrastive Policy Optimization
• How Different from the Past? Spatio-Temporal Time Series Forecasting with Self-Supervised Deviation Learning
• Time Is Effort: Estimating Human Post-Editing Time for Grammar Error Correction Tool Evaluation
• Representation Potentials of Foundation Models for Multimodal Alignment: A Survey
• LongTail-Swap: benchmarking language models' abilities on rare words
• Multimodal Learning with Augmentation Techniques for Natural Disaster Assessment
• TreePrompt: Leveraging Hierarchical Few-Shot Example Selection for Improved English-Persian and English-German Translation
• Fine-Tuning Large Language Models with QLoRA for Offensive Language Detection in Roman Urdu-English Code-Mixed Text
• Towards Sampling Data Structures for Tensor Products in Turnstile Streams
• Consistent Kernel Change-Point Detection under m-Dependence for Text Segmentation
• HyperAdaLoRA: Accelerating LoRA Rank Allocation During Training via Hypernetworks without Sacrificing Performance
• Dissecting Transformers: A CLEAR Perspective towards Green AI
• Semantic Differentiation in Speech Emotion Recognition: Insights from Descriptive and Expressive Speech Roles
• Fine-Tuning on Noisy Instructions: Effects on Generalization and Performance
• Coevolutionary Continuous Discrete Diffusion: Make Your Diffusion Language Model a Latent Reasoner
• Brain-Language Model Alignment: Insights into the Platonic Hypothesis and Intermediate-Layer Advantage
• Sample-Efficient Differentially Private Fine-Tuning via Gradient Matrix Denoising
• GLAI: GreenLightningAI for Accelerated Training through Knowledge Decoupling
• OTTER: Open-Tagging via Text-Image Representation for Multi-modal Understanding
• Social Processes in the Intensification of Online Hate: The Effects of Verbal Replies to Anti-Muslim and Anti-Jewish Posts Following 7 October 2023
• Automated Alignment of Math Items to Content Standards in Large-Scale Assessments Using Language Models
• Efficient Layer-wise LLM Fine-tuning for Revision Intention Prediction
• SecureBERT 2.0: Advanced Language Model for Cybersecurity Intelligence
• ACT: Agentic Classification Tree
• Text-Based Approaches to Item Alignment to Content Standards in Large-Scale Reading & Writing Tests
• Communication-Efficient and Accurate Approach for Aggregation in Federated Low-Rank Adaptation
• ProfVLM: A Lightweight Video-Language Model for Multi-View Proficiency Estimation
• Type-Less yet Type-Aware Inductive Link Prediction with Pretrained Language Models
• VietBinoculars: A Zero-Shot Approach for Detecting Vietnamese LLM-Generated Text
• MGen: Millions of Naturally Occurring Generics in Context
• Reinforced Strategy Optimization for Conversational Recommender Systems via Network-of-Experts
• FITS: Towards an AI-Driven Fashion Information Tool for Sustainability
• Bringing Emerging Architectures to Sequence Labeling in NLP
• Rotation Control Unlearning: Quantifying and Controlling Continuous Unlearning for LLM with The Cognitive Rotation Space
• The Flaw of Averages: Quantifying Uniformity of Performance on Benchmarks
• CustomIR: Unsupervised Fine-Tuning of Dense Embeddings for Known Document Corpora
• GateMABSA: Aspect-Image Gated Fusion for Multimodal Aspect-based Sentiment Analysis
• HarmMetric Eval: Benchmarking Metrics and Judges for LLM Harmfulness Assessment
• Dynamic Orchestration of Multi-Agent System for Real-World Multi-Image Agricultural VQA
• Fin-Ally: Pioneering the Development of an Advanced, Commonsense-Embedded Conversational AI for Money Matters
• Metamorphic Testing for Audio Content Moderation Software
• Talk in Pieces, See in Whole: Disentangling and Hierarchical Aggregating Representations for Language-based Object Detection
• Bacterial proteome foundation model enhances functional prediction from enzymes to ecological interactions
• Evaluating Spatiotemporal Consistency in Automatically Generated Sewing Instructions
• LLM/Agent-as-Data-Analyst: A Survey
• MASH: A Multiplatform and Multimodal Annotated Dataset for Societal Impact of Hurricane
• Open-DeBias: Toward Mitigating Open-Set Bias in Language Models
• MaskSQL: Safeguarding Privacy for LLM-Based Text-to-SQL via Abstraction
• SynDoc: A Hybrid Discriminative-Generative Framework for Enhancing Synthetic Domain-Adaptive Document Key Information Extraction
• Fin-ExBERT: User Intent based Text Extraction in Financial Context using Graph-Augmented BERT and trainable Plugin
• PDE-Transformer: A Continuous Dynamical Systems Approach to Sequence Modeling
• Beyond Embeddings: Interpretable Feature Extraction for Binary Code Similarity
• Reinforcement Learning-Based Prompt Template Stealing for Text-to-Image Models
• Bridging Fairness and Explainability: Can Input-Based Explanations Promote Fairness in Hate Speech Detection?
• Mixture of Detectors: A Compact View of Machine-Generated Text Detection
• RedNote-Vibe: A Dataset for Capturing Temporal Dynamics of AI-Generated Text in Social Media
• Fuzzy Reasoning Chain (FRC): An Innovative Reasoning Framework from Fuzziness to Clarity
• Boundary on the Table: Efficient Black-Box Decision-Based Attacks for Structured Data
• Towards Transparent AI: A Survey on Explainable Language Models
• Sycophancy Is Not One Thing: Causal Separation of Sycophantic Behaviors in LLMs
• An Improved Quantum Software Challenges Classification Approach using Transfer Learning and Explainable AI
• A short survey on almost orthogonal vectors in a few specific large dimensions
• CTI Dataset Construction from Telegram
• PseudoBridge: Pseudo Code as the Bridge for Better Semantic and Logic Alignment in Code Retrieval
• RedHerring Attack: Testing the Reliability of Attack Detection
• RJE: A Retrieval-Judgment-Exploration Framework for Efficient Knowledge Graph Question Answering with LLMs
• Understanding and Enhancing Mask-Based Pretraining towards Universal Representations
• Performance Consistency of Learning Methods for Information Retrieval Tasks
• Leveraging Temporally Extended Behavior Sharing for Multi-task Reinforcement Learning
• InconVAD: A Two-Stage Dual-Tower Framework for Multimodal Emotion Inconsistency Detection
• Faster Than SVD, Smarter Than SGD: The OPLoRA Alternating Update
• SINAI at eRisk@CLEF 2025: Transformer-Based and Conversational Strategies for Depression Detection
• Revisiting Performance Claims for Chest X-Ray Models Using Clinical Context
• Every Character Counts: From Vulnerability to Defense in Phishing Detection
• SeMob: Semantic Synthesis for Dynamic Urban Mobility Prediction
• GuessingGame: Measuring the Informativeness of Open-Ended Questions in Large Language Models
• Confidence Calibration in Large Language Model-Based Entity Matching
• A comparative analysis of automated techniques for security bug report identification
• VESTIGE: A Knowledge-Guided Masking Strategy for Corruption-Aware Fine-Tuning of Genomic Transformers, Validated on Ancient DNA Reconstruction
• GyRot: Leveraging Hidden Synergy between Rotation and Fine-grained Group Quantization for Low-bit LLM Inference
• AWARE-FX: An Auditable Knowledge-Guided AI System for Measuring Corporate Foreign-Exchange Hedging Disclosure
• GGC: Selective Query Correction for Reliable Text-to-SPARQL Generation
• Improving Mental Health Screening and Early Risk Detection in Spanish
• Hand-Object Interaction in the Age of Large Foundation Models:Reconstruction, Generation, and Embodied Transfer
• NorBERTo: A ModernBERT Model Trained for Portuguese with 331 Billion Tokens Corpus
• CAIT: A Syntactic Parsing Toolkit for Child-Adult InTeractions
• Is Child-Directed Language Optimized for Word Learning? A Computational Study of Verb Meaning Acquisition
• Can LLMs capture stable human-generated sentence entropy measures?
• Optimizing protein tokenization: reduced amino acid alphabets for efficient and accurate protein language models
• A study of word embedding models for measuring topic coherence
• Reading the unreadable: creating a dataset of 19th century English newspapers using image-to-text language models
• Multiple agroecological practices use and climate change mitigation. A review
• ISACL: Internal State Analyzer for Copyrighted Training Data Leakage
• HD-PPT: Hierarchical Decoding of Content- and Prompt-Preference Tokens for Instruction-based TTS
• Reinforcement Learning on Pre-Training Data
• Systematic Comparative Analysis of Large Pretrained Language Models on Contextualized Medication Event Extraction
• Enhancing LLM-Based Social Bot via an Adversarial Learning Framework
• Trigger Where It Hurts: Unveiling Hidden Backdoors through Sensitivity with Sensitron
• HyperAdapt: Simple High-Rank Adaptation
• Financial Risk Relation Identification through Dual-view Adaptation
• Trace Is In Sentences: Unbiased Lightweight ChatGPT-Generated Text Detector
• M4SER: Multimodal, Multirepresentation, Multitask, and Multistrategy Learning for Speech Emotion Recognition
• Multi-Hierarchical Feature Detection for Large Language Model Generated Text
• RL-Finetuned LLMs for Privacy-Preserving Synthetic Rewriting
• TensLoRA: Tensor Alternatives for Low-Rank Adaptation
• Fine-Grained Detection of AI-Generated Text Using Sentence-Level Segmentation
• On-the-Fly Adaptation to Quantization: Configuration-Aware LoRA for Efficient Fine-Tuning of Quantized LLMs
• Shifting norms in scholarly publications: trends in readability, objectivity, authorship, and AI use
• LLaVul: A Multimodal LLM for Interpretable Vulnerability Reasoning about Source Code
• Modeling the Attack: Detecting AI-Generated Text by Quantifying Adversarial Perturbations
• DRES: Fake news detection by dynamic representation and ensemble selection
• Variational Task Vector Composition
• Segment-to-Act: Label-Noise-Robust Action-Prompted Video Segmentation Towards Embodied Intelligence
• AISTAT lab system for DCASE2025 Task6: Language-based audio retrieval
• Less Is More? Examining Fairness in Pruned Large Language Models for Summarising Opinions
• GRIL: Knowledge Graph Retrieval-Integrated Learning with Large Language Models
• Leveraging Multilingual Training for Authorship Representation: Enhancing Generalization across Languages and Domains
• Causality-Induced Positional Encoding for Transformer-Based Representation Learning of Non-Sequential Features
• A Multi-Level Benchmark for Causal Language Understanding in Social Media Discourse
• MPCG: Multi-Round Persona-Conditioned Generation for Modeling the Evolution of Misinformation with LLMs
• The Role of Vocabularies in Learning Sparse Representations for Ranking
• LLM-Guided Co-Training for Text Classification
• CausalSent: Interpretable Sentiment Classification with RieszNet
• BEFT: Bias-Efficient Fine-Tuning of Language Models
• EmoQ: Speech Emotion Recognition via Speech-Aware Q-Former and Large Language Model
• KITE: Kernelized and Information Theoretic Exemplars for In-Context Learning
• Toward Efficient Influence Function: Dropout as a Compression Tool
• Latent Zoning Network: A Unified Principle for Generative Modeling, Representation Learning, and Classification
• Attention Schema-based Attention Control (ASAC): A Cognitive-Inspired Approach for Attention Management in Transformers
• KRAST: Knowledge-Augmented Robotic Action Recognition with Structured Text for Vision-Language Models
• Real, Fake, or Manipulated? Detecting Machine-Influenced Text
• SynParaSpeech: Automated Synthesis of Paralinguistic Datasets for Speech Generation and Understanding
• SINAI at eRisk@CLEF 2022: Approaching Early Detection of Gambling and Eating Disorders with Natural Language Processing
• SINAI at eRisk@CLEF 2023: Approaching Early Detection of Gambling with Natural Language Processing
• Spatial-CLAP: Learning Spatially-Aware audio--text Embeddings for Multi-Source Conditions
• Comparative Analysis of Tokenization Algorithms for Low-Resource Language Dzongkha
• Understanding the Thinking Process of Reasoning Models: A Perspective from Schoenfeld's Episode Theory
• Leveraging IndoBERT and DistilBERT for Indonesian Emotion Classification in E-Commerce Reviews
• Adaptive LoRA Experts Allocation and Selection for Federated Fine-Tuning
• Patent Language Model Pretraining with ModernBERT
• Hierarchical Self-Attention: Generalizing Neural Attention Mechanics to Multi-Scale Problems
• Not What the Doctor Ordered: Surveying LLM-based De-identification and Quantifying Clinical Information Loss
• Do LLMs Align Human Values Regarding Social Biases? Judging and Explaining Social Biases with LLMs
• Words to Waves: Emotion-Adaptive Music Recommendation System
• CrowdAgent: Multi-Agent Managed Multi-Source Annotation System
• Long-context Reference-based MT Quality Estimation
• HSGM: Hierarchical Segment-Graph Memory for Scalable Long-Text Semantics
• Translate, then Detect: Leveraging Machine Translation for Cross-Lingual Toxicity Classification
• Op-Fed: Opinion, Stance, and Monetary Policy Annotations on FOMC Transcripts Using Active Learning
• Evaluating LLM Alignment on Personality Inference from Real-World Interview Data
• A Multi-Component AI Framework for Computational Psychology: From Robust Predictive Modeling to Deployed Generative Dialogue
• The Few-shot Dilemma: Over-prompting Large Language Models
• Investigating ReLoRA: Effects on the Learning Dynamics of Small Language Models
• Are You Sure You're Positive? Consolidating Chain-of-Thought Agents with Uncertainty Quantification for Aspect-Category Sentiment Analysis
• Towards Inclusive Toxic Content Moderation: Addressing Vulnerabilities to Adversarial Attacks in Toxicity Classifiers Tackling LLM-generated Content
• Module-Aware Parameter-Efficient Machine Unlearning on Transformers
• Automated Generation of Research Workflows from Academic Papers: A Full-text Mining Framework
• InfoGain-RAG: Boosting Retrieval-Augmented Generation via Document Information Gain-based Reranking and Filtering
• Natural Language Satisfiability: Exploring the Problem Distribution and Evaluating Transformer-based Language Models
• SENTRA: Selected-Next-Token Transformer for LLM Text Detection
• Pun Unintended: LLMs and the Illusion of Humor Understanding
• PledgeTracker: A System for Monitoring the Fulfilment of Pledges
• Query-Focused Extractive Summarization for Sentiment Explanation
• Dynamic Span Interaction and Graph-Aware Memory for Entity-Level Sentiment Classification
• GRAID: Synthetic Data Generation with Geometric Constraints and Multi-Agentic Reflection for Harmful Content Detection
• Topic Coverage-based Demonstration Retrieval for In-Context Learning
• RanAT4BIE: Random Adversarial Training for Biomedical Information Extraction
• We Argue to Agree: Towards Personality-Driven Argumentation-Based Negotiation Dialogue Systems for Tourism
• Opportunities and Challenges of Integrating ChatGPT in Education: Sentiment Analysis and Topic Modeling
• Beyond Token Limits: Assessing Language Model Performance on Long Text Classification
• Fluent but Unfeeling: The Emotional Blind Spots of Language Models
• DeMeVa at LeWiDi-2025: Modeling Perspectives with In-Context Learning and Label Distribution Learning
• Modelling Analogies and Analogical Reasoning: Connecting Cognitive Science Theory and NLP Research
• Curriculum-Based Multi-Tier Semantic Exploration via Deep Reinforcement Learning
• Sensitivity-LoRA: Low-Load Sensitivity-Based Fine-Tuning for Large Language Models
• SALMAN: Stability Analysis of Language Models Through the Maps Between Graph-based Manifolds
• CoSwin: Convolution Enhanced Hierarchical Shifted Window Attention For Small-Scale Vision
• Documents Are People and Words Are Items: A Psychometric Approach to Textual Data with Contextual Embeddings
• Learning from Diverse Reasoning Paths with Routing and Collaboration
• TriagerX: Dual Transformers for Bug Triaging Tasks with Content and Interaction Based Rankings
• Adversarial Attacks Against Automated Fact-Checking: A Survey
• Explainable Semantic Text Relations: A Question-Answering Framework for Comparing Document Content
• Query Expansion in the Age of Pre-trained and Large Language Models: A Comprehensive Survey
• Are LLMs Enough for Hyperpartisan, Fake, Polarized and Harmful Content Detection? Evaluating In-Context Learning vs. Fine-Tuning
• Assess and Prompt: A Generative RL Framework for Improving Engagement in Online Mental Health Communities
• M-BRe: Discovering Training Samples for Relation Extraction from Unlabeled Texts with Large Language Models
• PianoBind: A Multimodal Joint Embedding Model for Pop-piano Music
• Uncovering the Vulnerability of Large Language Models in the Financial Domain via Risk Concealment
• mmBERT: A Modern Multilingual Encoder with Annealed Language Learning
• ALPHA: LLM-Enabled Active Learning for Human-Free Network Anomaly Detection
• Revealing the Numeracy Gap: An Empirical Investigation of Text Embedding Models
• From Joy to Fear: A Benchmark of Emotion Estimation in Pop Song Lyrics
• Natural Language-Programming Language Software Traceability Link Recovery Needs More than Textual Similarity
• Few-Shot Query Intent Detection via Relation-Aware Prompt Learning
• A Survey of the State-of-the-Art in Conversational Question Answering Systems
• Differential Robustness in Transformer Language Models: Empirical Evaluation Under Adversarial Text Attacks
• Masked Diffusion Language Models with Frequency-Informed Training
• Dynamic Adaptive Shared Experts with Grouped Multi-Head Attention Mixture of Experts
• Rethinking Reasoning in LLMs: Neuro-Symbolic Local RetoMaton Beyond ICL and CoT
• Optimizing Small Transformer-Based Language Models for Multi-Label Sentiment Analysis in Short Texts
• Finding your MUSE: Mining Unexpected Solutions Engine
• Comparative Analysis of Transformer Models in Disaster Tweet Classification for Public Safety
• A RoBERTa-Based Functional Syntax Annotation Model for Chinese Texts
• Joint Modeling of Entities and Discourse Relations for Coherence Assessment
• LMAE4Eth: Generalizable and Robust Ethereum Fraud Detection by Exploring Transaction Semantics and Masked Graph Embedding
• KGBERT4Eth: A Feature-Complete Transformer Powered by Knowledge Graph for Multi-Task Ethereum Fraud Detection
• Anti-establishment sentiment on TikTok: Implications for understanding influence(rs) and expertise on social media
• MLSD: A Novel Few-Shot Learning Approach to Enhance Cross-Target and Cross-Domain Stance Detection
• Cetvel: A Unified Benchmark for Evaluating Language Understanding, Generation and Cultural Capacity of LLMs for Turkish
• NoteBar: An AI-Assisted Note-Taking System for Personal Knowledge Management
• Scalable and Loosely-Coupled Multimodal Deep Learning for Breast Cancer Subtyping
• Advancing Minority Stress Detection with Transformers: Insights from the Social Media Datasets
• Training LLMs to be Better Text Embedders through Bidirectional Reconstruction
• Revealing the Paper Mill Iceberg: AI-Based Screening of Cancer Research Publications
• Efficient Training-Free Online Routing for High-Volume Multi-LLM Serving
• Attributes as Textual Genes: Leveraging LLMs as Genetic Algorithm Simulators for Conditional Synthetic Data Generation
• Towards Agents That Know When They Don't Know: Uncertainty as a Control Signal for Structured Reasoning
• Abex-rat: Synergizing Abstractive Augmentation and Adversarial Training for Classification of Occupational Accident Reports
• chDzDT: Word-level morphology-aware language model for Algerian social media text
• Insight-LLM: LLM-enhanced Multi-view Fusion in Insider Threat Detection
• The Fools are Certain; the Wise are Doubtful: Exploring LLM Confidence in Code Completion
• The AudioMOS Challenge 2025
• REFRAG: Rethinking RAG based Decoding
• TransGAT: Transformer-Based Graph Neural Networks for Multi-Dimensional Automated Essay Scoring
• AttnBoost: Retail Supply Chain Sales Insights via Gradient Boosting Perspective
• Street-Level Geolocalization Using Multimodal Large Language Models and Retrieval-Augmented Generation
• Enhancing Partially Relevant Video Retrieval with Robust Alignment Learning
• Serialized Output Prompting for Large Language Model-based Multi-Talker Speech Recognition
• Benchmarking the Detection of LLMs-Generated Modern Chinese Poetry
• Testing the assumptions about the geometry of sentence embedding spaces: the cosine measure need not apply
• Supervised In-Context Fine-Tuning for Generative Sequence Labeling
• LLM Encoder vs. Decoder: Robust Detection of Chinese AI-Generated Text with LoRA
• Speaker-Conditioned Phrase Break Prediction for Text-to-Speech with Phoneme-Level Pre-trained Language Model
• A Multi-Strategy Approach for AI-Generated Text Detection
• HePGA: A Heterogeneous Processing-in-Memory based GNN Training Accelerator
• A Hybrid AI-based and Rule-based Approach to DICOM De-identification: A Solution for the MIDI-B Challenge
• Backdoor Samples Detection Based on Perturbation Discrepancy Consistency in Pre-trained Language Models
• Adaptive Point-Prompt Tuning: Fine-Tuning Heterogeneous Foundation Models for 3D Point Cloud Analysis
• A Survey on Current Trends and Recent Advances in Text Anonymization
• Large Language Model Integration with Reinforcement Learning to Augment Decision-Making in Autonomous Cyber Operations
• Strategic Sample Selection for Improved Clean-Label Backdoor Attacks in Text Classification
• Re-Representation in Sentential Relation Extraction with Sequence Routing Algorithm
• Multi-Lingual Implicit Discourse Relation Recognition with Multi-Label Hierarchical Learning
• Overview of BioASQ 2024: The twelfth BioASQ challenge on Large-Scale Biomedical Semantic Indexing and Question Answering
• Prediction of mortality and resource utilization in critical care: a deep learning approach using multimodal electronic health records with natural language processing techniques
• Automated Bug Triaging using Instruction-Tuned Large Language Models
• SurGE: A Benchmark and Evaluation Framework for Scientific Survey Generation
• Speech Emotion Recognition via Entropy-Aware Score Selection
• Exploring Machine Learning and Language Models for Multimodal Depression Detection
• Leveraging Semantic Triples for Private Document Generation with Local Differential Privacy Guarantees
• GDLLM: A Global Distance-aware Modeling Approach Based on Large Language Models for Event Temporal Relation Extraction
• FedReFT: Federated Representation Fine-Tuning with All-But-Me Aggregation
• Forewarned is Forearmed: Pre-Synthesizing Jailbreak-like Instructions to Enhance LLM Safety Guardrail to Potential Attacks
• Cross-Platform E-Commerce Product Categorization and Recategorization: A Multimodal Hierarchical Classification Approach
• Exploring Selective Retrieval-Augmentation for Long-Tail Legal Text Classification
• FlowletFormer: Network Behavioral Semantic Aware Pre-training Model for Traffic Classification
• QuesGenie: Intelligent Multimodal Question Generation
• SCAR: A Characterization Scheme for Multi-Modal Dataset
• Emotion Transfer with Enhanced Prototype for Unseen Emotion Recognition in Conversation
• Inference Gap in Domain Expertise and Machine Intelligence in Named Entity Recognition: Creation of and Insights from a Substance Use-related Dataset
• Stack Trace-Based Crash Deduplication with Transformer Adaptation
• Reflective Agreement: Combining Self-Mixture of Agents with a Sequence Tagger for Robust Event Extraction
• ProPy: Building Interactive Prompt Pyramids upon CLIP for Partially Relevant Video Retrieval
• CoBA: Counterbias Text Augmentation for Mitigating Various Spurious Correlations via Semantic Triples
• Granite Embedding R2 Models
• AnomalyExplainer Explainable AI for LLM-based anomaly detection using BERTViz and Captum
• COMET-poly: Machine Translation Metric Grounded in Other Candidates
• DiscussLLM: Teaching Large Language Models When to Speak
• How Quantization Shapes Bias in Large Language Models
• End-to-End On-Device Quantization-Aware Training for LLMs at Inference Cost
• Influence-driven Curriculum Learning for Pre-training on Limited Data
• Identifying and Answering Questions with False Assumptions: An Interpretable Approach
• A Robust BERT-Based Deep Learning Model for Automated Cancer Type Extraction from Unstructured Pathology Reports
• Self-Disguise Attack: Induce the LLM to disguise itself for AIGT detection evasion
• Don't Think Twice! Over-Reasoning Impairs Confidence Calibration
• Declarative Data Pipeline for Large Scale ML Services
• Tokens with Meaning: A Hybrid Tokenization Approach for NLP
• MGT-Prism: Enhancing Domain Generalization for Machine-Generated Text Detection via Spectral Alignment
• Extracting Structured Requirements from Unstructured Building Technical Specifications for Building Information Modeling
• Towards Unified Multimodal Financial Forecasting: Integrating Sentiment Embeddings and Market Indicators via Cross-Modal Attention
• RepreGuard: Detecting LLM-Generated Text by Revealing Hidden Representation Patterns
• Improving Detection of Watermarked Language Models
• Word Meanings in Transformer Language Models
• Structuring the Unstructured: A Systematic Review of Text-to-Structure Generation for Agentic AI with a Universal Evaluation Framework
• LinkAnchor: An Autonomous LLM-Based Agent for Issue-to-Commit Link Recovery
• MAD: A Benchmark for Multi-Turn Audio Dialogue Fact-Checking
• The Structural Sources of Verb Meaning Revisited: Large Language Models Display Syntactic Bootstrapping
• Temporal Grounding as a Learning Signal for Referring Video Object Segmentation
• Handwritten Text Recognition of Historical Manuscripts Using Transformer-Based Models
• Expressive Speech Retrieval using Natural Language Descriptions of Speaking Style
• Labels or Input? Rethinking Augmentation in Multimodal Hate Detection
• STEP: Stepwise Curriculum Learning for Context-Knowledge Fusion in Conversational Recommendation
• Unpacking the Implicit Norm Dynamics of Sharpness-Aware Minimization in Tensorized Models
• Improving OCR for Historical Texts of Multiple Languages
• From Surface to Semantics: Semantic Structure Parsing for Table-Centric Document Analysis
• On the synchronization between Hugging Face pre-trained language models and their upstream GitHub repository
• Shaping Event Backstories to Estimate Potential Emotion Contexts
• Can LLM-Generated Textual Explanations Enhance Model Classification Performance? An Empirical Study
• SinLlama -- A Large Language Model for Sinhala
• Understanding Syntactic Generalization in Structure-inducing Language Models
• LLMs for Law: Evaluating Legal-Specific LLMs on Contract Understanding
• Large Language Models for Subjective Language Understanding: A Survey
• Effortless Vision-Language Model Specialization in Histopathology without Annotation
• Training-Free ANN-to-SNN Conversion for High-Performance Spiking Transformer
• Modeling and Detecting Company Risks from News: A Case Study in Bloomberg News
• Planner-Refiner: Dynamic Space-Time Refinement for Vision-Language Alignment in Videos
• A Survey on Non-Intrusive ASR Refinement: From Output-Level Correction to Full-Model Distillation
• MAQuA: Adaptive Question-Asking for Multidimensional Mental Health Screening using Item Response Theory
• EventRR: Event Referential Reasoning for Referring Video Object Segmentation
• Arce: Augmented Roberta with Contextualized Elucidations for Ner in Automated Rule Checking
• Enhancing Rumor Detection Methods with Propagation Structure Infused Language Model
• Towards Real-World Rumor Detection: Anomaly Detection Framework with Graph Supervised Contrastive Learning
• ESNERA: Empirical and semantic named entity alignment for named entity dataset merging
• SEVADE: Self-Evolving Multi-Agent Analysis with Decoupled Evaluation for Hallucination-Resistant Irony Detection
• Adversarial Video Promotion Against Text-to-Video Retrieval
• BoRA: Towards More Expressive Low-Rank Adaptation with Block Diversity
• End-to-End Text-to-SQL with Dataset Selection: Leveraging LLMs for Adaptive Query Generation
• LLMCARE: early detection of cognitive impairment via transformer models enhanced by LLM-generated synthetic data
• Bifrost-1: Bridging Multimodal LLMs and Diffusion Models with Patch-level CLIP Latents
• Omni Geometry Representation Learning vs Large Language Models for Geospatial Entity Resolution
• "Mirror" Language AI Models of Depression are Criterion-Contaminated
• Guardians and Offenders: A Survey on Harmful Content Generation and Safety Mitigation of LLM
• Streamlining Admission with LOR Insights: AI-Based Leadership Assessment in Online Master's Program
• VS-LLM: Visual-Semantic Depression Assessment based on LLM for Drawing Projection Test
• Decision-Making with Deliberation: Meta-reviewing as a Document-grounded Dialogue
• HFedATM: Hierarchical Federated Domain Generalization via Optimal Transport and Regularized Mean Aggregation
• Multimodal Fact Checking with Unified Visual, Textual, and Contextual Representations
• A Study of the Framework and Real-World Applications of Language Embedding for 3D Scene Understanding
• Advancing Hate Speech Detection with Transformers: Insights from the MetaHate
• Sensitivity of Stability: Theoretical & Empirical Analysis of Replicability for Adaptive Data Selection in Transfer Learning
• Boosting Visual Knowledge-Intensive Training for LVLMs Through Causality-Driven Visual Object Completion
• Graph Representation Learning with Massive Unlabeled Data for Rumor Detection
• Guided Perturbation Sensitivity (GPS): Detecting Adversarial Text via Embedding Stability and Word Importance
• MegaWika 2: A More Comprehensive Multilingual Collection of Articles and their Sources
• Block: Balancing Load in LLM Serving with Context, Knowledge and Predictive Scheduling
• UPLME: Uncertainty-Aware Probabilistic Language Modelling for Robust Empathy Regression
• Cropping outperforms dropout as an augmentation strategy for training self-supervised text embeddings
• Exploring Stability-Plasticity Trade-offs for Continual Named Entity Recognition
• RooseBERT: A New Deal For Political Language Modelling
• Current State in Privacy-Preserving Text Preprocessing for Domain-Agnostic NLP
• Adaptive Sparse Softmax: An Effective and Efficient Softmax Variant
• Multimodal Human-Intent Modeling for Contextual Robot-to-Human Handovers of Arbitrary Objects
• CTR-Sink: Attention Sink for Language Models in Click-Through Rate Prediction
• From literature to biodiversity data: mining arthropod organismal traits with machine learning
• HiTeC: Hierarchical Contrastive Learning on Text-Attributed Hypergraph with Semantic-Aware Augmentation
• Tricks and Plug-ins for Gradient Boosting with Transformers
• SLIM-LLMs: Modeling of Style-Sensory Language RelationshipsThrough Low-Dimensional Representations
• Modeling Annotator Disagreement with Demographic-Aware Experts and Synthetic Perspectives
• LOST: Low-rank and Sparse Pre-training for Large Language Models
• "Harmless to You, Hurtful to Me!": Investigating the Detection of Toxic Languages Grounded in the Perspective of Youth
• Charting 15 years of progress in deep learning for speech emotion recognition: A replication study
• IMoRe: Implicit Program-Guided Reasoning for Human Motion Q&A
• Fine-grained Multiple Supervisory Network for Multi-modal Manipulation Detecting and Grounding
• Zero-shot Compositional Action Recognition with Neural Logic Constraints
• GlaBoost: A multimodal Structured Framework for Glaucoma Risk Stratification
• Authorship Attribution in Multilingual Machine-Generated Texts
• DRKF: Decoupled Representations with Knowledge Fusion for Multimodal Emotion Recognition
• Empowering Tabular Data Preparation with Language Models: Why and How?
• Am I Blue or Is My Hobby Counting Teardrops? Expression Leakage in Large Language Models as a Symptom of Irrelevancy Disruption
• Instruction-based Time Series Editing
• HT-Transformer: Event Sequences Classification by Accumulating Prefix Information with History Tokens
• Foundation Models for Bioacoustics -- a Comparative Review
• A Coarse-to-Fine Approach to Multi-Modality 3D Occupancy Grounding
• FlashSVD: Memory-Efficient Inference with Streaming for Low-Rank Models
• R²-CoD: Understanding Text-Graph Complementarity in Relational Reasoning via Knowledge Co-Distillation
• WarriorMath: Enhancing the Mathematical Ability of Large Language Models with a Defect-aware Framework
• Interpreting Performance Profiles with Deep Learning
• The Missing Parts: Augmenting Fact Verification with Half-Truth Detection
• Bidirectional Action Sequence Learning for Long-term Action Anticipation with Large Language Models
• Multimodal Referring Segmentation: A Survey
• Towards Higher Effective Rank in Parameter-efficient Fine-tuning using Khatri--Rao Product
• VAULT: Vigilant Adversarial Updates via LLM-Driven Retrieval-Augmented Generation for NLI
• DACTYL: Diverse Adversarial Corpus of Texts Yielded from Large Language Models
• MMBERT: Scaled Mixture-of-Experts Multimodal BERT for Robust Chinese Hate Speech Detection under Cloaking Perturbations
• Improving Multimodal Contrastive Learning of Sentence Embeddings with Object-Phrase Alignment
• Semantic Compression for Word and Sentence Embeddings using Discrete Wavelet Transform
• Role-Aware Language Models for Secure and Contextualized Access Control in Organizations
• Causal2Vec: Improving Decoder-only LLMs as Versatile Embedding Models
• MemoCue: Empowering LLM-Based Agents for Human Memory Recall via Strategy-Guided Querying
• Improved Algorithms for Kernel Matrix-Vector Multiplication Under Sparsity Assumptions
• Distilling Knowledge from Large Language Models: A Concept Bottleneck Model for Hate and Counter Speech Recognition
• Modeling Human Gaze Behavior with Diffusion Models for Unified Scanpath Prediction
• Multilingual Political Views of Large Language Models: Identification and Steering
• Hate in Plain Sight: On the Risks of Moderating AI-Generated Hateful Illusions

Related

• How to Train BERT with an Academic Budget
• ExtremeBERT: A Toolkit for Accelerating Pretraining of Customized BERT
• Visualizing and Understanding the Effectiveness of BERT
• On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines
• bert2BERT: Towards Reusable Pretrained Language Models