Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone

2024/04/22 by Marah Abdin, Abdin, Marah, Jyoti Aneja +262 · 8 voices · 682 citations
Computer Science · #Computational Physics and Python Applications #Computer science #Linguistics #Philosophy #Phone

paper · pdf · doi:10.48550/arxiv.2404.14219

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2024/04/22 · openalex created_date 2024/04/26 · openalex updated_date 2026/08/05

Abstract

We introduce phi-3-mini, a 3.8 billion parameter language model trained on 3.3 trillion tokens, whose overall performance, as measured by both academic benchmarks and internal testing, rivals that of models such as Mixtral 8x7B and GPT-3.5 (e.g., phi-3-mini achieves 69% on MMLU and 8.38 on MT-bench), despite being small enough to be deployed on a phone. Our training dataset is a scaled-up version of the one used for phi-2, composed of heavily filtered publicly available web data and synthetic data. The model is also further aligned for robustness, safety, and chat format. We also provide parameter-scaling results with a 7B, 14B models trained for 4.8T tokens, called phi-3-small, phi-3-medium, both significantly more capable than phi-3-mini (e.g., respectively 75%, 78% on MMLU, and 8.7, 8.9 on MT-bench). To enhance multilingual, multimodal, and long-context capabilities, we introduce three models in the phi-3.5 series: phi-3.5-mini, phi-3.5-MoE, and phi-3.5-Vision. The phi-3.5-MoE, a 16 x 3.8B MoE model with 6.6 billion active parameters, achieves superior performance in language reasoning, math, and code tasks compared to other open-source models of similar scale, such as Llama 3.1 and the Mixtral series, and on par with Gemini-1.5-Flash and GPT-4o-mini. Meanwhile, phi-3.5-Vision, a 4.2 billion parameter model derived from phi-3.5-mini, excels in reasoning tasks and is adept at handling both single-image and text prompts, as well as multi-image and text prompts.

Cited by

• Unified Static-Dynamic Pruning for Efficient LLM Inference
• Offline Vision-Language Navigation with Geometric Goal Localization for Outdoor Environments
• QLPO: Quadrant-weighted Sampling for Length-aware Policy Optimization
• LaSEr-Edit: Localized Span-level Error Editing with Energy-based Localization
• From Bit-Position Sensitivity to Unequal Error Protection for DNN Inference Memory
• RubricRL: Simple Generalizable Rewards for Text-to-Image Generation
• PathAgentBench: Benchmarking Evidence-Seeking Vision-Language Models on Whole-Slide Pathology Image
• Benchmarking Resource-Efficient LLMs for Research Topic Ontology Generation in the Biomedical Field
• Breaking Refusal in the First Half: A Mechanistic Study of the Prefill Jailbreak
• Octopus v4: Graph of language models
• When Words Are Safe But Actions Kill: Probing Physical Danger Beyond Text Safety in Hidden-State Risk Space
• In-Place Tokenizer Expansion for Pre-trained LLMs
• From Particles to Perils: SVGD-Based Hazardous Scenario Generation for Autonomous Driving Systems Testing
• CARPRT: Class-Aware Zero-Shot Prompt Reweighting for Black-Box Vision-Language Models
• Leveraging Large Language Models for Generating Research Topic Ontologies: A Multi-Disciplinary Study
• Routing Without Training: Controllable-Ratio LLM Offloading via Reliability Gating
• Democratizing AI with Small Language Models: Structured Benchmarking and Parameter-Efficient Fine-Tuning for Local Deployment
• GLAN-QnA-KR: A Seedless Taxonomy-Driven Korean Instruction Corpus
• Routing Subspaces: Auditing Evaluation-to-Deployment Mismatch in Fine-Tuned Language Models
• Is MoE Routing a Huffman Code? Discovering the Frequency-Diversity Law in Chain-of-Thought
• Fifty Shades of Greenwashing: The Political Economy of Climate Change Advertising on Social Media
• LLMs can hide text in other text of the same length
• SAM 3D: 3Dfy Anything in Images
• Is Misinformation More Open? A Study of robots.txt Gatekeeping on the Web
• Chain-of-Experts: Unlocking the Communication Power of Mixture-of-Experts Models
• Small Language Models are the Future of Agentic AI
• Just as Humans Need Vaccines, So Do Models: Model Immunization to Combat Falsehoods
• From Tokens to Thoughts: How LLMs and Humans Trade Compression for Meaning
• Enough Coin Flips Can Make LLMs Act Bayesian
• Layers at Similar Depths Generate Similar Activations Across LLM Architectures
• Measuring Chain of Thought Faithfulness by Unlearning Reasoning Steps
• Does Time Have Its Place? Temporal Heads: Where Language Models Recall Time-specific Information
• Position: It's Time to Act on the Risk of Efficient Personalized Text Generation
• CalibQuant: 1-Bit KV Cache Quantization for Multimodal LLMs
• TAID: Temporally Adaptive Interpolated Distillation for Efficient Knowledge Transfer in Language Models
• CME-CAD: Heterogeneous Collaborative Multi-Expert Reinforcement Learning for CAD Code Generation
• Agentic Physical AI toward a Domain-Specific Foundation Model for Energy Systems: A Case Study on Nuclear Reactor Control
• When LLM Defenses Backfire: Characterizing Safety, Performance, and Cost Trade-offs
• Masking Teacher and Reinforcing Student for Distilling Vision-Language Models
• Open-Source Multimodal Moxin Models with Moxin-VLM and Moxin-VLA
• TRACE-CTI: Auditable Post-Extraction Governance of TTP Claims with Knowledge Graphs
• AMPBench-MT: A Homology-Controlled Benchmark for Antimicrobial Peptide Potency, Spectrum, and Safety Prediction
• Conformal Cascade: Distribution-Free Accuracy Guarantees for Multi-Tier LLM Inference
• Mage-VL: An Efficient Codec-Native Streaming Multimodal Foundation Model
• TAMEing Long Contexts in Personalization: Towards Training-Free and State-Aware MLLM Personalized Assistant
• Gamayun's Path to Multilingual Mastery: Cost-Efficient Training of a 1.5B-Parameter LLM
• MoRAgent: Parameter Efficient Agent Tuning with Mixture-of-Roles
• GateBreaker: Gate-Guided Attacks on Mixture-of-Expert LLMs
• Large Language Models Approach Expert Pedagogical Quality in Math Tutoring but Differ in Instructional and Linguistic Profiles
• QuantiPhy: A Quantitative Benchmark Evaluating Physical Reasoning Abilities of Vision-Language Models
• Neuro-Symbolic Control with Large Language Models for Language-Guided Spatial Tasks
• CheXPO-v2: Preference Optimization for Chest X-ray VLMs with Knowledge Graph Consistency
• CitySeeker: How Do VLMS Explore Embodied Urban Navigation With Implicit Human Needs?
• Efficient CPU-GPU Collaborative Inference for MoE-based LLMs on Memory-Limited Systems
• Sigma-MoE-Tiny Technical Report
• FoodLogAthl-218: Constructing a Real-World Food Image Dataset Using Dietary Management Applications
• Autonomous Construction-Site Safety Inspection Using Mobile Robots: A Multilayer VLM-LLM Pipeline
• NL2SpaTiaL: Generating Geometric Spatio-Temporal Logic Specifications from Natural Language for Manipulation Tasks
• Integrating Causal Reasoning into Automated Fact-Checking
• Moment and Highlight Detection via MLLM Frame Segmentation
• Using GUI Agent for Electronic Design Automation
• Leveraging LLMs for Title and Abstract Screening for Systematic Review: A Cost-Effective Dynamic Few-Shot Learning Approach
• CLINIC: Evaluating Multilingual Trustworthiness in Language Models for Healthcare
• VLM2GeoVec: Toward Universal Multimodal Embeddings for Remote Sensing
• Network and Compiler Optimizations for Efficient Linear Algebra Kernels in Private Transformer Inference
• Local LLM Ensembles for Zero-shot Portuguese Named Entity Recognition
• Chasing Shadows: Pitfalls in LLM Security Research
• Same Content, Different Answers: Cross-Modal Inconsistency in MLLMs
• SATGround: A Spatially-Aware Approach for Visual Grounding in Remote Sensing
• HalluShift++: Bridging Language and Vision through Internal Representation Shifts for Hierarchical Hallucinations in MLLMs
• Persian-Phi: Efficient Cross-Lingual Adaptation of Compact LLMs via Curriculum Learning
• Towards Unified Semantic and Controllable Image Fusion: A Diffusion Transformer Approach
• VideoVLA: Video Generators Can Be Generalizable Robot Manipulators
• Policy-based Sentence Simplification: Replacing Parallel Corpora with LLM-as-a-Judge
• David vs. Goliath: Can Small Models Win Big with Agentic AI in Hardware Design?
• AfriStereo: A Culturally Grounded Dataset for Evaluating Stereotypical Bias in Large Language Models
• Can Multi-Modal LLMs Provide Live Step-by-Step Task Guidance?
• Jina-VLM: Small Multilingual Vision Language Model
• Overcoming State Inertia: Minimally Invasive Temporal Alignment for Evolving Contexts
• Query-Level Uncertainty in Large Language Models
• ChromouVQA: Benchmarking Vision-Language Models under Chromatic Camouflaged Images
• ChartPoint: Guiding MLLMs with Grounding Reflection for Chart Reasoning
• Video-R2: Reinforcing Consistent and Grounded Reasoning in Multimodal Language Models
• Unexplored flaws in multiple-choice VQA evaluations
• Mirror, Mirror on the Wall -- Which is the Best Model of Them All?
• WaymoQA: A Multi-View Visual Question Answering Dataset for Safety-Critical Reasoning in Autonomous Driving
• HKRAG: Holistic Knowledge Retrieval-Augmented Generation Over Visually-Rich Documents
• How Learning Rate Decay Wastes Your Best Data in Curriculum-Based LLM Pretraining
• Skypilot: Fine-Tuning LLM with Physical Grounding for AAV Coverage Search
• VCU-Bridge: Hierarchical Visual Connotation Understanding via Semantic Bridging
• MGA-VQA: Secure and Interpretable Graph-Augmented Visual Question Answering with Memory-Guided Protection Against Unauthorized Knowledge Use
• Downscaling Intelligence: Exploring Perception and Reasoning Bottlenecks in Small Multimodal Models
• Multimodal Evaluation of Russian-language Architectures
• Attention Grounded Enhancement for Visual Document Retrieval
• Detecting and Steering LLMs' Empathy in Action
• TZ-LLM: Protecting On-Device Large Language Models with Arm TrustZone
• Analyzing Sustainability Messaging in Large-Scale Corporate Social Media
• LAET: A Layer-wise Adaptive Ensemble Tuning Framework for Pretrained Language Models
• Structured Definitions and Segmentations for Legal Reasoning in LLMs: A Study on Indian Legal Data
• ChEmREF: Evaluating Language Model Readiness for Chemical Emergency Response
• Think-at-Hard: Selective Latent Iterations to Improve Reasoning Language Models
• ChexFract: From General to Specialized -- Enhancing Fracture Description Generation
• Revisiting NLI: Towards Cost-Effective and Human-Aligned Metrics for Evaluating LLMs in Question Answering
• Sensitivity of Small Language Models to Fine-tuning Data Contamination
• Towards Resource-Efficient Multimodal Intelligence: Learned Routing among Specialized Expert Models
• ThaiOCRBench: A Task-Diverse Benchmark for Vision-Language Understanding in Thai
• Seeing Straight: Document Orientation Detection for Efficient OCR
• From Prompts to Power: Measuring the Energy Footprint of LLM Inference
• Contamination Detection for VLMs using Multi-Modal Semantic Perturbation
• UTF-8 Plumbing: Byte-level Tokenizers Unavoidably Enable LLMs to Generate Ill-formed UTF-8
• TAUE: Training-free Noise Transplant and Cultivation Diffusion Model
• Assessing LLM Reasoning Steps via Principal Knowledge Grounding
• ShadowLogic: Backdoors in Any Whitebox LLM
• RzenEmbed: Towards Comprehensive Multimodal Retrieval
• H-FA: A Hybrid Floating-Point and Logarithmic Approach to Hardware Accelerated FlashAttention
• ChartAB: A Benchmark for Chart Grounding & Dense Alignment
• Cross-Platform Evaluation of Reasoning Capabilities in Foundation Models
• Normative Reasoning in Large Language Models: A Comparative Benchmark from Logical and Modal Perspectives
• Beyond CNNs: Efficient Fine-Tuning of Multi-Modal LLMs for Object Detection on Low-Data Regimes
• Thinking About Thinking: Evaluating Reasoning in Post-Trained Language Models
• Beyond Marginal Distributions: A Framework to Evaluate the Representativeness of Demographic-Aligned LLMs
• GReF: A Unified Generative Framework for Efficient Reranking via Ordered Multi-token Prediction
• Optimizing Retrieval for RAG via Reinforced Contrastive Learning
• SelecTKD: Selective Token-Weighted Knowledge Distillation for LLMs
• DynaStride: Dynamic Stride Windowing with MMCoT for Instructional Multi-Scene Captioning
• Modeling Expert Interactions in Sparse Mixture of Experts via Graph Structures
• A Survey on Efficient Vision-Language-Action Models
• Lightweight Robust Direct Preference Optimization
• LightKGG: Simple and Efficient Knowledge Graph Generation from Textual Data
• A Survey on LLM Mid-Training
• Agentic Meta-Orchestrator for Multi-task Copilots
• LooGLE v2: Are LLMs Ready for Real World Long Dependency Challenges?
• Efficient Low Rank Attention for Long-Context Inference in Large Language Models
• Flight Delay Prediction via Cross-Modality Adaptation of Large Language Models and Aircraft Trajectory Representation
• Unified Reinforcement and Imitation Learning for Vision-Language Models
• Knowledge Distillation of Uncertainty using Deep Latent Factor Model
• Difficulty-Controllable Multiple-Choice Question Generation Using Large Language Models and Direct Preference Optimization
• EdgeReasoning: Characterizing Reasoning LLM Deployment on Edge GPUs
• A Benchmark Dataset And LLMs Comparison For NFR Classification With Explainable AI
• Multilingual Text-to-Image Person Retrieval via Bidirectional Relation Reasoning and Aligning
• An Evaluation of LLMs Inference on Popular Single-board Computers
• LC-Eval: A Bilingual Multi-Task Evaluation Benchmark for Long-Context Understanding
• Train a Unified Multimodal Data Quality Classifier with Synthetic Data
• CaptionFormer: Unified Segmentation, Tracking, and Captioning for Spatio-Temporal Objects
• Map the Flow: Revealing Hidden Pathways of Information in VideoLLMs
• Mirror Speculative Decoding: Breaking the Serial Barrier in LLM Inference
• SAIL-Embedding Technical Report: Omni-modal Embedding Foundation Model
• Multi-stage Prompt Refinement for Mitigating Hallucinations in Large Language Models
• ExpVid: A Benchmark for Experiment Video Understanding & Reasoning
• Find Your Optimal Teacher: Personalized Data Synthesis via Router-Guided Multi-Teacher Distillation
• Preserving LLM Capabilities through Calibration Data Curation: From Analysis to Optimization
• Bridging Semantics & Structure for Software Vulnerability Detection using Hybrid Network Models
• The Achilles' Heel of LLMs: How Altering a Handful of Neurons Can Cripple Language Abilities
• Semantic Visual Anomaly Detection and Reasoning in AI-Generated Images
• PVDetector: Detecting Prompt Injection Attacks on Purpose-Specific LLM Agents through Policy-Violation Concept Analysis
• On the Representations of Entities in Auto-regressive Large Language Models
• Zero-shot image privacy classification with Vision-Language Models
• Understanding the Effects of Domain Finetuning on LLMs
• PhysToolBench: Benchmarking Physical Tool Understanding for MLLMs
• ProxRouter: Proximity-Weighted LLM Query Routing for Improved Robustness to Outliers
• To Sink or Not to Sink: Visual Information Pathways in Large Vision-Language Models
• Automatic Text Box Placement for Supporting Typographic Design
• RetouchLLM: Training-free Code-based Image Retouching with Vision Language Models
• Deploying Tiny LVLM Judges for Real-World Evaluation of Chart Models: Lessons Learned and Best Practices
• Mid-Training of Large Language Models: A Survey
• Efficient Discriminative Joint Encoders for Large Scale Vision-Language Reranking
• JAI-1: A Thai-Centric Large Language Model
• Compressed Convolutional Attention: Efficient Attention in a Compressed Latent Space
• Person-Centric Annotations of LAION-400M: Auditing Bias and Its Transfer to Models
• H-DDx: A Hierarchical Evaluation Framework for Differential Diagnosis
• Towards Sampling Data Structures for Tensor Products in Turnstile Streams
• MITS: Enhanced Tree Search Reasoning for LLMs via Pointwise Mutual Information
• Dirichlet-Prior Shaping: Guiding Expert Specialization in Upcycled MoEs
• ModernVBERT: Towards Smaller Visual Document Retrievers
• ASGuard: Activation-Scaling Guard to Mitigate Targeted Jailbreaking Attack
• Predicting Training Re-evaluation Curves Enables Effective Data Curriculums for LLMs
• Towards Trustworthy Lexical Simplification: Exploring Safety and Efficiency with Small LLMs
• Generalized Correctness Models: Learning Calibrated and Model-Agnostic Correctness Predictors from Historical Patterns
• From Code to Action: Hierarchical Learning of Diffusion-VLM Policies
• Pushing LLMs to Their Logical Reasoning Bound: The Role of Data Reasoning Intensity
• AstroMMBench: A Benchmark for Evaluating Multimodal Large Language Models Capabilities in Astronomy
• Analyzing and Evaluating Unbiased Language Model Watermark
• An Ensemble Framework for Unbiased Language Model Watermarking
• PCRI: Measuring Context Robustness in Multimodal Models for Enterprise Applications
• LUQ: Layerwise Ultra-Low Bit Quantization for Multimodal Large Language Models
• Evaluating Program Semantics Reasoning with Type Inference in System F
• LLMSQL: Upgrading WikiSQL for the LLM Era of Text-to-SQL
• Learning Human-Perceived Fakeness in AI-Generated Videos via Multimodal LLMs
• Linear Causal Representation Learning by Topological Ordering, Pruning, and Disentanglement
• Ground-Truthing AI Energy Consumption: Validating CodeCarbon Against External Measurements
• Customizing Visual Emotion Evaluation for MLLMs: An Open-vocabulary, Multifaceted, and Scalable Approach
• Quantifying the Impact of Structured Output Format on Large Language Models through Causal Inference
• Tiny but Mighty: A Software-Hardware Co-Design Approach for Efficient Multimodal Inference on Battery-Powered Small Devices
• Query-Centric Graph Retrieval Augmented Generation
• Seeing Through Words, Speaking Through Pixels: Deep Representational Alignment Between Vision and Language Models
• Accelerate Creation of Product Claims Using Generative AI
• GEP: A GCG-Based method for extracting personally identifiable information from chatbots built on small language models
• BurstEngine: an Efficient Distributed Framework for Training Transformers on Extremely Long Sequences of over 1M Tokens
• Tokenization and Representation Biases in Multilingual Models on Dialectal NLP Tasks
• Polarity Detection of Sustainable Detection Goals in News Text
• Let's Play Across Cultures: A Large Multilingual, Multicultural Benchmark for Assessing Language Models' Understanding of Sports
• Beyond Rephrasing: Book-Level Organization Improves Synthetic Textbook Data for Mid-Training
• Selecting Open-Weight Language Models for Zero-Shot Intent Classification: A Systematic Evaluation of 41 Models
• AHA-Memes: A Fine-Grained Multimodal Benchmark for Understanding Hate in Arabic Memes
• From Global to Local: Social Bias Transfer in CLIP
• OmniBridge: Unified Multimodal Understanding, Generation, and Retrieval via Latent Space Alignment
• RealBench: A Chinese Multi-image Understanding Benchmark Close to Real-world Scenarios
• AccessEval: Benchmarking Disability Bias in Large Language Models
• MetaEmbed: Scaling Multimodal Retrieval at Test-Time with Flexible Late Interaction
• Are VLMs Ready for Lane Topology Awareness in Autonomous Driving?
• mmExpert: Integrating Large Language Models for Comprehensive mmWave Data Synthesis and Understanding
• MANZANO: A Simple and Scalable Unified Multimodal Model with a Hybrid Vision Tokenizer
• Language-Instructed Reasoning for Group Activity Detection via Multimodal Large Language Model
• ORIC: Benchmarking Object Recognition under Contextual Incongruity in Large Vision-Language Models
• Understand Before You Generate: Self-Guided Training for Autoregressive Image Generation
• CLEAR: A Comprehensive Linguistic Evaluation of Argument Rewriting by Large Language Models
• MoE-Inference-Bench: Performance Evaluation of Mixture of Expert Large Language and Vision Models
• AToken: A Unified Tokenizer for Vision
• TENET: An Efficient Sparsity-Aware LUT-Centric Architecture for Ternary LLM Inference On Edge
• Estimating Semantic Alphabet Size for LLM Uncertainty Quantification
• Automated and Context-Aware Code Documentation Leveraging Advanced LLMs
• Positional Encoding via Token-Aware Phase Attention
• CLAIRE: A Dual Encoder Network with RIFT Loss and Phi-3 Small Language Model Based Interpretability for Cross-Modality Synthetic Aperture Radar and Optical Land Cover Segmentation
• A Dynamic Knowledge Update-Driven Model with Large Language Models for Fake News Detection
• Pluralistic Off-policy Evaluation and Alignment
• Continually Adding New Languages to Multilingual Language Models
• Enhancing Generalization in Vision-Language-Action Models by Preserving Pretrained Representations
• LoRALib: A Standardized Benchmark for Evaluating LoRA-MoE Methods
• LLMAP: LLM-Assisted Multi-Objective Route Planning with User Preferences
• TrueSkin: Towards Fair and Accurate Skin Tone Recognition and Generation
• Humanizing Automated Programming Feedback: Fine-Tuning Generative Models with Student-Written Feedback
• RefactorCoderQA: Benchmarking LLMs for Multi-Domain Coding Question Solutions in Cloud and Edge Deployment
• Towards Better Dental AI: A Multimodal Benchmark and Instruction Dataset for Panoramic X-ray Analysis
• TaDiCodec: Text-aware Diffusion Speech Tokenizer for Speech Language Modeling
• Assess and Prompt: A Generative RL Framework for Improving Engagement in Online Mental Health Communities
• WebMMU: A Benchmark for Multimodal Multilingual Website Understanding and Code Generation
• Competitive Audio-Language Models with Data-Efficient Single-Stage Training on Public Data
• HealthSLM-Bench: Benchmarking Small Language Models for Mobile and Wearable Healthcare Monitoring
• MoGU V2: Toward a Higher Pareto Frontier Between Model Usability and Security
• Spatial Reasoning with Vision-Language Models in Ego-Centric Multi-View Scenes
• MedBench-IT: A Comprehensive Benchmark for Evaluating Large Language Models on Italian Medical Entrance Examinations
• Self-Aligned Reward: Towards Effective and Efficient Reasoners
• WildScore: Benchmarking MLLMs in-the-Wild Symbolic Music Reasoning
• How Small is Enough? Empirical Evidence of Quantized Small Language Models for Automated Program Repair
• Strefer: Empowering Video LLMs with Space-Time Referring and Reasoning via Synthetic Instruction Data
• Unlearning That Lasts: Utility-Preserving, Robust, and Almost Irreversible Forgetting in LLMs
• RoboBuddy in the Classroom: Exploring LLM-Powered Social Robots for Storytelling in Learning and Integration Activities
• Implicit Reasoning in Large Language Models: A Comprehensive Survey
• Top-H Decoding: Adapting the Creativity and Coherence with Bounded Entropy in Text Generation
• ShadowNPU: System and Algorithm Co-design for NPU-Centric On-Device LLM Inference
• Improving Large Vision and Language Models by Learning from a Panel of Peers
• Kwai Keye-VL 1.5 Technical Report
• DaMoC: Efficiently Selecting the Optimal Large Language Model for Fine-tuning Domain Tasks Based on Data and Model Compression
• VideoRewardBench: Comprehensive Evaluation of Multimodal Reward Models for Video Understanding
• DriveQA: Passing the Driving Knowledge Test
• Med-RewardBench: Benchmarking Reward Models and Judges for Medical Multimodal Large Language Models
• MindGuard: Intrinsic Decision Inspection for Securing LLM Agents Against Metadata Poisoning
• KRETA: A Benchmark for Korean Reading and Reasoning in Text-Rich VQA Attuned to Diverse Visual Contexts
• NLKI: A lightweight Natural Language Knowledge Integration Framework for Improving Small VLMs in Commonsense VQA Tasks
• Scalable Object Detection in the Car Interior With Vision Foundation Models
• Ensemble Debates with Local Large Language Models for AI Alignment
• Reflective Agreement: Combining Self-Mixture of Agents with a Sequence Tagger for Robust Event Extraction
• MOSA: Mixtures of Simple Adapters Outperform Monolithic Approaches in LLM-based Multilingual ASR
• Hidden Tail: Adversarial Image Causing Stealthy Resource Consumption in Vision-Language Models
• Knowing or Guessing? Robust Medical Visual Question Answering via Joint Consistency and Contrastive Learning
• PKG-DPO: Optimizing Domain-Specific AI systems with Physics Knowledge Graphs and Direct Preference Optimization
• Unveiling Trust in Multimodal Large Language Models: Evaluation, Analysis, and Mitigation
• Nemotron-CC-Math: A 133 Billion-Token-Scale High Quality Math Pretraining Dataset
• Prompt Orchestration Markup Language
• The Hidden Cost of Readability: How Code Formatting Silently Consumes Your LLM Budget
• Evaluating Open-Source Vision Language Models for Facial Emotion Recognition against Traditional Deep Learning Models
• GLASS: Global-Local Aggregation for Inference-time Sparsification of LLMs
• Beyond Ethical Alignment: Evaluating LLMs as Artificial Moral Assistants
• Is GPT-OSS Good? A Comprehensive Evaluation of OpenAI's Latest Open Source Models
• Rethinking Safety in LLM Fine-tuning: An Optimization Perspective
• VimoRAG: Video-based Retrieval-augmented 3D Motion Generation for Motion Language Models
• Dynamic Quality-Latency Aware Routing for LLM Inference in Wireless Edge-Device Networks
• Diffusion is a code repair operator and generator
• AddressVLM: Cross-view Alignment Tuning for Image Address Localization using Large Vision-Language Models
• Benchmark Dataset Generation and Evaluation for Excel Formula Repair with LLMs
• Apriel-Nemotron-15B-Thinker
• BigCharts-R1: Enhanced Chart Reasoning with Visual Reinforcement Finetuning
• MoIIE: Mixture of Intra- and Inter-Modality Experts for Large Vision Language Models
• SHALE: A Scalable Benchmark for Fine-grained Hallucination Evaluation in LVLMs
• Learning Facts at Scale with Active Reading
• From Charts to Fair Narratives: Uncovering and Mitigating Geo-Economic Biases in Chart-to-Text
• SinLlama -- A Large Language Model for Sinhala
• Classifier Language Models: Unifying Sparse Finetuning and Adaptive Tokenization for Specialized Classification Tasks
• BASIC: Boosting Visual Alignment with Intrinsic Refined Embeddings in Multimodal Large Language Models
• Effective Training Data Synthesis for Improving MLLM Chart Understanding
• Sample-efficient LLM Optimization with Reset Replay
• LLM Unlearning Without an Expert Curated Dataset
• InfoCausalQA:Can Models Perform Non-explicit Causal Reasoning Based on Infographic?
• DP-LLM: Runtime Model Adaptation with Dynamic Layer-wise Precision Assignment
• RL-MoE: An Image-Based Privacy Preserving Approach In Intelligent Transportation System
• Do Political Opinions Transfer Between Western Languages? An Analysis of Unaligned and Aligned Multilingual LLMs
• mKG-RAG: Multimodal Knowledge Graph-Enhanced RAG for Visual Question Answering
• EvoGraph: Hybrid Directed Graph Evolution toward Software 3.0
• Collaborative Inference and Learning between Edge SLMs and Cloud LLMs: A Survey of Algorithms, Execution, and Open Challenges
• Modelling and Classifying the Components of a Literature Review
• Sustainability assessment using multimodal AI agents
• ChartCap: Mitigating Hallucination of Dense Chart Captioning
• VLM4D: Towards Spatiotemporal Awareness in Vision Language Models
• SmallKV: Small Model Assisted Compensation of KV Cache Compression for Efficient LLM Inference
• EAC-MoE: Expert-Selection Aware Compressor for Mixture-of-Experts Large Language Models
• SketchAgent: Generating Structured Diagrams from Hand-Drawn Sketches
• From Generator to Embedder: Harnessing Innate Abilities of Multimodal LLMs via Building Zero-Shot Discriminative Embedding Model
• Evaluating Contrast Localizer for Identifying Causal Units in Social & Mathematical Tasks in Language Models
• What's Taboo for You? - An Empirical Evaluation of LLMs Behavior Toward Sensitive Content
• XAutoLM: Efficient Fine-Tuning of Language Models via Meta-Learning and AutoML
• MoHoBench: Assessing Honesty of Multimodal Large Language Models via Unanswerable Visual Questions
• Multimodal LLMs as Customized Reward Models for Text-to-Image Generation
• On The Role of Pretrained Language Models in General-Purpose Text Embeddings: A Survey
• Enhancing Spatial Reasoning through Visual and Textual Thinking
• CodeNER: Code Prompting for Named Entity Recognition
• Infogen: Generating Complex Statistical Infographics from Documents
• How Much Do Large Language Model Cheat on Evaluation? Benchmarking Overestimation under the One-Time-Pad-Based Framework
• True Multimodal In-Context Learning Needs Attention to the Visual Context
• The Benchmark Illusion: Pruned LLMs Can Pass Multiple Choice but Fail to Answer
• Beyond Isolated Dots: Benchmarking Structured Table Construction as Deep Knowledge Extraction
• Scale Your Instructions: Enhance the Instruction-Following Fidelity of Unified Image Generation Model by Self-Adaptive Attention Scaling
• Evaluating the Effectiveness of Cost-Efficient Large Language Models in Benchmark Biomedical Tasks
• AD²-Bench: A Hierarchical CoT Benchmark for MLLM in Autonomous Driving under Adverse Conditions
• Do LLMs Give Psychometrically Plausible Responses in Educational Assessments?
• Characterizing State Space Model (SSM) and SSM-Transformer Hybrid Language Model Performance with Long Context Length
• AutoVDC: Automated Vision Data Cleaning Using Vision-Language Models
• Probing for Arithmetic Errors in Language Models
• Improving Contextual ASR via Multi-grained Fusion with Large Language Models
• POLYCHARTQA: Benchmarking Large Vision-Language Models with Multilingual Chart Question Answering
• IAM: Efficient Inference through Attention Mapping between Different-scale LLMs
• Dataset of News Articles with Provenance Metadata for Media Relevance Assessment
• Beyond Bias Scores: Unmasking Vacuous Neutrality in Small Language Models
• Temperature and Persona Shape LLM Agent Consensus With Minimal Accuracy Gains in Qualitative Coding
• EtiCor++: Towards Understanding Etiquettical Bias in LLMs
• CoralVQA: A Large-Scale Visual Question Answering Dataset for Coral Reef Image Understanding
• FaceLLM: A Multimodal Large Language Model for Face Understanding
• Evaluating LLMs Across Multi-Cognitive Levels: From Medical Knowledge Mastery to Scenario-Based Problem Solving
• Uncertainty-Driven Expert Control: Enhancing the Reliability of Medical Vision-Language Models
• Scaling Laws for Optimal Data Mixtures
• Multilingual Multimodal Software Developer for Code Generation
• Hybrid Systolic Array Accelerator with Optimized Dataflow for Edge Large Language Model Inference
• Transfer Learning and Mixup for Fine-Grained Few-Shot Fungi Classification
• Orchestration for Domain-specific Edge-Cloud Language Models
• Multi-Actor Generative Artificial Intelligence as a Game Engine
• Beyond the Linear Separability Ceiling: Aligning Representations in VLMs
• Towards Privacy-Preserving and Personalized Smart Homes via Tailored Small Language Models
• Prompt Perturbations Reveal Human-Like Biases in Large Language Model Survey Responses
• CheXPO: Preference Optimization for Chest X-ray VLMs with Counterfactual Rationale
• Beyond Scale: Small Language Models are Comparable to GPT-4 in Mental Health Understanding
• Exploring Task Performance with Interpretable Models via Sparse Auto-Encoders
• BlueLM-2.5-3B Technical Report
• Vision-Language Models Can't See the Obvious
• Small Is Enough: Per-User Style Rewriting of AI-Edited Text via LoRA Adapters
• A Technical Survey of Reinforcement Learning Techniques for Large Language Models
• Conformal Information Pursuit for Interactively Guiding Large Language Models
• BMMR: A Large-Scale Bilingual Multimodal Multi-Discipline Reasoning Dataset
• The Asymmetric Effects of Knowledge Distillation on Bias in Small Language Models
• Kwai Keye-VL Technical Report
• Emotionally Intelligent Task-oriented Dialogue Systems: Architecture, Representation, and Optimisation
• CAVALRY-V: A Large-Scale Generator Framework for Adversarial Attacks on Video MLLMs
• VisioMath: Benchmarking Figure-based Mathematical Reasoning in LMMs
• Failure by Interference: Language Models Make Balanced Parentheses Errors When Faulty Mechanisms Overshadow Sound Ones
• Enhancing Reasoning Capabilities of Small Language Models with Blueprints and Prompt Template Search
• SynMotion: Semantic-Visual Adaptation for Motion Customized Video Generation
• How large language models judge and influence human cooperation
• Theoretical Modeling of Large Language Model Self-Improvement Training Dynamics Through Solver-Verifier Gap
• MoCa: Modality-aware Continual Pre-training Makes Better Bidirectional Multimodal Embeddings
• MusiXQA: Advancing Visual Music Understanding in Multimodal Large Language Models
• EgoM2P: Egocentric Multimodal Multitask Pretraining
• Snap, Segment, Deploy: A Visual Data and Detection Pipeline for Wearable Industrial Assistants
• BEST-Route: Adaptive LLM Routing with Test-Time Optimal Compute
• Remote Sensing Large Vision-Language Model: Semantic-augmented Multi-level Alignment and Semantic-aware Expert Modeling
• Zero-Shot Learning for Obsolescence Risk Forecasting
• Exploring the Design Space of 3D MLLMs for CT Report Generation
• Agent-RewardBench: Towards a Unified Benchmark for Reward Modeling across Perception, Planning, and Safety in Real-World Multimodal Agents
• OctoThinker: Mid-training Incentivizes Reinforcement Learning Scaling
• Fine-grained Token Allocation Via Operation Pruning for Efficient MLLMs
• PrivacyXray: Detecting Privacy Breaches in LLMs through Semantic Consistency and Probability Certainty
• Synthetic Visual Genome
• Generalizing vision-language models to novel domains: A comprehensive survey
• LOGICPO: Efficient Translation of NL-based Logical Problems to FOL using LLMs and Preference Optimization
• RePIC: Reinforced Post-Training for Personalizing Multi-Modal Language Models
• Spiritual-LLM : Gita Inspired Mental Health Therapy In the Era of LLMs
• See-in-Pairs: Reference Image-Guided Comparative Vision-Language Models for Medical Diagnosis
• InspireDebate: Multi-Dimensional Subjective-Objective Evaluation-Guided Reasoning and Optimization for Debating
• ConsumerBench: Benchmarking Generative AI Applications on End-User Devices
• WebUIBench: A Comprehensive Benchmark for Evaluating Multimodal Large Language Models in WebUI-to-Code
• The Hidden Cost of an Image: Quantifying the Energy Consumption of AI Image Generation
• Towards Effective Complementary Security Analysis using Large Language Models
• Differentiation-Based Extraction of Proprietary Data from Fine-Tuned LLMs
• Structured Attention Matters to Multimodal LLMs in Document Understanding
• Quality over Quantity: An Effective Large-Scale Data Reduction Strategy Based on Pointwise V-Information
• Demystifying the Visual Quality Paradox in Multimodal Large Language Models
• Can AI Validate Science? Benchmarking LLMs for Accurate Scientific Claim → Evidence Reasoning
• A Neurosymbolic Agent System for Compositional Visual Reasoning
• Alignment between Brains and AI: Evidence for Convergent Evolution across Modalities, Scales and Training Trajectories
• Quantile Regression with Large Language Models for Price Prediction
• A Comparative Study of Task Adaptation Techniques of Large Language Models for Identifying Sustainable Development Goals
• GenRecal: Generation after Recalibration from Large to Small Vision-Language Models
• SciVer: Evaluating Foundation Models for Multimodal Scientific Claim Verification
• Utility-Driven Speculative Decoding for Mixture-of-Experts
• PeRL: Permutation-Enhanced Reinforcement Learning for Interleaved Vision-Language Reasoning
• Can Vision Language Models Understand Mimed Actions?
• Optimal Embedding Learning Rate in LLMs: The Effect of Vocabulary Size
• Adapting Lightweight Vision Language Models for Radiological Visual Question Answering
• ASCD: Attention-Steerable Contrastive Decoding for Reducing Hallucination in MLLM
• MoTE: Mixture of Ternary Experts for Memory-efficient Large Multimodal Models
• MotiveBench: How Far Are We From Human-Like Motivational Reasoning in Large Language Models?
• SPOT: Bridging Natural Language and Geospatial Search for Investigative Journalists
• Enhancing Goal-oriented Proactive Dialogue Systems via Consistency Reflection and Correction
• PRISM2: Unlocking Multi-Modal General Pathology AI with Clinical Dialogue
• Assessing the Limits of In-Context Learning beyond Functions using Partially Ordered Relation
• Rethinking Explainability in the Era of Multimodal AI
• Align-then-Unlearn: Embedding Alignment for LLM Unlearning
• Dual-Priv Pruning : Efficient Differential Private Fine-Tuning in Multimodal Large Language Models
• HypER: Literature-grounded Hypothesis Generation and Distillation with Provenance
• Jailbreak Transferability Emerges from Shared Representations
• Chain of Methodologies: Scaling Test Time Computation without Training
• ComplexBench-Edit: Benchmarking Complex Instruction-Driven Image Editing via Compositional Dependencies
• ConsistencyChecker: Tree-based Evaluation of LLM Generalization Capabilities
• OpenUnlearning: Accelerating LLM Unlearning via Unified Benchmarking of Methods and Metrics
• Theorem-of-Thought: A Multi-Agent Framework for Abductive, Deductive, and Inductive Reasoning in Language Models
• Curriculum-Guided Layer Scaling for Language Model Pretraining
• MTabVQA: Evaluating Multi-Tabular Reasoning of Language Models in Visual Space
• Persona-driven Simulation of Voting Behavior in the European Parliament with Large Language Models
• The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity
• VRBench: A Benchmark for Multi-Step Reasoning in Long Narrative Videos
• Team QUST at SemEval-2025 Task 10: Evaluating Large Language Models in Multiclass Multi-label Classification of News Entity Framing
• Burn After Reading: Do Multimodal Large Language Models Truly Capture Order of Events in Image Sequences?
• Discovering Hierarchical Latent Capabilities of Language Models via Causal Representation Learning
• Vision-EKIPL: External Knowledge-Infused Policy Learning for Visual Reasoning
• Scaling Laws for Uncertainty in Deep Learning
• HeartcareGPT: A Unified Multimodal ECG Suite for Dual Signal-Image Modeling and Understanding
• Token Signature: Predicting Chain-of-Thought Gains with Token Decoding Feature in Large Language Models
• Movie Facts and Fibs (MF²): A Benchmark for Long Movie Understanding
• Unleashing Hour-Scale Video Training for Long Video-Language Understanding
• Mobile Traffic Prediction using LLMs with Efficient In-context Demonstration Selection
• A MISMATCHED Benchmark for Scientific Natural Language Inference
• Structured Labeling Enables Faster Vision-Language Models for End-to-End Autonomous Driving
• AuthGuard: Generalizable Deepfake Detection via Language Guidance
• T-MAC: CPU Renaissance via Table Lookup for Low-Bit LLM Deployment on Edge
• R³-VQA: "Read the Room" by Video Social Reasoning
• REVEAL: Multi-turn Evaluation of Image-Input Harms for Vision LLM
• STELLA: A Multimodal LLM for Protein Functional Annotation via Unified Sequence-Structure Encoding
• Learning to Insert [PAUSE] Tokens for Better Reasoning
• RedDebate: Safer Responses Through Multi-Agent Red Teaming Debates
• Watermarking Degrades Alignment in Language Models: Analysis and Mitigation
• A Statistical Physics of Language Model Reasoning
• Vision Remember: Recovering Visual Information in Efficient LVLM with Vision Feature Resampling
• PARC: A Quantitative Framework Uncovering the Symmetries within Vision Language Models
• Conditioning Large Language Models on Legal Systems? Detecting Punishable Hate Speech
• Learning Together to Perform Better: Teaching Small-Scale LLMs to Collaborate via Preferential Rationale Tuning
• FlySearch: Exploring how vision-language models explore
• LocalGPT: Benchmarking and Advancing Large Language Models for Local Life Services in Meituan
• Beyond Text Compression: Evaluating Tokenizers Across Scales
• A Controllable Examination for Long-Context Language Models
• StochasTok: Improving Fine-Grained Subword Understanding in LLMs
• Self-Refining Language Model Anonymizers via Adversarial Distillation
• KDRL: Post-Training Reasoning LLMs via Unified Knowledge Distillation and Reinforcement Learning
• Contra4: Evaluating Contrastive Cross-Modal Reasoning in Audio, Video, Image, and 3D
• Abstractive Visual Understanding of Multi-modal Structured Knowledge: A New Perspective for MLLM Evaluation
• Adversarial Attacks in Multimodal Systems: A Practitioner's Survey
• Zoom-Refine: Boosting High-Resolution Multimodal Understanding via Localized Zoom and Self-Refinement
• DRAG: Distilling RAG for SLMs from LLMs to Transfer Knowledge and Mitigate Hallucination via Evidence and Graph-based Distillation
• Improve MLLM Benchmark Efficiency through Interview
• Improving Automatic Evaluation of Large Language Models (LLMs) in Biomedical Relation Extraction via LLMs-as-the-Judge
• CReFT-CAD: Boosting Orthographic Projection Reasoning for CAD via Reinforcement Fine-Tuning
• SATA-BENCH: Select All That Apply Benchmark for Multiple Choice Questions
• RoboOS: A Hierarchical Embodied Framework for Cross-Embodiment and Multi-Agent Collaboration
• Seeing the Abstract: Translating the Abstract Language for Vision Language Models
• DefenderBench: A Toolkit for Evaluating Language Agents in Cybersecurity Environments
• The Road to Generalizable Neuro-Symbolic Learning Should be Paved with Foundation Models
• Reflect, Retry, Reward: Self-Improving LLMs via Reinforcement Learning
• Adaptable Cardiovascular Disease Risk Prediction from Heterogeneous Data using Large Language Models
• On Fairness of Task Arithmetic: The Role of Task Vectors
• Minifinetuning: Low-Data Generation Domain Adaptation through Corrective Self-Distillation
• On the Expressive Power of Mixture-of-Experts for Structured Complex Tasks
• Structuring Radiology Reports: Challenging LLMs with Lightweight Models
• Beyond Semantic Entropy: Boosting LLM Uncertainty Quantification with Pairwise Semantic Similarity
• FinMME: Benchmark Dataset for Financial Multi-Modal Reasoning Evaluation
• MIRAGE: Assessing Hallucination in Multimodal Reasoning Chains of MLLM
• Tag-Evol: Achieving Efficient Instruction Evolving via Tag Injection
• ThinkGeo: Evaluating Tool-Augmented Agents for Remote Sensing Tasks
• VF-Eval: Evaluating Multimodal LLMs for Generating Feedback on AIGC Videos
• Using Knowledge Graphs to harvest datasets for efficient CLIP model training
• Efficient AllReduce with Stragglers
• UniRL: Self-Improving Unified Multimodal Models via Supervised and Reinforcement Learning
• DIP-R1: Deep Inspection and Perception with RL Looking Through and Understanding Complex Scenes
• MEDAL: A Framework for Benchmarking LLMs as Multilingual Open-Domain Dialogue Evaluators
• Pre-Training Curriculum for Multi-Token Prediction in Language Models
• LaMDAgent: An Autonomous Framework for Post-Training Pipeline Optimization via LLM Agents
• From Large AI Models to Agentic AI: A Tutorial on Future Intelligent Communications
• New Tools are Needed for Tracking Adherence to AI Model Behavioral Use Clauses
• Scrapers selectively respect robots.txt directives: evidence from a large-scale empirical study
• AutoJudger: An Agent-Driven Framework for Efficient Benchmarking of MLLMs
• EntroLLM: Entropy Encoded Weight Compression for Efficient Large Language Model Inference on Edge Devices
• OmniResponse: Online Multimodal Conversational Response Generation in Dyadic Interactions
• What happens when generative AI models train recursively on each others' outputs?
• Will It Still Be True Tomorrow? Multilingual Evergreen Question Classification to Improve Trustworthy QA
• Large Language Models for IT Automation Tasks: Are We There Yet?
• Embodied AI with Foundation Models for Mobile Service Robots: A Systematic Review
• VisualToolAgent (VisTA): A Reinforcement Learning Framework for Visual Tool Selection
• MMGeoLM: Hard Negative Contrastive Learning for Fine-Grained Geometric Understanding in Large Multimodal Models
• Attention! Your Vision Language Model Could Be Maliciously Manipulated
• Correlating instruction-tuning (in multimodal models) with vision-language processing (in the brain)
• Can Compressed LLMs Truly Act? An Empirical Evaluation of Agentic Capabilities in LLM Compression
• Balancing Computation Load and Representation Expressivity in Parallel Hybrid Neural Networks
• Teaching Models to Understand (but not Generate) High-risk Data
• Delving into Multilingual Ethical Bias: The MSQAD with Statistical Hypothesis Tests for Large Language Models
• Jodi: Unification of Visual Generation and Understanding via Joint Modeling
• Are Vision Language Models Ready for Clinical Diagnosis? A 3D Medical Benchmark for Tumor-centric Visual Question Answering
• 100-LongBench: Are de facto Long-Context Benchmarks Literally Evaluating Long-Context Ability?
• The Price of Format: Diversity Collapse in LLMs
• ReFineVLA: Reasoning-Aware Teacher-Guided Transfer Fine-Tuning
• S²GPT-PINNs: Sparse and Small models for PDEs
• Sci-LoRA: Mixture of Scientific LoRAs for Cross-Domain Lay Paraphrasing
• VISTA: Vision-Language Inference for Training-Free Stock Time-Series Analysis
• Rethinking Causal Mask Attention for Vision-Language Inference
• Efficient Long CoT Reasoning in Small Language Models
• Instructify: Demystifying Metadata to Visual Instruction Tuning Data Conversion
• Illuminating Visual Identity in Universal Multimodal Embeddings
• On the Design of KL-Regularized Policy Gradient Algorithms for LLM Reasoning
• FS-DAG: Few Shot Domain Adapting Graph Networks for Visually Rich Document Understanding
• RBench-V: A Primary Assessment for Visual Reasoning Models with Multi-modal Outputs
• From Evaluation to Defense: Advancing Safety in Video Large Language Models
• Large Language Models for Predictive Analysis: How Far Are They?
• MDIT-Bench: Evaluating the Dual-Implicit Toxicity in Large Multimodal Models
• Data Doping or True Intelligence? Evaluating the Transferability of Injected Knowledge in LLMs
• Continually Self-Improving Language Models for Bariatric Surgery Question--Answering
• Date Fragments: A Hidden Bottleneck of Tokenization for Temporal Reasoning
• Circle-RoPE: Cone-like Decoupled Rotary Positional Embedding for Large Vision-Language Models
• Understanding Differential Transformer Unchains Pretrained Self-Attentions
• From EduVisBench to EduVisAgent: A Benchmark and Multi-Agent Framework for Reasoning-Driven Pedagogical Visualization
• LLM as Effective Streaming Processor: Bridging Streaming-Batch Mismatches with Group Position Encoding
• Selective Invocation for Multilingual ASR: A Cost-effective Approach Adapting to Speech Recognition Difficulty
• PolyMicros: Bootstrapping a Foundation Model for Polycrystalline Material Structure
• STAR-R1: Spatial TrAnsformation Reasoning by Reinforcing Multimodal LLMs
• Better Safe Than Sorry? Overreaction Problem of Vision Language Models in Visual Emergency Recognition
• Enhancing Large Language Models for Detecting Mental Manipulation via Annotation-Free Data Augmentation and Anti-Curriculum Distillation
• Highlighting What Matters: Promptable Embeddings for Attribute-Focused Image Retrieval
• Instruction-Conditioned Exploration for Reinforcement Learning with Self-Distillation to an Unconditioned Policy
• CineTechBench: A Benchmark for Cinematographic Technique Understanding and Generation
• ReasonCast: Towards Explainable Time Series Forecasting with Reasoning
• Streamline Without Sacrifice -- Squeeze out Computation Redundancy in LMM
• Leveraging Online Data to Enhance Medical Knowledge in a Small Persian Language Model
• TimeCausality: Evaluating the Causal Ability in Time Dimension for Vision Language Models
• Understanding the Anchoring Effect of LLM with Synthetic Data: Existence, Mechanism, and Potential Mitigations
• KERL: Knowledge-Enhanced Personalized Recipe Recommendation using Large Language Models
• RADAR: Enhancing Radiology Report Generation with Supplementary Knowledge Injection
• MORALISE: A Structured Benchmark for Moral Alignment in Visual Language Models
• UniGen: Enhanced Training & Test-Time Strategies for Unified Multimodal Understanding and Generation
• Robust and Efficient AI-Based Attack Recovery in Autonomous Drones
• Quaff: Quantized Parameter-Efficient Fine-Tuning under Outlier Spatial Stability Hypothesis
• Power Lines: Scaling Laws for Weight Decay and Batch Size in LLM Pre-training
• Occult: Optimizing Collaborative Communication across Experts for Accelerated Parallel MoE Training and Inference
• LLM-KG-Bench 3.0: A Compass for SemanticTechnology Capabilities in the Ocean of LLMs
• Learnware of Language Models: Specialized Small Language Models Can Do Big
• A3 : an Analytical Low-Rank Approximation Framework for Attention
• Personalized Fine-Tuning with Controllable Synthetic Speech from LLM-Generated Transcripts for Dysarthric Speech Recognition
• Video-GPT via Next Clip Diffusion
• Security practices in AI development
• Fine-grained Contrastive Learning for ECG-Report Alignment with Waveform Enhancement
• Gaokerena: A Small Persian Medical Language Model Family
• UniMoCo: Unified Modality Completion for Robust Multi-Modal Embeddings
• ChartEdit: How Far Are MLLMs From Automating Chart Analysis? Evaluating MLLMs' Capability via Chart Editing
• ProxyPrompt: Securing System Prompts against Prompt Extraction Attacks
• Dynam3D: Dynamic Layered 3D Tokens Empower VLM for Vision-and-Language Navigation
• DRAGON: A Large-Scale Dataset of Realistic Images Generated by Diffusion Models
• LARGO: Latent Adversarial Reflection through Gradient Optimization for Jailbreaking LLMs
• A Systematic Analysis of Base Model Choice for Reward Modeling
• WildDoc: How Far Are We from Achieving Comprehensive and Robust Document Understanding in the Wild?
• Ranked Voting based Self-Consistency of Large Language Models
• A Modular Approach for Clinical SLMs Driven by Synthetic Data with Pre-Instruction Tuning, Model Merging, and Clinical-Tasks Alignment
• RouteNator: A Router-Based Multi-Modal Architecture for Generating Synthetic Training Data for Function Calling LLMs
• FaceShield: Explainable Face Anti-Spoofing with Multimodal Large Language Models
• Flash-VL 2B: Optimizing Vision-Language Model Performance for Ultra-Low Latency and High Throughput
• Adaptive Schema-aware Event Extraction with Retrieval-Augmented Generation
• CAD-Coder:Text-Guided CAD Files Code Generation
• Trustworthiness Costs of Domain Adaptation in Small Language Models:A Cross-Architecture Empirical Study
• Resource-Efficient Language Models: Quantization for Fast and Accessible Inference
• Achieving Scalable Robot Autonomy via neurosymbolic planning using lightweight local LLM
• Judging the Judges: Can Large Vision-Language Models Fairly Evaluate Chart Comprehension and Reasoning?
• Large Language Models for Computer-Aided Design: A Survey
• Reproducibility, Replicability, and Insights into Visual Document Retrieval with Late Interaction
• Circuit Partitioning Using Large Language Models for Quantum Compilation and Simulations
• Multi-modal Synthetic Data Training and Model Collapse: Insights from VLMs and Diffusion Models
• The ML.ENERGY Benchmark: Toward Automated Inference Energy Measurement and Optimization
• Natural Reflection Backdoor Attack on Vision Language Model for Autonomous Driving
• Challenging GPU Dominance: When CPUs Outperform for On-Device LLM Inference
• Exploring the Feasibility of Multilingual Grammatical Error Correction with a Single LLM up to 9B parameters: A Comparative Study of 17 Models
• Privacy-Preserving Transformers: SwiftKey's Differential Privacy Implementation
• LiteLMGuard: Seamless and Lightweight On-Device Prompt Filtering for Safeguarding Small Language Models against Quantization-induced Risks and Vulnerabilities
• SITE: towards Spatial Intelligence Thorough Evaluation
• LSRP: A Leader-Subordinate Retrieval Framework for Privacy-Preserving Cloud-Device Collaboration
• Attention Sinks as Internal Signals for Hallucination Detection in Large Language Models
• Can small language models handle context-summarized multi-turn customer-service QA? A synthetic data-driven comparative evaluation
• Towards Safer Pretraining: Analyzing and Filtering Harmful Content in Webscale datasets for Responsible LLMs
• Enhancing LLM Code Generation: A Systematic Evaluation of Multi-Agent Collaboration and Runtime Debugging for Improved Accuracy, Reliability, and Latency
• Amplifying Your Social Media Presence: Personalized Influential Content Generation with LLMs
• Identifying the Geographic Foci of US Local News
• Position: Foundation Models Need Digital Twin Representations
• Learning When to Act or Refuse: Guarding Agentic Reasoning Models for Safe Multi-Step Tool Use
• Reasoning Capabilities and Invariability of Large Language Models
• AVA: Towards Agentic Video Analytics with Vision Language Models
• Improving Topic Modeling by Distilling Soft Labels from Language Models
• In-Place Test-Time Training
• On-Device Generative AI for GDPR-Compliant Visual Monitoring: Natural Language Alerts from Local Object Detection
• Phi-4-reasoning Technical Report
• MAC-Tuning: LLM Multi-Compositional Problem Reasoning with Enhanced Knowledge Boundary Awareness
• Reinforced MLLM: A Survey on RL-Based Reasoning in Multimodal Large Language Models
• X-Fusion: Introducing New Modality to Frozen Large Language Models
• Classifier-to-Bias: Toward Unsupervised Automatic Bias Detection for Visual Classifiers
• SpaRE: Enhancing Spatial Reasoning in Vision-Language Models with Synthetic Data
• Antidote: A Unified Framework for Mitigating LVLM Hallucinations in Counterfactual Presupposition and Object Perception
• HyPerAlign: Interpretable Personalized LLM Alignment via Hypothesis Generation
• Enhancing Non-Core Language Instruction-Following in Speech LLMs via Semi-Implicit Cross-Lingual CoT Reasoning
• Brief Is Better: Non-Monotonic Chain-of-Thought Budget Effects in Function-Calling Language Agents
• Structured Distillation for Personalized Agent Memory: 11x Token Reduction with Retrieval Preservation
• Zero-shot adaptable task planning for autonomous construction robots: a comparative study of lightweight single and multi-AI agent systems
• VIST-GPT: Ushering in the Era of Visual Storytelling with LLMs?
• Hallucinations and Key Information Extraction in Medical Texts: A Comprehensive Assessment of Open-Source Large Language Models
• Small Models, Big Tasks: An Exploratory Empirical Study on Small Language Models for Function Calling
• HRScene: How Far Are VLMs from Effective High-Resolution Image Understanding?
• Evaluating Evaluation Metrics -- The Mirage of Hallucination Detection
• RAG LLMs are Not Safer: A Safety Analysis of Retrieval-Augmented Generation for Large Language Models
• Pocket RAG: On-Device RAG for First Aid Guidance in Offline Mobile Environment
• In-Browser Agents for Search Assistance
• Towards Harnessing the Collaborative Power of Large and Small Models for Domain Tasks
• Breaking the Modality Barrier: Universal Embedding Learning with Multimodal LLMs
• BadMoE: Backdooring Mixture-of-Experts LLMs via Optimizing Routing Triggers and Infecting Dormant Experts
• Fast Autoregressive Models for Continuous Latent Generation
• Benchmarking Multimodal Mathematical Reasoning with Explicit Visual Dependency
• SMART: Tuning a symbolic music generation system with an audio domain aesthetic reward
• Traceable LLM-Generated Hazard Scenarios for Operational Safety Analysis of Aviation Systems Using ASRS Reports
• Recurrent Residual Quantization: A Progressive Multi-Precision Representation for LLMs
• Probing the Visualization Literacy of Vision Language Models: the Good, the Bad, and the Ugly
• Language Models Are Implicitly Continuous
• Ternarization of Vision Language Models for use on edge devices
• Towards Visual Text Grounding of Multimodal Large Language Model
• Safety Pretraining: Toward the Next Generation of Safe AI
• Optimizing Large Language Models: Metrics, Energy Efficiency, and Case Study Insights
• Algorithm Discovery With LLMs: Evolutionary Search Meets Reinforcement Learning
• DreamO: A Unified Framework for Image Customization
• Lightweight Latent Verifiers for Efficient Meta-Generation Strategies
• FinNLI: Novel Dataset for Multi-Genre Financial Natural Language Inference Benchmarking
• ArXivBench: When You Should Avoid Using ChatGPT for Academic Writing
• Decoding Recommendation Behaviors of In-Context Learning LLMs Through Gradient Descent
• FaceInsight: A Multimodal Large Language Model for Face Perception
• Kuwain 1.5B: An Arabic SLM via Language Injection
• Efficient Pretraining Length Scaling
• Object-Level Verbalized Confidence Calibration in Vision-Language Models via Semantic Perturbation
• Synergistic Weak-Strong Collaboration by Aligning Preferences
• In-context Ranking Preference Optimization
• Video-MMLU: A Massive Multi-Discipline Lecture Understanding Benchmark
• How Well Can General Vision-Language Models Learn Medicine By Watching Public Educational Videos?
• Manipulating Multimodal Agents via Cross-Modal Prompt Injection
• An Empirical Study of LLM Reasoning Ability Under Strict Output Length Constraint
• Learning to Attribute with Attention
• One Jump Is All You Need: Short-Cutting Transformers for Early Exit Prediction with One Jump to Fit All Exit Levels
• ChartQA-X: Generating Explanations for Visual Chart Reasoning
• Embodied-R: Collaborative Framework for Activating Embodied Spatial Reasoning in Foundation Models via Reinforcement Learning
• Dense Backpropagation Improves Training for Sparse Mixture-of-Experts
• Position: The Most Expensive Part of an LLM should be its Training Data
• FLIP Reasoning Challenge
• Enhancing Autonomous Driving Systems with On-Board Deployed Large Language Models
• Omni²: Unifying Omnidirectional Image Generation and Editing in an Omni Model
• Resampling Benchmark for Efficient Comprehensive Evaluation of Large Vision-Language Models
• Breaking the Data Barrier -- Building GUI Agents Through Task Generalization
• VDocRAG: Retrieval-Augmented Generation over Visually-Rich Documents
• DeepCompile: A Compiler-Driven Approach to Optimizing Distributed Deep Learning Training
• Cat, Rat, Meow: On the Alignment of Language Model and Human Term-Similarity Judgments
• Scaling Laws for Native Multimodal Models
• Plan-and-Refine: Diverse and Comprehensive Retrieval-Augmented Generation
• AttentionDefense: Leveraging System Prompt Attention for Explainable Defense Against Novel Jailbreaks
• Gen3DEval: Using vLLMs for Automatic Evaluation of Generated 3D Objects
• PR-Attack: Coordinated Prompt-RAG Attacks on Retrieval-Augmented Generation in Large Language Models via Bilevel Optimization
• Missing Premise exacerbates Overthinking: Are Reasoning Models losing Critical Thinking Skill?
• Encoder-Decoder Gemma: Improving the Quality-Efficiency Trade-Off via Adaptation
• Mosaic: Composite Projection Pruning for Resource-efficient LLMs
• ChartQAPro: A More Diverse and Challenging Benchmark for Chart Question Answering
• Do LLM Evaluators Prefer Themselves for a Reason?
• List of large language models [wikipedia]

Discussions

• Phi-3 Technical Report [hn, 411 points, 130 comments]
• Phi-3 is out, from MSFT. 1) Mini = small enough to deploy on a phone. 2) Medium = small enough to run on a laptop, but better than GPT-3.5. 3) They claim the advance is almost entirely due to data *qu [bsky, 14 points, 4 comments]
• arxiv.org/abs/2404.14219 なんだとお?! スマホでGPT-3.5クラスやと?! [bsky, 2 points, 1 comments]
• arxiv.org/abs/2404.14219 [bsky, 1 points, 0 comments]
• Microsoft Launches Phi-3 (arxiv.org) Main Link | Discussion [bsky, 0 points, 0 comments]
• arxiv.org/abs/2404.14219 [bsky, 0 points, 1 comments]
• Phi3-small, our new open model, at 7B parameters rivals GPT3.5, proving the quality of training data matters more than pure size: https://arxiv.org/pdf/2404.14219.pdf [bsky, 0 points, 0 comments]
• Phi-3 Technical Report https://arxiv.org/abs/2404.14219 [bsky, 0 points, 0 comments]

Related

• Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs
• Phi-4 Technical Report
• MiniCPM-V: A GPT-4V Level MLLM on Your Phone
• Textbooks Are All You Need II: phi-1.5 technical report
• Textbooks Are All You Need