NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Rails

2023/10/16 by Traian Rebedea, Razvan Dinu, Rebedea, Traian +7 · 93 citations
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Speech and dialogue systems #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2310.10501

openalex publication_date 2023/10/16 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems. Guardrails (or rails for short) are a specific way of controlling the output of an LLM, such as not talking about topics considered harmful, following a predefined dialogue path, using a particular language style, and more. There are several mechanisms that allow LLM providers and developers to add guardrails that are embedded into a specific model at training, e.g. using model alignment. Differently, using a runtime inspired from dialogue management, NeMo Guardrails allows developers to add programmable rails to LLM applications - these are user-defined, independent of the underlying LLM, and interpretable. Our initial results show that the proposed approach can be used with several LLM providers to develop controllable and safe LLM applications using programmable rails.

Cited by

• Agent Security is a Systems Problem
• ActPlane: Programmable OS-Level Policy Enforcement for Agent Harnesses
• What Can Be Enforced? A Theory of Certified Runtime Safety for Tool-Using Agents
• SAGE: Safety-First Defense-in-Depth Guardrails for Verified Lifecycle Control of High-Impact Generative AI
• Influence of Prompt Engineering on Small Language Models for Guarded Query Routing
• Semi-Supervised Learning for Large Language Models Safety and Content Moderation
• ARBITER: AI-Driven Filtering for Role-Based Access Control
• Efficient Jailbreak Mitigation Using Semantic Linear Classification in a Multi-Staged Pipeline
• QuadSentinel: Sequent Safety for Machine-Checkable Control in Multi-agent Systems
• CAPE: Capability Achievement via Policy Execution
• Trusted AI Agents in the Cloud
• Training-Free Policy Violation Detection via Activation-Space Whitening in LLMs
• Evaluating the Robustness of Large Language Model Safety Guardrails Against Adversarial Attacks
• Systems Security Foundations for Agentic Computing
• Toward a Safe Internet of Agents
• Are LLMs Good Safety Agents or a Propaganda Engine?
• Proposal of an AI-Based Support Assistant for the ALICE-FIT Detector Setup at CERN
• Beyond Task-Oriented and Chitchat Dialogues: Proactive and Transition-Aware Conversational Agents
• A Self-Improving Architecture for Dynamic Safety in Large Language Models
• Cognition Envelopes for Bounded Decision Making in Autonomous UAS Operations
• Broken-Token: Filtering Obfuscated Prompts by Counting Characters-Per-Token
• NetEcho: From Real-World Streaming Side-Channels to Full LLM Conversation Recovery
• SkillGate: Cost Efficient Runtime Malicious Skill File Detection in Coding Agents
• Choosing Where and How to Moderate: End-to-End Trade-offs in Filter Placement and Response Rewriting
• Agentic Relationship Harm: Benchmarking and Gating Relational Manipulation in AI Agents
• The Promptware Kill Chain: How Prompt Injections Gradually Evolved Into a Multistep Malware Delivery Mechanism
• Where to Search: Measure the Prior-Structured Search Space of LLM Agents
• From Refusal to Recovery: A Control-Theoretic Approach to Generative AI Guardrails
• An AI-Based Behavioral Health Safety Filter and Dataset for Identifying Mental Health Crises in Text-Based Conversations
• Intelligent AI Delegation
• RAG Makes Guardrails Unsafe? Investigating Robustness of Guardrails under RAG-style Contexts
• VeriGuard: Enhancing LLM Agent Safety via Verified Code Generation
• Toxicity in Online Platforms and AI Systems: A Survey of Needs, Challenges, Mitigations, and Future Directions
• Large-Scale Constraint Generation -- Can LLMs Parse Hundreds of Constraints?
• Cognition-of-Thought Elicits Social-Aligned Reasoning in Large Language Models
• Taxonomy of Comprehensive Safety for Clinical Agents
• DecipherGuard: Understanding and Deciphering Jailbreak Prompts for a Safer Deployment of Intelligent Software Systems
• Enterprise AI Must Enforce Participant-Aware Access Control
• A Taxonomy of Prompt Defects in LLM Systems
• Scam2Prompt: A Scalable Framework for Auditing Malicious Scam Endpoints in Production LLMs
• Reliable Weak-to-Strong Monitoring of LLM Agents
• Highlight & Summarize: RAG without the jailbreaks
• Dialogue Systems Engineering: A Survey and Future Directions
• Adaptive Content Restriction for Large Language Models via Suffix Optimization
• Provably Secure Retrieval-Augmented Generation
• Anyone Can Jailbreak: Prompt-Based Attacks on LLMs and T2Is
• The Problem with Safety Classification is not just the Models
• OneShield -- the Next Generation of LLM Guardrails
• Behind the Refusal: Determining Guardrail Activation via Behavioral Monitoring
• A Methodology for Selecting and Composing Runtime Architecture Patterns for Production LLM Agents
• Ethical Hyper-Velocity (EHV): A Hardware-Rooted Zero-Trust Runtime Enforcement Architecture for Agentic AI Systems
• Dr.Copilot: A Multi-Agent Prompt Optimized Assistant for Improving Patient-Doctor Communication in Romanian
• SEALGuard: Safeguarding the Multilingual Conversations in Southeast Asian Languages for LLM Software Systems
• UQLM: A Python Package for Uncertainty Quantification in Large Language Models
• AI-guided digital intervention with physiological monitoring reduces intrusive memories after experimental trauma
• GAF-Guard: An Agentic Framework for Risk Management and Governance in Large Language Models
• SoK: Evaluating Jailbreak Guardrails for Large Language Models
• Why LLM Safety Guardrails Collapse After Fine-tuning: A Similarity Analysis Between Alignment and Fine-tuning Datasets
• Maris: A Formally Verifiable Privacy Policy Enforcement Paradigm for Multi-Agent Collaboration Systems
• RedDebate: Safer Responses Through Multi-Agent Red Teaming Debates
• LlamaFirewall: An open source guardrail system for building secure AI agents
• CoDial: Interpretable Task-Oriented Dialogue Systems Through Dialogue Flow Alignment
• ACCESS DENIED INC: The First Benchmark Environment for Sensitivity Awareness
• Let Them Down Easy! Contextual Effects of LLM Guardrails on User Perceptions and Preferences
• Safety Through Reasoning: An Empirical Study of Reasoning Guardrail Models
• Robustness in Large Language Models: A Survey of Mitigation Strategies and Evaluation Metrics
• Relative Bias: A Comparative Framework for Quantifying Bias in LLMs
• sudoLLM: On Multi-role Alignment of Language Models
• Guarded Query Routing for Large Language Models
• The Hitchhikers Guide to Production-ready Trustworthy Foundation Model powered Software (FMware)
• AgentWall: A Runtime Safety Layer for Local AI Agents
• From Prompts to Contracts: Harness Engineering for Auditable Enterprise LLM Agents
• The Attack and Defense Landscape of Agentic AI: A Comprehensive Survey
• The Unfireable Safety Kernel: Execution-Time AI Alignment for AI Agents and Other Escapable AI Systems
• Cordon: Semantic Transactions for Tool-Using LLM Agents
• Beyond Refusal: Probing the Limits of Agentic Self-Correction for Semantic Sensitive Information
• Insuring Every Action: An Authority Frontier Framework for Runtime Actuarial Control of Autonomous AI Agents
• ADR: An Agentic Detection System for Enterprise Agentic AI Security
• AgentLeak: A Benchmark for Internal-Channel Privacy Leakage in Multi-Agent LLM Systems
• From Agent Traces to Trust: A Survey of Evidence Tracing and Execution Provenance in LLM Agents
• Owner-Harm: A Missing Threat Model for AI Agent Safety
• Harness as an Asset: Enforcing Determinism via the Convergent AI Agent Framework (CAAF)
• Unified Multi-Task Learning & Model Fusion for Efficient Language Model Guardrailing
• Adversarial Attacks on LLM-as-a-Judge Systems: Insights from Prompt Injections
• ProbeLogits: Kernel-Level LLM Inference Primitives for AI-Native Operating Systems
• Is Monitoring Enough? Strategic Agent Selection For Stealthy Attack in Multi-Agent Discussions
• Steering the CensorShip: Uncovering Representation Vectors for LLM "Thought" Control
• MrGuard: A Multilingual Reasoning Guardrail for Universal LLM Safety
• DualBreach: Efficient Dual-Jailbreaking via Target-Driven Initialization and Multi-Target Optimization
• ADL: A Declarative Language for Agent-Based Chatbots
• PROMPTEVALS: A Dataset of Assertions and Guardrails for Custom Production Large Language Model Pipelines
• The Structural Safety Generalization Problem
• X-Guard: Multilingual Guard Agent for Content Moderation

Related

• Building Guardrails for Large Language Models
• Fence: Specialized SLM Guardrails for LLM Applications
• Protect: Towards Robust Guardrailing Stack for Trustworthy Enterprise LLM Systems
• LLMGuard: Guarding Against Unsafe LLM Behavior
• Guardrails for trust, safety, and ethical development and deployment of Large Language Models (LLM)