AI Safety and LLM Guardrails encompasses the techniques, systems, and practices for ensuring large language models behave safely, reliably, and within intended boundaries — including alignment training (RLHF/Constitutional AI), input/output guardrails, red teaming for vulnerability discovery, jailbreak defense, content filtering, and runtime monitoring to prevent harmful, biased, or unauthorized model behavior in production deployments.

The Safety Stack

<svg viewBox="0 0 578 321" xmlns="http://www.w3.org/2000/svg" style="max-width:100%;height:auto" role="img"><rect x="0" y="0" width="578" height="321" rx="12" fill="#0d1117"/><g font-family="ui-monospace,SFMono-Regular,Menlo,Consolas,&quot;Liberation Mono&quot;,monospace" font-size="14"><text xml:space="preserve" x="20" y="31.7"><tspan fill="#c9d1d9">Training-time safety:</tspan></text><text xml:space="preserve" x="20" y="50.7"><tspan fill="#c9d1d9">  </tspan><tspan fill="#6e7681">└──</tspan><tspan fill="#c9d1d9"> Alignment: RLHF, DPO, Constitutional AI</tspan></text><text xml:space="preserve" x="20" y="69.7"><tspan fill="#c9d1d9">  </tspan><tspan fill="#6e7681">└──</tspan><tspan fill="#c9d1d9"> Safety fine-tuning: train on harmful prompt refusals</tspan></text><text xml:space="preserve" x="20" y="88.7"><tspan fill="#c9d1d9">  </tspan><tspan fill="#6e7681">└──</tspan><tspan fill="#c9d1d9"> Data filtering: remove toxic/dangerous training data</tspan></text><text xml:space="preserve" x="20" y="107.7"></text><text xml:space="preserve" x="20" y="126.7"><tspan fill="#c9d1d9">Inference-time safety:</tspan></text><text xml:space="preserve" x="20" y="145.7"><tspan fill="#c9d1d9">  </tspan><tspan fill="#6e7681">└──</tspan><tspan fill="#c9d1d9"> Input guardrails: classify/filter user prompts</tspan></text><text xml:space="preserve" x="20" y="164.7"><tspan fill="#c9d1d9">  </tspan><tspan fill="#6e7681">└──</tspan><tspan fill="#c9d1d9"> Output guardrails: classify/filter model responses</tspan></text><text xml:space="preserve" x="20" y="183.7"><tspan fill="#c9d1d9">  </tspan><tspan fill="#6e7681">└──</tspan><tspan fill="#c9d1d9"> System prompts: behavioral constraints and role definition</tspan></text><text xml:space="preserve" x="20" y="202.7"><tspan fill="#c9d1d9">  </tspan><tspan fill="#6e7681">└──</tspan><tspan fill="#c9d1d9"> Tool use restrictions: limit what the model can do</tspan></text><text xml:space="preserve" x="20" y="221.7"></text><text xml:space="preserve" x="20" y="240.7"><tspan fill="#c9d1d9">Monitoring:</tspan></text><text xml:space="preserve" x="20" y="259.7"><tspan fill="#c9d1d9">  </tspan><tspan fill="#6e7681">└──</tspan><tspan fill="#c9d1d9"> Red teaming: adversarial testing before deployment</tspan></text><text xml:space="preserve" x="20" y="278.7"><tspan fill="#c9d1d9">  </tspan><tspan fill="#6e7681">└──</tspan><tspan fill="#c9d1d9"> Runtime monitoring: detect and log safety violations</tspan></text><text xml:space="preserve" x="20" y="297.7"><tspan fill="#c9d1d9">  </tspan><tspan fill="#6e7681">└──</tspan><tspan fill="#c9d1d9"> Feedback loops: user reports </tspan><tspan fill="#6e7681">→</tspan><tspan fill="#c9d1d9"> model improvement</tspan></text></g></svg>

Jailbreak Attack Categories

Guardrail Implementations

# NeMo Guardrails / Guardrails AI pattern

# Input rail: check user message before sending to LLM
def input_rail(user_message):
    # 1. Topic classifier: is this an allowed topic?
    if topic_classifier(user_message) == "restricted":
        return BLOCKED_RESPONSE
    # 2. Jailbreak detector
    if jailbreak_classifier(user_message) > 0.9:
        return BLOCKED_RESPONSE
    # 3. PII detector
    user_message = redact_pii(user_message)
    return PASS

# Output rail: check LLM response before returning to user
def output_rail(llm_response):
    # 1. Toxicity classifier
    if toxicity_score(llm_response) > threshold:
        return REGENERATE or BLOCKED_RESPONSE
    # 2. Factuality check (for RAG)
    if not grounded_in_context(llm_response, retrieved_docs):
        return flag_hallucination(llm_response)
    # 3. PII/code execution scanner
    return sanitize(llm_response)

Constitutional AI (Anthropic)

1. Red-team the model → collect harmful outputs
2. Ask the model to critique its own harmful output
   using constitutional principles ('Is this harmful?')
3. Ask the model to revise its output based on the critique
4. Train on (prompt, revised_response) pairs → RLAIF

Result: Self-improving safety without human annotators for each case

Red Teaming at Scale

• Manual red teaming: Domain experts craft adversarial prompts across risk categories (violence, deception, bias, privacy, illegal activity)
• Automated red teaming: Use an adversarial LLM to generate attack prompts, evaluate with a safety classifier, iterate ('red-LLM vs. blue-LLM')
• Structured testing: NIST AI Risk Management Framework, OWASP LLM Top 10, EU AI Act compliance testing

AI safety is not a single feature but a defense-in-depth discipline — requiring coordinated layers of training-time alignment, inference-time guardrails, adversarial testing, and ongoing monitoring to create systems that are simultaneously capable, safe, and robust against the full spectrum of misuse attempts.