output filter
**Output Filtering and Moderation**
**Why Filter Outputs?**
Prevent harmful, inappropriate, or incorrect content from reaching users.
**Filtering Strategies**
**Rule-Based Filtering**
```python
class RuleBasedFilter:
def __init__(self):
self.blocklist = load_blocklist("harmful_words.txt")
self.pii_patterns = [
r"\b\d{3}-\d{2}-\d{4}\b", # SSN
r"\b[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,}\b", # Email
r"\b\d{16}\b", # Credit card
]
def filter(self, text):
# Check blocklist
for word in self.blocklist:
if word.lower() in text.lower():
return self.redact(text, word)
# Redact PII
for pattern in self.pii_patterns:
text = re.sub(pattern, "[REDACTED]", text, flags=re.IGNORECASE)
return text
```
**LLM-Based Moderation**
```python
def moderate_output(response):
result = moderator_llm.generate(f"""
Analyze this AI response for policy violations:
Response:
{response}
Check for:
1. Harmful content (violence, illegal activities)
2. Personal information disclosure
3. Misinformation or false claims
4. Bias or discrimination
5. Inappropriate professional advice
Is this response safe to show? (yes/no)
If no, explain the issue:
""")
is_safe = result.strip().lower().startswith("yes")
return is_safe, result
```
**Classifier-Based**
```python
from transformers import pipeline
toxicity_classifier = pipeline("text-classification",
model="unitary/toxic-bert")
def classify_toxicity(text):
result = toxicity_classifier(text)
return result[0]["label"], result[0]["score"]
```
**OpenAI Moderation API**
```python
from openai import OpenAI
def check_output(text):
client = OpenAI()
response = client.moderations.create(input=text)
result = response.results[0]
if result.flagged:
return {
"safe": False,
"categories": {k: v for k, v in result.categories.dict().items() if v}
}
return {"safe": True}
```
**Multi-Stage Pipeline**
```
LLM Output
|
v
[PII Filter] -> Redact personal data
|
v
[Toxicity Classifier] -> Block harmful content
|
v
[Fact Checker] -> Flag uncertain claims
|
v
[Final Review] -> LLM moderation
|
v
User
```
**Handling Blocked Content**
```python
def safe_response(original, filter_result):
if filter_result["safe"]:
return original
# Option 1: Return generic message
return "I am unable to provide that response."
# Option 2: Request regeneration
# return regenerate_with_guidance(original, filter_result)
# Option 3: Return redacted version
# return filter_result["redacted_text"]
```
**Best Practices**
- Layer multiple filtering methods
- Log filtered content for review
- Balance safety with helpfulness
- Regular updates to filter rules
- A/B test filter thresholds