limitations
**LLM Limitations and Boundaries**
**Fundamental Limitations**
**Knowledge Cutoff**
LLMs have training data cutoff dates:
| Model | Knowledge Cutoff |
|-------|------------------|
| GPT-4o | Varies by version |
| Claude 3 | Early 2024 |
| Llama 3 | December 2023 |
**Implication**: Cannot answer about recent events without retrieval.
**Context Window Constraints**
- Maximum tokens per request (e.g., 128K, 200K)
- "Lost in the middle" problem for very long contexts
- Cost scales with context length
**Hallucinations**
LLMs may generate:
- Plausible-sounding but false information
- Non-existent citations or references
- Confident answers about things they do not know
**What LLMs Cannot Do Well**
**Reliable Computation**
| Task | Problem | Workaround |
|------|---------|------------|
| Complex math | May make arithmetic errors | Use code execution |
| Counting | Inconsistent for large sets | Use programmatic counting |
| Logical proofs | May skip steps or err | Verify with formal tools |
**Real-time Information**
- No access to current events
- Cannot check live stock prices, weather
- Solution: Tool use, RAG with current data
**Precision Tasks**
| Task | Issue | Better Approach |
|------|-------|-----------------|
| Exact text matching | May paraphrase | Use regex/code |
| Character counting | Tokenization obscures | Use len() |
| Consistent formatting | May drift | Use structured output |
**Guaranteed Safety**
- Jailbreaks and prompt injection possible
- Cannot guarantee 100% filter compliance
- Requires defense in depth approach
**Things to Be Careful About**
**High-Stakes Decisions**
❌ LLMs should not be sole deciders for:
- Medical diagnoses
- Legal advice
- Financial decisions
- Safety-critical systems
✅ Use as assistants with human oversight
**Private Information**
- LLMs may memorize training data
- API calls may be logged by providers
- Consider privacy implications
**Consistency**
- Same prompt may give different outputs
- Temperature=0 helps but not guaranteed
- For critical consistency, verify programmatically
**Mitigation Strategies**
**For Hallucinations**
1. Use RAG with verified sources
2. Request citations and verify them
3. Cross-check with multiple queries
4. Add fact-checking step
**For Math/Logic**
1. Use code execution tools
2. Chain-of-thought prompting
3. Self-consistency (multiple samples)
4. Formal verification where possible
**For Safety**
1. Layer multiple guardrails
2. Content filtering on input/output
3. Human review for sensitive content
4. Rate limiting and monitoring