troubleshooting
**Troubleshooting AI/ML Applications**
**Common LLM Issues and Solutions**
**API Errors**
| Error | Cause | Solution |
|-------|-------|----------|
| 401 Unauthorized | Invalid API key | Check API key, regenerate if needed |
| 429 Rate Limited | Too many requests | Add retry with backoff, reduce concurrency |
| 500 Internal Error | Provider issue | Retry, check status page |
| Timeout | Long response time | Increase timeout, reduce prompt size |
| Context length exceeded | Prompt too long | Summarize, truncate, use RAG |
**Response Quality Issues**
| Issue | Possible Causes | Solutions |
|-------|-----------------|-----------|
| Hallucinations | No grounding data | Add RAG, fact-checking, citations |
| Wrong format | Unclear instructions | Provide examples, use structured output |
| Too verbose | No length constraint | Add "Be concise" or max_tokens |
| Off-topic | Weak system prompt | Strengthen constraints, add examples |
| Inconsistent | High temperature | Lower temperature, use seed |
**GPU/Memory Problems**
**CUDA Out of Memory**
```python
**Diagnosis**
import torch
print(f"GPU Memory: {torch.cuda.memory_allocated()/1e9:.1f}GB / "
f"{torch.cuda.get_device_properties(0).total_memory/1e9:.1f}GB")
**Solutions**
**1. Reduce batch size**
**2. Enable gradient checkpointing**
model.gradient_checkpointing_enable()
**3. Use mixed precision**
from torch.cuda.amp import autocast
with autocast():
output = model(input)
**4. Clear cache**
torch.cuda.empty_cache()
**5. Use quantization**
model = model.quantize(bits=4)
```
**Slow Performance**
| Bottleneck | Diagnosis | Solution |
|------------|-----------|----------|
| Data loading | CPU at 100%, GPU idle | More workers, prefetch |
| GPU compute | Low GPU utilization | Increase batch size |
| Memory bandwidth | High memory usage | Quantize, reduce model size |
| Network | High latency to API | Cache, batch requests |
**Debugging Checklist**
**Before Debugging**
1. ✅ Read the full error message
2. ✅ Check logs and stack traces
3. ✅ Verify versions (Python, packages, CUDA)
4. ✅ Test with minimal example
**Systematic Approach**
```
1. Reproduce consistently
2. Isolate the component
3. Create minimal test case
4. Check inputs/outputs at each step
5. Compare working vs. broken state
6. Make one change at a time
```
**Common Quick Fixes**
- Restart Python kernel / container
- Clear all caches (pip, torch, HF)
- Update packages to latest versions
- Try different model / provider
- Reduce complexity temporarily
**Getting Help**
- Include: versions, error messages, minimal code
- Search: GitHub issues, Stack Overflow, Discord
- Ask: r/LocalLLaMA, Hugging Face forums, provider Discord