self consistency
**Self-Consistency Decoding**
**What is Self-Consistency?**
Self-consistency generates multiple reasoning paths for the same problem, then selects the most common final answer through majority voting.
**How It Works**
**Standard Chain-of-Thought**
```
Problem ---> [Single reasoning path] ---> Answer
```
Single point of failure: if reasoning is wrong, answer is wrong.
**Self-Consistency**
```
Problem ---> [Path 1] ---> Answer A
---> [Path 2] ---> Answer B
---> [Path 3] ---> Answer A
---> [Path 4] ---> Answer A
---> [Path 5] ---> Answer C
Majority vote: Answer A (3/5)
```
**Implementation**
```python
from collections import Counter
def self_consistent_answer(prompt: str, n_samples: int = 5) -> str:
answers = []
for _ in range(n_samples):
# Sample with temperature > 0 for diversity
response = llm.generate(
prompt + "Let us think step by step.",
temperature=0.7
)
answer = extract_final_answer(response)
answers.append(answer)
# Majority vote
counts = Counter(answers)
return counts.most_common(1)[0][0]
```
**Temperature for Diversity**
| Temperature | Effect |
|-------------|--------|
| 0.0 | No diversity, same answer every time |
| 0.5-0.7 | Moderate diversity, good for self-consistency |
| 1.0+ | High diversity, may include wrong paths |
**When Self-Consistency Helps**
**Good Use Cases**
| Task | Why It Helps |
|------|--------------|
| Math problems | Multiple valid solution paths |
| Logic puzzles | Different reasoning approaches |
| Code generation | Try multiple implementations |
**Less Effective**
| Task | Why |
|------|-----|
| Factual recall | Only one correct answer, no reasoning paths |
| Open-ended generation | No "correct" answer to vote on |
**Confidence from Agreement**
Agreement level indicates confidence:
```python
def get_answer_with_confidence(answers):
counts = Counter(answers)
top_answer, top_count = counts.most_common(1)[0]
confidence = top_count / len(answers)
return top_answer, confidence
```
**Cost Considerations**
| Samples | Accuracy Gain | Cost |
|---------|---------------|------|
| 1 (baseline) | 0% | 1x |
| 3 | ~5-10% | 3x |
| 5 | ~10-15% | 5x |
| 10 | ~15-20% | 10x |
Diminishing returns beyond 5-10 samples.
Self-consistency is especially valuable for high-stakes reasoning where accuracy matters more than cost.