Self-Consistency Decoding

What is Self-Consistency? Self-consistency generates multiple reasoning paths for the same problem, then selects the most common final answer through majority voting.

How It Works

Standard Chain-of-Thought

Problem ---> [Single reasoning path] ---> Answer

Single point of failure: if reasoning is wrong, answer is wrong.

Self-Consistency

Problem ---> [Path 1] ---> Answer A
        ---> [Path 2] ---> Answer B
        ---> [Path 3] ---> Answer A
        ---> [Path 4] ---> Answer A
        ---> [Path 5] ---> Answer C

Majority vote: Answer A (3/5)

Implementation

from collections import Counter

def self_consistent_answer(prompt: str, n_samples: int = 5) -> str:
    answers = []

    for _ in range(n_samples):
        # Sample with temperature > 0 for diversity
        response = llm.generate(
            prompt + "Let us think step by step.",
            temperature=0.7
        )
        answer = extract_final_answer(response)
        answers.append(answer)

    # Majority vote
    counts = Counter(answers)
    return counts.most_common(1)[0][0]

Temperature for Diversity

When Self-Consistency Helps

Good Use Cases

Less Effective

Confidence from Agreement Agreement level indicates confidence:

def get_answer_with_confidence(answers):
    counts = Counter(answers)
    top_answer, top_count = counts.most_common(1)[0]
    confidence = top_count / len(answers)
    return top_answer, confidence

Cost Considerations

Diminishing returns beyond 5-10 samples.

Self-consistency is especially valuable for high-stakes reasoning where accuracy matters more than cost.