test time compute

**Test-Time Compute Scaling** is the **paradigm of allocating more computational resources at inference time to improve output quality** — contrasting with training-time scaling (more data/parameters) by spending more FLOPS per query to achieve better answers. **The Core Insight** - Training scaling: 10x more compute → 10x better model (Chinchilla law). - Inference scaling: Generate N answers → select best → improves accuracy without retraining. - Key finding (Snell et al., 2024): "Beyond the chinchilla optimum, test-time compute is more efficient than training compute for difficult tasks." **Test-Time Compute Methods** **Best-of-N Sampling**: - Generate N independent responses → select best by reward model score. - Simple but effective. O(N) compute. Linear in N, but diminishing returns. **Sequential Refinement**: - Generate → self-critique → revise → repeat K times. - Each iteration improves quality, especially for complex tasks. **Monte Carlo Tree Search (MCTS)**: - Expand reasoning tree, evaluate leaf nodes with process reward model. - Backpropagate scores → select best reasoning path. - AlphaGo approach applied to language reasoning. **OpenAI o1 and "Chain of Thought"**: - o1 generates an internal "thinking chain" before answering — extended CoT. - More thinking tokens → better accuracy (log-linear relationship). - o1: 83.3% on AIME 2024 (vs. GPT-4o: 9.3%). - o3: >90% on ARC-AGI challenge with heavy test-time compute. **Scaling Laws for Inference** - Accuracy vs. compute: ~log-linear on difficult reasoning benchmarks. - Crossover point: For hard tasks, spending 10x inference compute beats training a 10x larger model. - Cost implication: Test-time compute shifts cost from upfront (training) to per-query. **Efficient Test-Time Compute** - **Adaptive compute**: Allocate more compute for harder questions, less for easy. - **Speculative thinking**: Draft short CoT; extend only if initial answer uncertain. Test-time compute scaling is **the new frontier of AI capability improvement** — the o1/o3 results show that reasoning quality can be traded against compute budget, opening a new axis of scaling beyond model size and training data.