constant folding

**Constant folding** is the **compile-time optimization that precomputes expressions involving only constants** - it eliminates redundant runtime work by replacing static subgraphs with literal values. **What Is Constant folding?** - **Definition**: Evaluate constant-only operations during compilation and substitute final constants in graph. - **Typical Cases**: Arithmetic on fixed scalars, static shape calculations, and compile-known lookup expressions. - **Runtime Impact**: Removes kernel invocations and memory operations for deterministic constant branches. - **Constraint**: Applies only where input values are compile-time known and side-effect free. **Why Constant folding Matters** - **Lower Runtime Cost**: Avoids repeatedly computing values that never change between executions. - **Graph Simplification**: Reduces node count and unlocks additional downstream optimization passes. - **Startup Efficiency**: Cuts initialization overhead in inference and training graph execution. - **Compiler Synergy**: Improves effectiveness of dead code elimination and operator fusion. - **Predictability**: Fewer runtime operations reduce variance in step timing. **How It Is Used in Practice** - **Pass Enablement**: Ensure compiler optimization pipeline includes constant-folding stage. - **Static Annotation**: Mark known-constant parameters to maximize foldable subgraphs. - **Result Verification**: Inspect optimized IR to confirm expected expressions were folded correctly. Constant folding is **a basic but effective graph optimization primitive** - precomputing static expressions reduces runtime work and creates cleaner execution graphs.