soft actor-critic

**SAC** (Soft Actor-Critic) is a **state-of-the-art off-policy reinforcement learning algorithm for continuous action spaces** — based on maximum entropy RL, SAC simultaneously maximizes expected reward and policy entropy, achieving sample-efficient, stable learning with automatic temperature tuning. **SAC Components** - **Actor**: Policy network $pi_ heta(a|s)$ outputs a Gaussian distribution over continuous actions. - **Twin Critics**: Two Q-networks $Q_{phi_1}, Q_{phi_2}$ — use the minimum to reduce overestimation bias. - **Entropy Term**: Loss includes $-alpha H(pi)$ — temperature $alpha$ is automatically tuned. - **Off-Policy**: Stores transitions in a replay buffer — high sample efficiency. **Why It Matters** - **Sample Efficient**: Off-policy + replay buffer makes SAC one of the most sample-efficient model-free RL algorithms. - **Stable**: Entropy regularization + twin critics prevent training instability common in actor-critic methods. - **Continuous Control**: State-of-art for robotics, process control, and continuous optimization tasks. **SAC** is **the stable explorer** — combining maximum entropy RL with twin critics for robust, sample-efficient continuous control.