backpropagation through time

**Backpropagation Through Time (BPTT)** is the **standard algorithm for computing gradients in recurrent neural networks** — unrolling the recurrent computation through time steps and applying the chain rule to propagate error gradients backward through the entire sequence. **How BPTT Works** - **Unrolling**: Unfold the RNN recurrence into a feedforward computation graph over $T$ time steps. - **Forward Pass**: Compute all hidden states $h_1, h_2, ldots, h_T$ and the loss $L$. - **Backward Pass**: Apply the chain rule backward through all time steps to compute $partial L / partial heta$. - **Weight Sharing**: Gradients from all time steps are accumulated for the shared weight parameters. **Why It Matters** - **Standard Method**: BPTT is how all RNNs, LSTMs, and GRUs are trained. - **Vanishing Gradients**: Gradients can vanish or explode over long sequences — motivating LSTM and gradient clipping. - **Truncated BPTT**: Practical variant that limits backpropagation to a fixed window for memory and stability. **BPTT** is **the chain rule unrolled through time** — the fundamental algorithm for training sequence models by propagating gradients through temporal computation.