time series forecasting
**Time Series Forecasting with Deep Learning** is the **application of neural network architectures to predict future values of temporal sequences** — leveraging patterns in historical data including trends, seasonality, and complex nonlinear dependencies, where modern transformer and SSM-based forecasters now compete with and often surpass traditional statistical methods (ARIMA, ETS) on diverse benchmarks from energy demand to financial markets to weather prediction.
**Deep Learning Architecture Timeline for Time Series**
| Era | Architecture | Key Advantage |
|-----|------------|---------------|
| 2015-2017 | LSTM/GRU | Captures sequential dependencies |
| 2017-2019 | WaveNet/TCN (Temporal CNN) | Parallelizable, dilated convolutions |
| 2019-2021 | Informer/Autoformer (Transformer) | Long-range attention, multi-horizon |
| 2022+ | PatchTST, TimesNet | Channel-independent patching |
| 2023+ | TimesFM, Chronos (Foundation) | Pre-trained on many datasets |
| 2024+ | Mamba/SSM variants | Linear complexity, long sequences |
**Forecasting Paradigms**
| Paradigm | Method | Best For |
|----------|--------|----------|
| Point forecast | Predict single future value at each step | Simple predictions |
| Probabilistic forecast | Predict distribution (quantiles, parameters) | Risk-aware decisions |
| Multi-horizon | Predict multiple future steps simultaneously | Planning applications |
| Multivariate | Predict multiple correlated series jointly | Interconnected systems |
**PatchTST (2023)**
- Key insight: Treat time series as sequence of **patches** (subsequences), not individual points.
- Patch size P=16: Reduces sequence length by 16x → attention cost reduced 256x!
- Channel-independent: Each variable processed independently → better scaling.
- Result: SOTA on long-term forecasting benchmarks, beating complex Transformer designs.
**Foundation Models for Time Series**
| Model | Developer | Approach |
|-------|----------|----------|
| TimesFM | Google | Pre-trained decoder-only on 100B+ timepoints |
| Chronos | Amazon | T5-style tokenization of time series values |
| Lag-Llama | Salesforce | LLaMA-based probabilistic forecaster |
| MOIRAI | Salesforce | Universal forecaster, any-variate |
**Input Representation**
- **Raw values**: Direct numerical input → often normalized per-series.
- **Patching**: Group consecutive values into patches → reduce length, capture local patterns.
- **Tokenization (Chronos)**: Bin continuous values into discrete tokens → use language model.
- **Frequency features**: Add day-of-week, month, hour as covariates.
- **Lag features**: Include values at known seasonal lags (e.g., same hour yesterday).
**Evaluation Metrics**
| Metric | Formula | What It Measures |
|--------|---------|------------------|
| MAE | Mean Absolute Error | Average absolute deviation |
| MSE/RMSE | (Root) Mean Squared Error | Penalizes large errors |
| MAPE | Mean Absolute Percentage Error | Scale-independent accuracy |
| CRPS | Continuous Ranked Probability Score | Probabilistic forecast quality |
| WQL | Weighted Quantile Loss | Quantile prediction accuracy |
Time series forecasting with deep learning is **entering a foundation model era** — pre-trained temporal models that generalize across domains are beginning to match or exceed specialized models, promising to make high-quality forecasting accessible without domain expertise, much as language models democratized NLP.