Time Series Forecasting Arima Prophet Deep Learning Methods
# Time Series Forecasting: ARIMA, Prophet & Deep Learning Methods
## Introduction & Motivation
Time series forecasting: predict future values from historical. ARIMA: autoregressive integrated moving average; classical statistical. Prophet: decomposition; trend + seasonality. LSTM/Transformers: deep learning; sequence modeling. Applications: stock prediction, weather, demand planning, anomaly detection.
Motivation: Sequential dependence; temporal patterns. Specialized methods capture structure.
Applications: Finance, weather, inventory, healthcare.
---
## Core Concepts & Theory
### Stationarity
Constant mean/variance; ARIMA requires; differencing helps.
### Seasonality
Repeating patterns; Prophet handles explicitly.
### Autoregressive
AR: predict via past values.
---
## Mathematical Formulation
ARIMA(p,d,q):
$$\phi(B)(1-B)^d y_t = heta(B) \epsilon_t$$
where B = backshift, φ, θ = polynomials.
Prophet:
$$y_t = g(t) + s(t) + h(t) + \epsilon_t$$
trend + seasonality + holidays + error.
LSTM:
$$h_t = ext{LSTM}([y_{t-p}, \ldots, y_{t-1}])$$
encode sequence → predict next.
---
## Advanced Theory & Extensions
### Temporal Convolutional Networks (TCN)
Causal convolutions; parallel computation.
### Attention Mechanisms
Temporal attention; focus on relevant past.
### Transformer-Based
Self-attention; state-of-the-art sequence.
---
## Computational Considerations
ARIMA: O(N·p·q) parameter estimation.
Prophet: O(N) inference; interpretable.
LSTM: O(T·h²) per step; h = hidden dim.
---
## Practical Implementation Strategies
### Feature Engineering
Lag features, rolling statistics, temporal features.
### Hyperparameter Selection
ACF/PACF for ARIMA; grid search for deep.
### Train-Test Split
Respect temporal order; no information leakage.
---
## Benchmark Datasets & Evaluation
Stock Prices: Complex; benchmark datasets.
Weather: Highly seasonal; Prophet standard.
Traffic: High-frequency; deep learning dominant.
---
## Key Challenges & Limitations
### Nonstationarity
Requires differencing; seasonality complex.
### Distribution Shift
Data evolves; retrain essential.
### Interpretability
Deep methods black-box; Prophet interpretable.
---
## Hyperparameter Tuning
ARIMA p,d,q: Grid search; AIC/BIC criteria.
Prophet seasonality: Weekly, yearly; domain-specific.
LSTM layers: 1-3; hidden dim 32-256.
---
## Real-World Applications & Case Studies
Finance: Stock forecasting; risk management.
Retail: Demand planning; inventory optimization.
Energy: Load forecasting; grid management.
---
## Integration with Other Methods
Forecasting + Confidence Intervals → uncertainty.
Forecasting + Anomaly Detection → alert systems.
---
## Summary & Key Takeaways
Time series forecasting via ARIMA, Prophet, and deep learning methods leverage temporal structure for accurate future predictions through autoregression, decomposition, and sequence models.
Principles:
1. Stationarity: ARIMA prerequisite.
2. Trend + Seasonality: decomposition.
3. Autoregressive: past predicts future.
4. LSTM/Attention: sequence deep learning.
5. Evaluation: respect temporal order.
---
---
## Appendix: Practical Labs
### Lab 1: Stationarity Test (ADF)
import numpy as np
def check_stationarity(series, order=1):
"""Augmented Dickey-Fuller approximation"""
# Differencing
if order > 0:
diff = np.diff(series, n=order)
else:
diff = series
# Mean of differenced series
mean_diff = np.mean(diff)
# Stationarity heuristic: mean close to 0
is_stationary = abs(mean_diff) < 0.1
return is_stationary, diff
# Test
np.random.seed(42)
series = np.cumsum(np.random.randn(100)) # Random walk (non-stationary)
stationary, diff = check_stationarity(series, order=1)
assert not stationary or np.isfinite(diff).all(), "Differencing computed"
print("✓ Stationarity check working")
if __name__ == "__main__":
print("Lab 1: Stationarity - PASSED")### Lab 2: Lag Features
import numpy as np
def create_lag_features(series, n_lags=3):
"""Create lag features for time series"""
features = []
targets = []
for i in range(n_lags, len(series)):
lags = series[i-n_lags:i]
features.append(lags)
targets.append(series[i])
return np.array(features), np.array(targets)
# Test
np.random.seed(42)
series = np.sin(np.arange(100) / 10)
X, y = create_lag_features(series, n_lags=5)
assert X.shape == (95, 5), "Lag features shape"
assert y.shape == (95,), "Targets shape"
print("✓ Lag features working")
if __name__ == "__main__":
print("Lab 2: Lags - PASSED")### Lab 3: Time Series Decomposition
import numpy as np
def decompose_series(series, period=12):
"""Simple time series decomposition (trend + seasonal)"""
# Trend: moving average
trend = np.convolve(series, np.ones(period) / period, mode='same')
# Detrended
detrended = series - trend
# Seasonal: average per period
seasonal = np.zeros_like(series)
for i in range(period):
seasonal[i::period] = np.mean(detrended[i::period])
# Residual
residual = series - trend - seasonal
return trend, seasonal, residual
# Test
np.random.seed(42)
series = np.sin(np.arange(100) / 5) + np.arange(100) / 50
trend, seasonal, residual = decompose_series(series)
assert trend.shape == series.shape, "Trend shape"
assert seasonal.shape == series.shape, "Seasonal shape"
print("✓ Decomposition working")
if __name__ == "__main__":
print("Lab 3: Decomposition - PASSED")### Lab 4: Forecasting Metrics
import numpy as np
def compute_forecast_metrics(y_true, y_pred):
"""Compute RMSE, MAE, MAPE"""
rmse = np.sqrt(np.mean((y_true - y_pred) ** 2))
mae = np.mean(np.abs(y_true - y_pred))
mape = np.mean(np.abs((y_true - y_pred) / y_true)) * 100
return rmse, mae, mape
# Test
np.random.seed(42)
y_true = np.sin(np.arange(50) / 5)
y_pred = y_true + 0.1 * np.random.randn(50)
rmse, mae, mape = compute_forecast_metrics(y_true, y_pred)
assert rmse > 0, "RMSE positive"
assert mae > 0, "MAE positive"
assert mape > 0, "MAPE positive"
print("✓ Forecast metrics working")
if __name__ == "__main__":
print("Lab 4: Metrics - PASSED")