Time Series Analysis Arima Forecasting
# Time Series Analysis: ARIMA & Forecasting
## Introduction & Motivation
ARIMA (AutoRegressive Integrated Moving Average) models sequential dependence via AR (past values), I (differencing), MA (past errors). Stationary assumption enables linear methods. Univariate; foundation before deep models.
Motivation: Time series not independent; past predicts future. Linear ARIMA simple, interpretable, effective for short-term forecasting. Baseline for time series problems.
Applications: Stock prices, weather, sales forecasting, anomaly detection, demand planning.
---
## Core Concepts & Theory
### AR Component
Regression on past values: y_t = c + φ₁y_{t-1} + ... + φ_py_{t-p} + ε_t.
### I Component
Differencing: Δy_t = y_t - y_{t-1}. Achieves stationarity.
### MA Component
Regression on past residuals: y_t = μ + ε_t + θ₁ε_{t-1} + ... + θ_qε_{t-q}.
---
## Mathematical Formulation
ARIMA(p,d,q):
$$\Phi(B)(1-B)^d y_t = Θ(B)ε_t$$
where B is backshift operator, Φ is AR polynomial, Θ is MA polynomial.
---
## Advanced Theory & Extensions
### Seasonal ARIMA (SARIMA)
Incorporates seasonal patterns: SARIMA(p,d,q)(P,D,Q,s).
### ARIMAX
Exogenous variables; multivariate extension.
---
## Computational Considerations
Parameter Estimation: O(n) via Kalman filter; iterative MLE.
Forecasting: O(d) per step ahead.
---
## Practical Implementation Strategies
### Stationarity Testing
ADF test checks null hypothesis of unit root. Reject → stationary.
### Order Selection
ACF/PACF plots guide p,d,q; PMDARIMA auto-selects.
### Seasonal Order
Seasonal ACF/PACF for P,D,Q.
---
## Benchmark Datasets & Evaluation
M3 Competition: 3000 time series.
Airline Passengers: Classic univariate.
---
## Key Challenges & Limitations
### Trend/Seasonality
Must difference to achieve stationarity; loses information.
### Forecasting Horizon
Uncertainty increases; not suitable for distant predictions.
---
## Hyperparameter Tuning
(p,d,q) via grid search + AIC; seasonal (P,D,Q) similarly.
---
## Real-World Applications & Case Studies
Retail: Sales forecasting for inventory.
Finance: ARIMA as benchmark vs. deep models.
---
## Integration with Other Methods
ARIMA + Exogenous Variables → ARIMAX.
ARIMA + Deep Learning → Hybrid models.
---
## Summary & Key Takeaways
ARIMA models time series via autoregressive, differencing, and moving average components, enabling stationary, interpretable forecasting.
Principles:
1. AR captures temporal dependence.
2. Differencing achieves stationarity.
3. MA captures error correlation.
4. Stationarity necessary for ARIMA.
5. Parameter order critical for performance.
---
---
## Appendix: Practical Labs
### Lab 1: Stationarity Testing
from statsmodels.tsa.stattools import adfuller
import numpy as np
np.random.seed(42)
non_stationary = np.cumsum(np.random.randn(100)) # Random walk
stationary = np.random.randn(100) # White noise
adf_ns = adfuller(non_stationary)
adf_s = adfuller(stationary)
print(f"Non-stationary p-value: {adf_ns[1]:.4f}")
print(f"Stationary p-value: {adf_s[1]:.4f}")
assert adf_ns[1] > 0.05, "Random walk should be non-stationary"
assert adf_s[1] < 0.05, "White noise should be stationary"
print("✓ Stationarity testing working")
if __name__ == "__main__":
print("Lab 1: Stationarity - PASSED")### Lab 2: ACF/PACF
from statsmodels.graphics.tsaplots import plot_acf, plot_pacf
import numpy as np
import matplotlib.pyplot as plt
np.random.seed(42)
data = np.cumsum(np.random.randn(100))
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(12, 3))
plot_acf(data, lags=20, ax=ax1)
plot_pacf(data, lags=20, ax=ax2)
plt.close()
print("✓ ACF/PACF plots working")
if __name__ == "__main__":
print("Lab 2: ACF/PACF - PASSED")### Lab 3: ARIMA Fitting
from statsmodels.tsa.arima.model import ARIMA
import numpy as np
np.random.seed(42)
data = np.cumsum(np.random.randn(100))
model = ARIMA(data, order=(1, 1, 1))
result = model.fit()
print(f"ARIMA fitted. AIC: {result.aic:.2f}")
assert result.aic > 0, "AIC should be positive"
print("✓ ARIMA fitting working")
if __name__ == "__main__":
print("Lab 3: ARIMA Fitting - PASSED")### Lab 4: Forecasting
from statsmodels.tsa.arima.model import ARIMA
import numpy as np
np.random.seed(42)
data = np.cumsum(np.random.randn(100))
model = ARIMA(data, order=(1, 1, 1))
result = model.fit()
forecast = result.get_forecast(steps=10)
forecast_values = forecast.predicted_mean
print(f"Forecast shape: {forecast_values.shape}")
assert len(forecast_values) == 10, "Should forecast 10 steps"
print("✓ Forecasting working")
if __name__ == "__main__":
print("Lab 4: Forecasting - PASSED")