Time Series Forecasting for Industrial Processes
# Time Series Forecasting for Industrial Processes
## Introduction & Motivation
Industrial processes generate continuous time series data (temperature, pressure, flow rate, concentration). Accurate forecasting enables predictive maintenance, process optimization, and anomaly detection. ML models capture temporal patterns for robust predictions essential to manufacturing and chemical processes.
Motivation: Forecast time series for process control and maintenance.
Applications: Equipment failure prediction, process optimization, demand forecasting, anomaly detection.
---
## Core Concepts & Theory
### Stationarity
Time-independent statistical properties.
### Autocorrelation
Temporal dependencies in series.
### Seasonality
Repeating patterns and cycles.
### Trend Decomposition
Long-term and short-term components.
---
## Mathematical Formulation
ARIMA Model:
$$\phi(B)(1-B)^d X_t = heta(B)\epsilon_t$$
Exponential Smoothing:
$$\hat{x}_{t+1} = \alpha x_t + (1-\alpha)\hat{x}_t$$
LSTM Prediction:
$$\mathbf{h}_t = ext{LSTM}(\mathbf{x}_t, \mathbf{h}_{t-1})$$
---
## Advanced Theory & Extensions
### Deep Learning Models
LSTM and attention-based forecasting.
### Multivariate Forecasting
Multiple interdependent time series.
### Uncertainty Quantification
Confidence intervals on predictions.
---
## Computational Considerations
ARIMA: O(n·p·q) for n samples, p lags, q errors.
LSTM Training: O(T·H²) for T time steps, H hidden units.
Prediction: O(H) per forecast step.
---
## Practical Implementation Strategies
### Data Preprocessing
Scaling, detrending, and missing value handling.
### Feature Engineering
Lagged features and rolling statistics.
### Model Selection
Cross-validation and information criteria.
---
## Benchmark Datasets & Evaluation
UCR Time Series Archive: Benchmark datasets.
Kaggle Forecasting: Competition datasets.
Industry Data: Proprietary process measurements.
---
## Key Challenges & Limitations
### Non-Stationarity
Changing statistical properties.
### Structural Breaks
Sudden regime changes.
### Long Horizons
Accuracy decay with forecast distance.
---
## Hyperparameter Tuning
ARIMA (p,d,q): (0-5, 0-2, 0-5).
LSTM hidden units: 50-200.
Forecast horizon: 1-100 steps ahead.
---
## Real-World Applications & Case Studies
Equipment Maintenance: Failure prediction.
Energy Demand: Load forecasting.
Manufacturing: Process monitoring.
---
## Integration with Other Methods
Time series + anomaly detection; + causal inference; + machine learning.
---
## Summary & Key Takeaways
ML forecasting of time series enables proactive process management.
Principles:
1. Exploration: Analyze temporal patterns.
2. Preprocessing: Handle stationarity and trends.
3. Modeling: Choose appropriate architecture.
4. Validation: Rigorous out-of-sample testing.
5. Deployment: Real-time prediction systems.
---
## Appendix: Practical Labs
### Lab 1: Time Series Components
import numpy as np
def decompose_time_series(time_series, period=12):
"""Decompose into trend, seasonal, residual"""
n = len(time_series)
# Trend: moving average
trend = np.convolve(time_series, np.ones(period)/period, mode='same')
# Detrended
detrended = time_series - trend
# Seasonal: average by season
seasonal = np.zeros(n)
for i in range(period):
season_indices = np.arange(i, n, period)
seasonal[season_indices] = np.mean(detrended[season_indices])
# Residual
residual = time_series - trend - seasonal
return trend, seasonal, residual
# Test
t = np.arange(100)
time_series = 50 + 0.1*t + 10*np.sin(2*np.pi*t/12) + np.random.randn(100)*2
trend, seasonal, residual = decompose_time_series(time_series, period=12)
print(f"✓ Time series decomposition:")
print(f" Trend range: [{trend.min():.1f}, {trend.max():.1f}]")
print(f" Seasonal range: [{seasonal.min():.1f}, {seasonal.max():.1f}]")
print(f" Residual std: {residual.std():.2f}")### Lab 2: Autoregressive Model
import numpy as np
class ARModel:
def __init__(self, order=1):
self.order = order
self.coefficients = None
self.intercept = None
def fit(self, time_series):
"""Fit AR model"""
# X: lagged features, y: current values
X = np.zeros((len(time_series)-self.order, self.order))
for i in range(self.order):
X[:, i] = time_series[self.order-1-i:-1-i]
y = time_series[self.order:]
# Solve least squares
self.coefficients = np.linalg.lstsq(X, y, rcond=None)[0]
self.intercept = np.mean(y - X @ self.coefficients)
def predict(self, time_series_history, n_ahead=1):
"""Predict n_ahead steps"""
predictions = []
history = time_series_history[-self.order:].copy()
for _ in range(n_ahead):
X_pred = history[-self.order:][::-1].reshape(1, -1)
y_pred = self.intercept + X_pred @ self.coefficients
predictions.append(y_pred[0])
history = np.append(history, y_pred[0])
return np.array(predictions)
# Test
time_series = np.cumsum(np.random.randn(100))
model = ARModel(order=2)
model.fit(time_series)
predictions = model.predict(time_series, n_ahead=10)
print(f"✓ AR(2) model predictions (next 10 steps):")
print(f" Mean: {predictions.mean():.2f}, Std: {predictions.std():.2f}")### Lab 3: Exponential Smoothing
import numpy as np
class ExponentialSmoothing:
def __init__(self, alpha=0.3, beta=0.1, gamma=0.1):
self.alpha = alpha # Level smoothing
self.beta = beta # Trend smoothing
self.gamma = gamma # Seasonal smoothing
def fit_and_predict(self, time_series, n_ahead=10, season_length=12):
"""Fit exponential smoothing"""
n = len(time_series)
# Initialize
level = time_series[0]
trend = (time_series[season_length] - time_series[0]) / season_length
seasonal = np.zeros(season_length)
# Fit
for t in range(1, n):
prev_level = level
level = self.alpha * time_series[t] + (1 - self.alpha) * (level + trend)
trend = self.beta * (level - prev_level) + (1 - self.beta) * trend
season_idx = t % season_length
seasonal[season_idx] = self.gamma * (time_series[t] - level) + (1 - self.gamma) * seasonal[season_idx]
# Predict
predictions = []
for h in range(1, n_ahead + 1):
y_pred = level + h * trend + seasonal[h % season_length]
predictions.append(y_pred)
return np.array(predictions)
# Test
time_series = 50 + 0.1*np.arange(100) + 10*np.sin(2*np.pi*np.arange(100)/12)
time_series += np.random.randn(100) * 2
smoother = ExponentialSmoothing(alpha=0.2, beta=0.05)
predictions = smoother.fit_and_predict(time_series, n_ahead=12)
print(f"✓ Exponential smoothing forecasts:")
print(f" Predictions: {predictions[:4]}")### Lab 4: Integrated Time Series System
import numpy as np
class TimeSeriesForecastingSystem:
def __init__(self, lookback=12):
self.lookback = lookback
self.model_weights = np.random.randn(lookback) * 0.1
def create_sequences(self, time_series):
"""Create training sequences"""
X, y = [], []
for i in range(len(time_series) - self.lookback):
X.append(time_series[i:i+self.lookback])
y.append(time_series[i+self.lookback])
return np.array(X), np.array(y)
def train(self, time_series, epochs=50, lr=0.01):
"""Train model"""
X, y = self.create_sequences(time_series)
for epoch in range(epochs):
predictions = X @ self.model_weights
errors = predictions - y
self.model_weights -= lr * X.T @ errors / len(y)
def forecast(self, recent_history, n_steps=10):
"""Forecast future values"""
predictions = []
current = recent_history[-self.lookback:].copy()
for _ in range(n_steps):
next_val = np.dot(current, self.model_weights)
predictions.append(next_val)
current = np.append(current[1:], next_val)
return np.array(predictions)
def forecast_with_uncertainty(self, recent_history, n_steps=10, n_simulations=100):
"""Monte Carlo uncertainty quantification"""
all_forecasts = []
for _ in range(n_simulations):
# Add noise to weights
noisy_weights = self.model_weights + np.random.randn(self.lookback) * 0.05
current = recent_history[-self.lookback:].copy()
forecast = []
for _ in range(n_steps):
next_val = np.dot(current, noisy_weights)
forecast.append(next_val)
current = np.append(current[1:], next_val)
all_forecasts.append(forecast)
all_forecasts = np.array(all_forecasts)
mean_forecast = np.mean(all_forecasts, axis=0)
std_forecast = np.std(all_forecasts, axis=0)
return mean_forecast, std_forecast
# Test
time_series = 50 + 0.1*np.arange(200) + 10*np.sin(2*np.pi*np.arange(200)/12)
time_series += np.random.randn(200) * 2
system = TimeSeriesForecastingSystem(lookback=12)
system.train(time_series[:150], epochs=100)
forecast, uncertainty = system.forecast_with_uncertainty(time_series[:150], n_steps=20)
print(f"✓ Time series forecast with uncertainty:")
print(f" Next value: {forecast[0]:.1f} ± {uncertainty[0]:.1f}")
print(f" In 20 steps: {forecast[-1]:.1f} ± {uncertainty[-1]:.1f}")---