Mcmc Sampling and Markov Chain Methods

# MCMC Sampling and Markov Chain Methods

## Introduction & Motivation

MCMC methods sample from complex posterior distributions, enabling Bayesian inference when closed-form solutions are unavailable. Critical for uncertainty quantification in scientific computing and complex engineering problems.

Motivation: Sample from intractable posteriors via MCMC.

Applications: Bayesian inference, parameter estimation, uncertainty quantification, model averaging.

---

## Core Concepts & Theory

### Markov Chain

State transition dynamics.

### Stationary Distribution

Convergence properties.

### Burn-in Period

Mixing time.

### Autocorrelation

Effective sample size.

---

## Mathematical Formulation

Metropolis-Hastings:
$$\alpha = \min\left(1, \frac{p( heta^*)q( heta| heta^*)}{p( heta)q( heta^*| heta)} ight)$$

Gibbs Sampling:
$$ heta_j^{(t+1)} \sim p( heta_j | heta_{-j}^{(t)})$$

Effective Sample Size:
$$N_{eff} = \frac{N}{1 + 2\sum_{k=1}^\infty ho(k)}$$

---

## Advanced Theory & Extensions

### Hamiltonian MC

Gradient-based sampling.

### Adaptive MCMC

Self-tuning proposals.

### Sequential MC

Particle filtering.

---

## Computational Considerations

Metropolis-Hastings: O(D) per iteration.

Gibbs: O(D·C) for conditional complexity C.

Convergence: O(mixing time) + O(samples needed).

---

## Practical Implementation Strategies

### Proposal Tuning

Acceptance rates 0.234.

### Burn-in Selection

Diagnostic plots.

### Thinning

Reducing autocorrelation.

---

## Benchmark Datasets & Evaluation

Synthetic Posteriors: Known distributions.

Regression Problems: Parameter inference.

Hierarchical Models: Multi-level structures.

---

## Key Challenges & Limitations

### Mixing

Slow convergence.

### Diagnostics

Assessing convergence.

### Computational Cost

High-dimensional problems.

---

## Hyperparameter Tuning

Proposal variance: Adaptive tuning.

Chain length: 1000-100000.

Burn-in: 10-50% of samples.

---

## Real-World Applications & Case Studies

Bayesian Linear Regression: Parameter inference.

Hierarchical Models: Multi-level analysis.

Model Selection: Evidence computation.

---

## Integration with Other Methods

MCMC + Bayesian methods; + model averaging; + uncertainty quantification.

---

## Summary & Key Takeaways

MCMC enables Bayesian inference from complex posteriors.

Principles:
1. Markov Chain: State transitions.
2. Acceptance: Metropolis criterion.
3. Stationarity: Convergence to target.
4. Mixing: Effective sampling.
5. Diagnostics: Assessing quality.

---

## Appendix: Practical Labs

### Lab 1: Metropolis-Hastings Algorithm

import numpy as np

def metropolis_hastings(log_posterior, theta0, proposal_cov, n_iter=5000):
 """Metropolis-Hastings sampling"""
 samples = [theta0.copy()]
 accepted = 0
 
 for i in range(n_iter):
 theta_prop = samples[-1] + np.random.multivariate_normal(np.zeros_like(theta0), proposal_cov))
 
 log_alpha = log_posterior(theta_prop) - log_posterior(samples[-1])
 
 if np.log(np.random.uniform()) < log_alpha:
 samples.append(theta_prop)
 accepted += 1
 else:
 samples.append(samples[-1])
 
 acceptance_rate = accepted / n_iter
 return np.array(samples), acceptance_rate

def log_posterior_normal(theta, y=0):
 """Simple normal posterior"""
 return -0.5 * np.sum(theta**2) - 0.5 * np.sum((y - theta)**2)

theta0 = np.array([0.0, 0.0])
proposal_cov = np.eye(2) * 0.1
y = np.array([1.0, 2.0])

samples, acc_rate = metropolis_hastings(lambda t: log_posterior_normal(t, y), theta0, proposal_cov, n_iter=1000)

print(f"✓ Acceptance rate: {acc_rate:.2%}")
print(f"✓ Posterior mean: {np.mean(samples[100:], axis=0)}")

### Lab 2: Gibbs Sampling

import numpy as np

def gibbs_sampling(n_iter=1000, initial=(0, 0)):
 """Gibbs sampling for bivariate normal"""
 samples = [initial]
 x, y = initial
 
 for _ in range(n_iter):
 # Sample x | y
 x = np.random.normal(0.9 * y, np.sqrt(1 - 0.9**2))
 
 # Sample y | x
 y = np.random.normal(0.9 * x, np.sqrt(1 - 0.9**2))
 
 samples.append((x, y))
 
 return np.array(samples)

samples = gibbs_sampling(n_iter=5000)

print(f"✓ Gibbs correlation: {np.corrcoef(samples.T)[0, 1]:.3f}")

### Lab 3: Convergence Diagnostics

import numpy as np

def rhat_statistic(chains):
 """Gelman-Rubin convergence diagnostic"""
 m, n = chains.shape
 chain_means = np.mean(chains, axis=1)
 overall_mean = np.mean(chain_means)
 
 B = n * np.sum((chain_means - overall_mean) ** 2) / (m - 1)
 W = np.mean(np.var(chains, axis=1))
 
 var_hat = (n - 1) / n * W + B / n
 rhat = np.sqrt(var_hat / W)
 
 return rhat

# Multiple chains
chains = np.array([
 np.random.randn(100) + 0.1 * np.arange(100),
 np.random.randn(100) + 0.1 * np.arange(100),
 np.random.randn(100) + 0.1 * np.arange(100)
])

rhat = rhat_statistic(chains)
print(f"✓ Rhat: {rhat:.3f} (< 1.1 indicates convergence)")

### Lab 4: Adaptive MCMC

import numpy as np

class AdaptiveMCMC:
 def __init__(self, theta0, target_acceptance=0.234):
 self.theta = theta0.copy()
 self.target_acc = target_acceptance
 self.proposal_cov = np.eye(len(theta0)) * 2.38**2 / len(theta0)
 
 def step(self, log_posterior, iteration):
 """Adaptive MH step"""
 theta_prop = self.theta + np.random.multivariate_normal(np.zeros_like(self.theta), self.proposal_cov))
 
 log_alpha = log_posterior(theta_prop) - log_posterior(self.theta)
 
 accept = np.log(np.random.uniform()) < log_alpha
 
 if accept:
 self.theta = theta_prop
 
 # Adapt proposal
 if iteration % 100 == 0:
 self.proposal_cov *= np.exp(0.1 * (accept - self.target_acc))
 
 return self.theta, accept

def log_post(theta):
 return -0.5 * np.sum(theta**2)

mcmc = AdaptiveMCMC(np.array([0.0, 0.0]))

for i in range(1000):
 theta, accept = mcmc.step(log_post, i)

print(f"✓ Adaptive MCMC complete")

---