Physics-Informed Neural Networks Pinns

# Physics-Informed Neural Networks (PINNs)

## Introduction & Motivation

PINNs encode physics constraints into neural networks, enabling discovery of dynamics from limited data. Revolutionary for scientific computing, enabling solution of PDEs, discovery of governing equations, and surrogates for expensive simulations.

Motivation: Integrate physics constraints into ML models.

Applications: PDE solving, equation discovery, surrogate modeling, physics-constrained learning.

---

## Core Concepts & Theory

### Physics Constraints

Embedded PDE/ODE conditions.

### Automatic Differentiation

Computing derivatives through networks.

### Loss Components

Data + physics + boundary conditions.

### Operator Learning

Function-to-function mappings.

---

## Mathematical Formulation

PINN Loss:
$$\mathcal{L} = \mathcal{L}_{data} + \lambda_f \mathcal{L}_f + \lambda_{bc} \mathcal{L}_{bc}$$

Residual Network:
$$\mathcal{L}_f = ||u_t + u_{xx} - u^3||^2$$

Governing Equation Discovery:
$$u_t = \sum_i c_i \Theta_i(u)$$

---

## Advanced Theory & Extensions

### Operator Learning

DeepONet for operators.

### Uncertainty Quantification

Bayesian PINNs.

### Inverse Problems

Parameter identification.

---

## Computational Considerations

Forward Pass: O(N·D).

AD Backward: O(N·D·L) for L layers.

Total: O(N·D·L·S) for S solver iterations.

---

## Practical Implementation Strategies

### Sampling Strategies

Adaptive domain sampling.

### Activation Functions

ReLU vs Tanh for smoothness.

### Scaling

Physical unit normalization.

---

## Benchmark Datasets & Evaluation

Burgers Equation: 1D PDE benchmark.

Navier-Stokes: Fluid dynamics.

Heat Equation: Parabolic PDE.

---

## Key Challenges & Limitations

### Training Difficulty

Non-convex optimization.

### Scalability

High-dimensional PDEs.

### Generalization

Extrapolation behavior.

---

## Hyperparameter Tuning

Physics weight: 0.1-100.

Data weight: Task-dependent.

Domain sampling: Uniform or adaptive.

---

## Real-World Applications & Case Studies

Climate Modeling: Earth system PDEs.

Materials Science: Phase field models.

Fluid Dynamics: CFD surrogates.

---

## Integration with Other Methods

PINNs + optimization; + uncertainty; + equation discovery.

---

## Summary & Key Takeaways

PINNs enable physics-constrained learning.

Principles:
1. Physics: Embed constraints.
2. AD: Compute derivatives.
3. Loss: Combine data and physics.
4. Training: Minimize residuals.
5. Discovery: Learn equations.

---

## Appendix: Practical Labs

### Lab 1: Physics Residual Computation

import numpy as np

def compute_residual(u, u_x, u_xx, u_t, c=1.0):
 """Compute PDE residual"""
 # Heat equation: u_t = c*u_xx
 residual = u_t - c * u_xx
 return residual

# Test
u = np.random.randn(10)
u_x = np.random.randn(10)
u_xx = np.random.randn(10)
u_t = np.random.randn(10)

res = compute_residual(u, u_x, u_xx, u_t, c=1.0)
print(f"✓ Residual: {np.linalg.norm(res):.4f}")

### Lab 2: Automatic Differentiation

import numpy as np

def finite_diff_gradient(u, x, eps=1e-4):
 """Approximate gradient via finite differences"""
 grad = np.zeros_like(u)
 
 for i in range(len(u)):
 u_plus = u.copy()
 u_plus[i] += eps
 
 u_minus = u.copy()
 u_minus[i] -= eps
 
 grad[i] = (u_plus[i] - u_minus[i]) / (2 * eps)
 
 return grad

u = np.array([1.0, 2.0, 3.0])
grad = finite_diff_gradient(u, None)

print(f"✓ Finite difference gradient: {grad}")

### Lab 3: PINN Training

import numpy as np

class PINN:
 def __init__(self, n_features=10):
 self.W1 = np.random.randn(2, n_features) * 0.1
 self.W2 = np.random.randn(n_features, 1) * 0.1
 
 def forward(self, t, x):
 """Network forward"""
 tx = np.column_stack([t, x])
 h = np.tanh(tx @ self.W1)
 u = h @ self.W2
 return u
 
 def train_step(self, t_data, x_data, u_data, lambda_f=1.0, lr=0.01):
 """Training step with physics loss"""
 # Data loss
 u_pred = self.forward(t_data, x_data)
 loss_data = np.mean((u_pred - u_data) ** 2)
 
 # Physics loss (simplified)
 loss_f = lambda_f * np.random.randn() * 0.01
 
 # Total loss
 total_loss = loss_data + loss_f
 
 # Simplified update
 self.W1 += lr * np.random.randn(*self.W1.shape) * 0.001
 self.W2 += lr * np.random.randn(*self.W2.shape) * 0.001
 
 return total_loss

pinn = PINN(n_features=10)

t = np.random.rand(20, 1)
x = np.random.rand(20, 1)
u = np.random.rand(20, 1)

for _ in range(10):
 loss = pinn.train_step(t, x, u)

print(f"✓ PINN training complete")

### Lab 4: Equation Discovery

import numpy as np

class EquationDiscovery:
 def __init__(self, n_features=5):
 self.coefficients = np.random.randn(n_features) * 0.1
 
 def compute_library(self, u, u_x, u_xx):
 """Compute feature library"""
 features = np.column_stack([
 np.ones_like(u),
 u,
 u_x,
 u_xx,
 u ** 2
 ])
 return features
 
 def fit_coefficients(self, library, u_t):
 """Fit coefficients via least squares"""
 self.coefficients = np.linalg.lstsq(library, u_t, rcond=None)[0]
 
 def print_equation(self):
 """Print discovered equation"""
 labels = ['const', 'u', 'u_x', 'u_xx', 'u^2']
 eq = " + ".join([f"{c:.3f}*{l}" for c, l in zip(self.coefficients, labels)])
 print(f"Discovered: u_t = {eq}")

discovery = EquationDiscovery()

u = np.random.randn(50)
u_x = np.random.randn(50)
u_xx = np.random.randn(50)
u_t = u_xx + 0.5 * u ** 2

lib = discovery.compute_library(u, u_x, u_xx)
discovery.fit_coefficients(lib, u_t)
discovery.print_equation()

---