torchscript

TorchScript is PyTorch's intermediate representation that compiles Python models into optimized computation graphs, enabling deployment without Python runtime and improving performance through JIT compilation and graph optimization. Purpose: (1) production deployment (remove Python dependency), (2) performance (graph optimization, fusion), (3) portability (run on C++ runtime, mobile devices), (4) serialization (save model as single file). Creation methods: (1) tracing (torch.jit.trace—record operations on example input, captures data flow), (2) scripting (torch.jit.script—parse Python code, captures control flow). Tracing: model(example_input) → records operations → creates graph. Limitations: doesn't capture control flow (if/for), uses fixed shapes from example. Scripting: analyzes Python source code → converts to TorchScript. Supports control flow, type annotations required. Hybrid: trace outer model, script inner modules with control flow. Optimizations: (1) operator fusion (Conv-BN-ReLU → single op), (2) constant folding (pre-compute constants), (3) dead code elimination, (4) algebraic simplification. Deployment: (1) save (torch.jit.save), (2) load in C++ (torch::jit::load), (3) run inference (no Python needed). Mobile: PyTorch Mobile uses TorchScript for on-device inference (iOS, Android). Advantages: (1) faster inference (optimized graph), (2) no Python overhead, (3) portable (C++, mobile), (4) serializable (single file). Limitations: (1) not all Python features supported (dynamic types, some libraries), (2) debugging harder (compiled code), (3) tracing limitations (control flow). Use cases: (1) production serving (C++ backend), (2) mobile deployment, (3) embedded systems, (4) performance-critical applications. Comparison: ONNX (framework-agnostic, wider tool support), TorchScript (PyTorch-native, better PyTorch integration). TorchScript is standard for deploying PyTorch models in production environments requiring performance and portability.