ODE-RNN is a hybrid sequence model that combines Neural ODEs for continuous-time state evolution between observations with Recurrent Neural Networks for discrete state updates at observation times — addressing the irregular time series challenge by modeling the continuous dynamics of a hidden state between measurement events and incorporating each new observation via a standard gated RNN update, providing a practical middle ground between purely continuous Neural ODE models and discrete RNNs that lack principled continuous-time semantics.

Motivation: The Best of Both Worlds

Standard RNNs process sequences at discrete time steps: h_{n+1} = RNN(h_n, x_{n+1}). For irregular sequences, this creates two problems: 1. The model cannot distinguish Δt = 1 hour from Δt = 1 day — both produce the same update 2. Zero-padding for missing time steps introduces artificial "no observation" signals that bias the hidden state

Neural ODEs provide continuous-time dynamics but are purely deterministic between observations — they cannot incorporate new information from sparse observations without adding encoder complexity (as in Latent ODEs).

ODE-RNN solves this by splitting the processing into two distinct phases:

Phase 1 — Between observations (Neural ODE): Given current hidden state h(tₙ) and next observation time tₙ₊₁, integrate the ODE: h(tₙ₊₁⁻) = h(tₙ) + ∫_{tₙ}^{tₙ₊₁} f(h(s), s; θ_ode) ds

The state evolves continuously, with dynamics that decay or oscillate according to the learned vector field f.

Phase 2 — At observations (GRU/LSTM update): Incorporate the new observation xₙ₊₁ using a standard gated RNN: h(tₙ₊₁) = GRU(h(tₙ₊₁⁻), xₙ₊₁)

The RNN update can also be replaced by an attention mechanism for long-range dependencies.

Architecture Diagram

h(t₀) →[Neural ODE: t₀→t₁]→ h(t₁⁻) →[GRU+x₁]→ h(t₁) →[Neural ODE: t₁→t₂]→ h(t₂⁻) →[GRU+x₂]→ h(t₂) → ...

The Neural ODE segments can have arbitrary, different durations — Δt₁ ≠ Δt₂ — and the model correctly accounts for this through the integration.

Temporal Decay Properties

The Neural ODE dynamics between observations can implement several principled behaviors:

• Exponential decay: f(h) = -λh forces the state to decay toward zero between observations (appropriate for sensor readings that become stale)
• Oscillatory dynamics: f(h) = Ah (linear system) captures periodic patterns in the underlying process
• Arbitrary nonlinear dynamics: The full neural network f(h, t; θ) can represent complex attractor dynamics

For many real-world processes, the learned dynamics often resemble exponential decay — the model effectively learns to discount stale information.

Comparison to Alternative Models

Applications

Electronic Health Records: Clinical notes, lab values, and vital signs arrive at irregular intervals determined by patient condition and care protocols. ODE-RNN outperforms standard LSTM on mortality prediction and disease onset prediction by properly accounting for time elapsed between measurements.

Event-Based Sensors: Neuromorphic cameras and event-based IMUs generate observations asynchronously. ODE-RNN processes these sparse event streams without discretization artifacts.

Financial Market Data: High-frequency trading data has variable inter-trade intervals. ODE-RNN captures the continuous price dynamics between trades rather than artificially resampling to a fixed grid.

ODE-RNN is implemented in the torchdiffeq library (alongside Neural ODEs) and has been replicated in Julia's DifferentialEquations.jl ecosystem. The simple conceptual structure — ODE between observations, RNN at observations — makes it the most accessible entry point to continuous-time sequence modeling.