Differentiable Simulation

A simulator is just a function applied over and over. Each step maps the current state to the next, $\mathbf{s}_{t+1}=f(\mathbf{s}_t,\theta)$, where $\theta$ holds the parameters you might want to tune: controls, masses, stiffnesses. A rollout of $T$ steps is the composition $f\circ f\circ\cdots\circ f$. Differentiable simulation propagates gradients through that composition, by autodiff or by analytic derivatives, so you can optimize $\theta$ with gradient descent instead of black-box search.¹²

The chain rule through a rollout

By the chain rule, the Jacobian of the whole rollout is the product of the per-step Jacobians. Take semi-implicit Euler on a free particle under constant acceleration $a$, with state $\mathbf{s}=(x,v)$:

$$v_{t+1}=v_t+a\,h,\qquad x_{t+1}=x_t+v_{t+1}\,h.$$

Differentiating one step with respect to the state (the acceleration is constant, so it drops out) gives a single $2\times2$ block:

$$\frac{\partial \mathbf{s}_{t+1}}{\partial \mathbf{s}_t}= \begin{bmatrix}1 & h\\ 0 & 1\end{bmatrix}.$$

At $h=\tfrac{1}{60}\approx0.0167$ that is

$$J=\begin{bmatrix}1 & 0.0167\\ 0 & 1\end{bmatrix}.$$

Composing $T$ steps multiplies $T$ copies of $J$. For $T=2$, multiply it out entry by entry:

$$J^2= \begin{bmatrix}1 & h\\ 0 & 1\end{bmatrix} \begin{bmatrix}1 & h\\ 0 & 1\end{bmatrix} =\begin{bmatrix}1 & 2h\\ 0 & 1\end{bmatrix} =\begin{bmatrix}1 & 0.0333\\ 0 & 1\end{bmatrix}.$$

The pattern holds for any $T$:

$$\frac{\partial \mathbf{s}_T}{\partial \mathbf{s}_0}=J^{T}= \begin{bmatrix}1 & T h\\ 0 & 1\end{bmatrix}.$$

The off-diagonal $Th$ is exactly the elapsed time, which reads as: nudging the initial velocity shifts the final position by $Th$. In smooth free flight the gradient is exact, cheap, and stable no matter how many steps you chain. This is the case differentiable simulators are built for.

Where contact breaks it

Contact is where the clean product falls apart. At a bounce the velocity flips,

$$v^{+}=-e\,v^{-},$$

with restitution $e$. Around the instant of contact, $f$ is not differentiable: the state jumps, and the sensitivity of "did it hit?" to the inputs is a step, not a slope. Push contact through a stiff spring instead and the derivative exists but is enormous and short-lived, so a single step can miss it entirely.

The consequence is measured, not just aesthetic. Naively differentiating through stiff or discontinuous contact yields gradients that are biased or high variance, and Suh et al. show first-order (differentiable-simulator) gradient estimators can be worse than the zeroth-order estimators that never differentiate the simulator at all.³ The usual fixes smooth the contact: replace the hard switch with an analytic softening, or average the gradient over randomized perturbations so the expectation is differentiable even where a sample is not.

The demo below is the smallest version of this. A projectile is launched at a fixed $45^\circ$; the only knob is the launch speed $v_0$. A wall sits between the launcher and the target. If the ball clears the wall it flies on to a range $R=v_0^2/g$; if not, it stops dead at the wall. The loss is the squared miss distance to the target. Watch the loss curve as you drag $v_0$.

A loss landscape with a contact cliffinteractive

v₀ = 8.00 m/s

landing x

6.00 m

loss (x − 10)²

16.00

clears wall

no

The step in the right-hand curve is where the contact mode flips from blocked to clearing. On the flat plateau the gradient dloss/dv₀ is zero; at the step it is undefined. Neither one points toward the target, which is why contact makes differentiable simulation hard.

On the flat plateau the gradient is zero: a little more speed changes nothing because the ball still hits the wall in the same place. At the cliff the gradient is undefined. Only inside the smooth basin, once contact is out of the way, does $\mathrm{d}\,\text{loss}/\mathrm{d}v_0$ actually point at the target. That narrow regime, exact where motion is smooth and blind where contact switches, is the whole story of why differentiable simulation is powerful in free flight and hard the moment things touch.

Footnotes

1. Hu et al., DiffTaichi: Differentiable Programming for Physical Simulation (ICLR 2020). ↩

2. Freeman et al., Brax: A Differentiable Physics Engine for Large Scale Rigid Body Simulation (2021). ↩

3. Suh, Simchowitz, Zhang & Tedrake, Do Differentiable Simulators Give Better Policy Gradients? (ICML 2022). ↩