# Physically Structured Neural Networks

$$\htmlData{fragment-index=1,class=fragment}{ x_0 } \qquad \htmlData{fragment-index=2,class=fragment}{ x_1 = x_0 + f(x_0)\Delta t } \qquad \htmlData{fragment-index=3,class=fragment}{ x_2 = x_1 + f(x_1)\Delta t } \qquad \htmlData{fragment-index=4,class=fragment}{ .. }$$

# Neural Ordinary Differential Equations

$$\htmlData{fragment-index=0,class=fragment}{ x_{t+1} = x_t + f(x_t,\theta) } \htmlData{fragment-index=1,class=fragment}{ \mathllap{\ubr{\phantom{x_{t+1} = x_t + f(x_t,\theta)}\vphantom{\frac{\d x_t}{\d t}}}{\t{residual network}}} } \qquad \htmlData{fragment-index=2,class=fragment}{ \leadsto } \qquad \htmlData{fragment-index=3,class=fragment}{ \frac{\d x_t}{\d t} = f(x_t, \theta) } \htmlData{fragment-index=4,class=fragment}{ \mathllap{\ubr{\phantom{\frac{\d x_t}{\d t} = f(x_t, \theta)}}{\t{neural ODE}}} }$$

# Physically Structured Recurrent Neural Networks

$$\htmlData{fragment-index=0,class=fragment}{ \frac{\d x_t}{\d t} = f(x_t, \theta) } \htmlData{fragment-index=1,class=fragment}{ \mathllap{\ubr{\phantom{\frac{\d x_t}{\d t} = f(x_t, \theta)}\vphantom{\frac{\pd L_\theta}{\pd q}}}{\t{black-box ODE}}} } \qquad \htmlData{fragment-index=2,class=fragment}{ \leadsto } \qquad \htmlData{fragment-index=3,class=fragment}{ \frac{\d}{\d t} \frac{\pd L_\theta}{\pd \dot{q}} - \frac{\pd L_\theta}{\pd q} = 0 } \htmlData{fragment-index=4,class=fragment}{ \mathllap{\ubr{\phantom{\frac{\d}{\d t} \frac{\pd L_\theta}{\pd \dot{q}} - \frac{\pd L_\theta}{\pd q} = 0}}{\t{Euler-Lagrange equations}}} }$$

# Physically Structured Recurrent Neural Networks

$$\footnotesize \htmlData{fragment-index=0,class=fragment}{ \begin{gathered} q_0 \\ p_0 \end{gathered} } \qquad \begin{gathered} \htmlData{fragment-index=1,class=fragment}{ q_1 = q_0 + {\textstyle\frac{\pd H}{\pd q}}(q_0,p_0) \Delta t } \\ \htmlData{fragment-index=2,class=fragment}{ p_1 = p_0 - {\textstyle\frac{\pd H}{\pd p}}(q_1,p_0) \Delta t } \end{gathered} \qquad \begin{gathered} \htmlData{fragment-index=3,class=fragment}{ q_2 = q_1 + {\textstyle\frac{\pd H}{\pd q}}(q_1,p_1) \Delta t } \\ \htmlData{fragment-index=4,class=fragment}{ p_2 = p_1 - {\textstyle\frac{\pd H}{\pd p}}(q_2,p_1) \Delta t } \end{gathered} \qquad \htmlData{fragment-index=5,class=fragment}{ .. }$$

# Variational Integrator Networks

$$\htmlData{fragment-index=0,class=fragment}{ \frac{\d}{\d t} \frac{\pd L_\theta}{\pd \dot{q}} - \frac{\pd L_\theta}{\pd q} = 0 } \htmlData{fragment-index=1,class=fragment}{ \mathllap{\ubr{\phantom{\frac{\d}{\d t} \frac{\pd L_\theta}{\pd \dot{q}} - \frac{\pd L_\theta}{\pd q} = 0}\vphantom{\int_{t_0}^{t_1}}}{\t{Euler-Lagrange equations}}} } \qquad \htmlData{fragment-index=2,class=fragment}{ \leadsto } \qquad \htmlData{fragment-index=3,class=fragment}{ S(\v{q},\v{\dot{q}}) = \int_{t_0}^{t_1} L_\theta(\v{q}_t,\v{\dot{q}}_t) \d t } \htmlData{fragment-index=4,class=fragment}{ \mathllap{\ubr{\phantom{S(\v{q},\v{\dot{q}}) = \int_{t_0}^{t_1} L_\theta(\v{q}_t,\v{\dot{q}}_t) \d t}}{\t{Principle of Least Action}}} }$$

# Variational Integrator Networks

$$L^d_\theta(\v{q}_t,\v{q}_{t+1}, h) \approx \int_t^{t+h} L_\theta(\v{q}(\tau),\v{\dot{q}}(\tau)) \d \tau$$

# Contact Dynamics

$$\htmlClass{fragment}{ \delta S(\v{q},\v{\dot{q}}) = 0 } \qquad \htmlClass{fragment}{ S(\v{q},\v{\dot{q}}) = \int_{t_0}^{t_1} L_\theta(\v{q}_t,\v{\dot{q}}_t) \d t }$$

# Thank you!

### https://avt.im/· @avt_im

S. Sæmundsson, A. Terenin, K. Hofmann, M. P. Deisenroth. Variational Integrator Networks for Physically Structured Embeddings. Artificial Intelligence and Statistics, 2020.

A. Hochlehnert, A. Terenin, S. Sæmundsson, M. P. Deisenroth. Learning Contact Dynamics using Physically Structured Neural Networks. Artificial Intelligence and Statistics, 2021.