Towards physically structured probabilistic reinforcement learning
· Talk · University College London
Abstract. Owing to their data-efficiency, sparse Gaussian process methods have proven to be a promising approach for tasks such as modeling of dynamical systems in robotics. In spite of this, their use in large-scale reinforcement learning remains limited, when compared to deep Q-learning. We present recent work that uses physically-meaningful deep networks to construct data-efficient embeddings of dynamical systems. We illustrate theoretical and computational issues that arise when attempting to apply similar constructions to obtain physically-meaningful Gaussian processes with dimensionally-efficient behavior. To overcome these issues, we present a Gaussian process discretization technique that does not rely on grids of points and instead produces entire function draws, which can be evaluated deterministically at arbitrary locations. We discuss approximation accuracy, and prove a bound on the approximation error induced by this technique. We conclude with a discussion on future work and directions that follow.