Actively Learning Gaussian Process Dynamics

  Copyright: © Buisson-Fenet / MPI-IS  

With the rising complexity of dynamical systems generating ever more data, learning dynamics models appears as a promising alternative to physics-based modeling. F or high dimensional systems with nonlinear effects, this can be a challenging task. In particular, it still remains an open problem to learn dynamical systems in a sample-efficient way , by gathering informative data that is representative of the whole state space. The question of active learning then becomes important: which control inputs should be chosen by the user so that the data generated during an experiment is informative, and enables efficient training of the dynamics model?

In this context, Gaussian processes can be a useful framework for approximating system dynamics. Indeed, they perform well on small and medium sized data sets, as opposed to most other machine learning frameworks. This is particularly important considering data is often costly to generate and process, most of all when producing it involves actuating a complex physical system. Gaussian processes also yield a notion of uncertainty, which indicates how sure the model is about its predictions.

In our papers L4DC 2019, Buisson-Fenet 2019 , we investigate in a principled way how to actively learn dynamical systems, by selecting control inputs that generate informative data. We model the system dynamics by a Gaussian process, and use information-theoretic criteria to identify control trajectories that maximize the information gain. Thus, the input space can be explored efficiently, leading to a data-efficient training of the model. We propose several methods, investigate their theoretical properties and compare them extensively in a numerical benchmark. The final method proves to be efficient at generating informative data. Thus, it yields the lowest prediction error with the same amount of samples on most benchmark systems. We propose several variants of this method, allowing the user to trade off computations with prediction accuracy, and show it is versatile enough to take additional objectives into account. In future work, we intend to study the effects of different cost functions and to generalize our framework to more realistic GP models, for example with noisy inputs and latent states. T he methods also need to be computationally optimized so that they can be validated on hardware experiments.