RAS Journal Article Published (Open Access)

15 September 2020

Photo: elsevier.com

Knowledge Technology got a new journal article published in the Journal of Robotics and Autonomous Systems (RAS). The article describes our research on meta-decision making and experience imagination for robot learning. We found that local learning progress of a latent-space world model can be used to dynamically arbitrate between model-based and model-free decisions and generate imagined experiences for efficient motor learning.

Paper title: Improving robot dual-system motor learning with intrinsically motivated meta-control and latent-space experience imagination
Authors: Burhan Hafez, Cornelius Weber, Matthias Kerzel, and Stefan Wermter
Abstract: Combining model-based and model-free learning systems has been shown to improve the sample efficiency of learning to perform complex robotic tasks. However, dual-system approaches fail to consider the reliability of the learned model when it is applied to make multiple-step predictions, resulting in a compounding of prediction errors and performance degradation. In this paper, we present a novel dual-system motor learning approach where a meta-controller arbitrates online between model-based and model-free decisions based on an estimate of the local reliability of the learned model. The reliability estimate is used in computing an intrinsic feedback signal, encouraging actions that lead to data that improves the model. Our approach also integrates arbitration with imagination where a learned latent-space model generates imagined experiences, based on its local reliability, to be used as additional training data. We evaluate our approach against baseline and state-of-the-art methods on learning vision-based robotic grasping in simulation and real world. The results show that our approach outperforms the compared methods and learns near-optimal grasping policies in dense-and sparse-reward environments.

The paper can be found here. The code is available on Github: https://github.com/knowledgetechnologyuhh/Imagination-Arbitration