Intrinsically Motivated Reinforcement Learning

Project Description

(Joint with Andrew G. Barto at UMass and Michael Littman at Rutgers)

The overall objective is to develop a theory and algorithms for building agents that can achieve a level of competence and mastery over their environment considerably greater than currently possible in machine learning and artificial intelligence. The basic approach is to have an extended developmental period during which an autonomous agent can learn collections of reusable skills that will be useful for a wide range of later challenges. While this basic idea has been explored by a number of researchers, past efforts in this direction have been mostly exploratory in nature and have not yet produced significant advances. Furthermore, they have not yet led to the kind of rigorous formulation needed to bring powerful mathematical methods to bear, a pre-requisite for engaging the largest part of the machine learning research community. At the core of our research in this topic are recent theoretical and algorithmic advances in computational reinforcement learning. We will also build on recent advances in the neuroscience of brain reward systems as well as classical and contemporary psychological theories of motivation.

The methods we will develop, if successful, will give artificial learning systems the ability to extend their abilities, in a generative and potentially unlimited way, through the accumulation of deep hierarchical repertoires of reusable skills. We will demonstrate this success on a succession of simulated and robotic agents.

Current Papers

  • Intrinsically Motivated Reinforcement Learning by Satinder Singh, Andrew G. Barto and Nuttapong Chentanez. To appear in Proceedings of Advances in Neural Information Processing Systems 17 (NIPS), 2005.
    pdf.

  • Intrinsically Motivated Learning of Hierarchical Collections of Skills by Andrew G. Barto, Satinder Singh, and Nuttapong Chentanez. To appear in Proceedings of International Conference on Developmental Learning (ICDL), 2004.
    pdf.