Projects

1. Theory of PSRs: We have developed a theory of PSRs (predictive state representations) based on a mathematical construct we call system dynamics matrix that allows us to relate the representational power of PSR to other knowledge representation schemes such as Markov models, POMDPs, and (Jaeger's) OOMs.
Papers:
• Predictive State Representations: A New Theory for Modeling Dynamical Systems by Satinder Singh, Michael R. James and Matthew R. Rudary. In Uncertainty in Artificial Intelligence: Proceedings of the Twentieth Conference (UAI), pages 512-519, 2004.
  pdf.
• Predictive Representations of State by Michael Littman, Richard Sutton and Satinder Singh. In Advances in Neural Information Processing Systems 14 (NIPS), pages 1555-1561, 2002.
  gzipped postscript pdf.
2. Nonlinear PSRs: We know that nonlinear PSRs can be significantly more compact than linear PSRs. Currently we have developed a class of nonlinear PSR models that are suited to deterministic domains. Our goal is to develop new families of nonlinear PSRs that are suited to other large subclasses of dynamical systems of interest.
Papers:
• A Nonlinear Predictive State Representation by Matthew Rudary and Satinder Singh. In Advances in Neural Information Processing Systems 16 (NIPS), pages 855-862, 2004.
  pdf.
3. Learning and Discovery in PSRs: We are still lacking efficient learning and discovery algorithms. This needs to be remedied.
Papers:
• Learning and Discovery of Predictive State Representations in Dynamical Systems with Reset by Michael James and Satinder Singh. In Proceedings of the Twenty-First International Conference on Machine Learning (ICML), pages 417-424, 2004.
  pdf.
• Learning Predictive State Representations by Satinder Singh, Michael Littman, Nicholas Jong, David Pardoe and Peter Stone. In Proceedings of the Twentieth International Conference on Machine Learning (ICML), pages 712-719, 2003.
  gzipped postscript.
4. Planning with PSRs: The goal here is to develop planning algorithms that can exploit the multi-scale temporal aspects of the state representation in PSRs.
Papers
• Planning with Predictive State Representations by Michael R. James, Satinder Singh, and Michael Littman. Submitted, 2004.
  pdf.
5. Options and PSRs: Options are temporally abstract representations of action. PSRs are temporally abstract predictions as representations of state. Combining both could lead to a powerful advance in our ability to build autonomous AI.