Back to the program.
The Ant Colony Learning Framework for Control Policy Learning in Continuous State Spaces
Jelmer van Ast
Delft Center for Systems and Control;; Delft University of Technology
On 2010-06-10 at 10:15:00 (Brussels Time)

Abstract

In this talk, I will present a framework for the automatic learning of control policies for non-linear systems with continuous state spaces. The framework is based on the Ant Colony Optimization class of algorithms, which is inspired by the foraging behavior of ants. We have called our framework Ant Colony Learning (ACL). A collection of agents - called ants - jointly interact with the system at hand in order to find the optimal mapping between states and actions. Through the stigmergic interaction by - so called - pheromones, the ants are guided by each others experience towards better control policies. In order to deal with continuous state spaces, we establish a generalization of the concept of pheromones and the local and global pheromone update rules. As a result of this generalization, we can integrate both crisp and fuzzy partitioning of the state space into the ACL framework. We analyze the performance of ACL by applying it to the control problem of swinging-up and stabilizing an under-actuated pendulum.

Keywords

ant colony optimization, control policy learning, fuzzy interpolation

References

  1. van Ast, J. M., Babuska, R., and De Schutter, B.. (2008) Ant colony optimization for optimal control. In Proceedings of the 2008 Congress on Evolutionary Computation (CEC 2008). pp. 2040-2046.
  2. van Ast, J. M., Babuska, R., and De Schutter, B.. (2009) Fuzzy ant colony optimization for optimal control. In Proceedings of the American Control Conference (ACC 2009). pp. 1003-1008.
  3. van Ast, J. M., Babuska, R., and De Schutter, B.. (2009) Novel ant colony optimization approach to optimal control, International Journal of Intelligent Computing and Cybernetics, 2(3):414 - 434.
  4. van Ast, J. M., Babuska, R., and De Schutter, B.. (2010) Ant colony learning algorithm for optimal control. In Babuska, R. and Groen, F. C. A. (ed.) Interactive Collaborative Information Systems, volume 281 of Studies in Computational Intelligence. Springer, Berlin / Heidelberg. pp. 155 - 182.
  5. van Ast, J. M., Babuska, R., and De Schutter, B.. (2010) Generalized pheromone update for ant colony learning in continuous state spaces. In Proceedings of the 2010 Congress on Evolutionary Computation (CEC 2010). Accepted for publication.