Neuroevolution is a new and emerging area of reinforcement learning (RL). Whereas the traditional value-function-based approach focuses mostly on MDP problems and on maximizing lifetime reward, neuroevolution work focuses mostly on POMDP tasks and on maximizing reward at the end of learning. Knowledge of neuroevolution should thus be most valuable for researchers and students in robotics, intelligent agents, and multiagent systems.
In the first part of this tutorial, Neuroevolution RL will first be motivated and compared with value-function RL. Neuroevolution techniques will then be reviewed and illustrated with applications in control, robotics, artificial life, and game playing. Throughout, future research opportunities will be highlighted, including generative and developmental approaches to neuroevolution, combining learning and evolution, and open-ended evolution through novelty search.
In the second part of the tutorial, the participants will have a chance to try neuroevolution in a challenging real-world problem: Designing intelligent agents for the NERO machine learning game. Through a series of tutorials and excercises, the promise and limitations of the approach will be made concrete, making it possible to develop new neuroevolution applications and techniques in the future.
(1) NERO:
Mac
version
Windows
version (a possible problem and a fix)
32-bit
Linux (suggestions for 64-bit linux).
(2) tutorial.cs.dso
The same file works in all platforms. You need to save it in your NERO
application folder, replacing the tutorial.cs.dso there. That folder is
in MacOS: NERO-2.0/nero/client/scripts/
in Windows: Nero\2.0\nero\client\scripts (a possible problem and a fix)
in Linux: nero2_linux_i386/nero/client/scripts/
Towards the end of the exercise, if you wish to share your team, please email it to risto@cs.utexas.edu. It will then be placed here so that other participants can test their team against it (by downloading, placing it in nero/data/saves/brains subdirectory, and then selecting it as an opponent in the battle mode).