Neuroevolution is a new and emerging area of reinforcement learning (RL). Whereas the traditional value-function-based approach focuses mostly on MDP problems and on maximizing lifetime reward, neuroevolution work focuses mostly on POMDP tasks and on maximizing reward at the end of learning. Knowledge of neuroevolution should thus be most valuable for researchers and students in robotics, intelligent agents, and multiagent systems.
In the presentation part of this tutorial, Neuroevolution RL will first be motivated and compared with value-function RL. Neuroevolution techniques will then be reviewed and illustrated with applications in control, robotics, artificial life, and game playing. Throughout, future research opportunities will be highlighted, including generative and developmental approaches to neuroevolution, combining learning and evolution, and open-ended evolution through novelty search.
In the hands-on part of the tutorial, the participants will have a chance to try neuroevolution in a challenging real-world problem: Designing intelligent agents for the NERO machine learning game. Through a series of tutorials and excercises, the promise and limitations of the approach will be made concrete, making it possible to develop new neuroevolution applications and techniques in the future.
(1) NERO:
Mac
version
Windows
version (a possible problem and a fix)
32-bit
Linux (suggestions for 64-bit linux).
(2) tutorial.cs.dso
The same file works in all platforms. You need to save it in your NERO
application folder, replacing the tutorial.cs.dso there. That folder is
in MacOS: NERO-2.0/nero/client/scripts/
in Windows: Nero\2.0\nero\client\scripts (a possible problem and a fix)
in Linux: nero2_linux_i386/nero/client/scripts/
Towards the end of the exercise (e.g. after step 27) you may want to test your team against teams that other people have created. NERO comes with a number of such teams, or you can play against a few teams that people attending this tutorial created; you can get such teams from here; download a team file, place it in nero/data/saves/brains subdirectory, and then select it as an opponent in the battle mode. If you want to contribute your team to this archive, please email it to risto@cs.utexas.edu.