Many challenging sequential decision-making problems require
agents to
master multiple tasks, such as defense and offense in many games. Learning
algorithms thus benefit from having separate policies for these tasks, and
from knowing when each one is appropriate. How well the methods work depends
on the nature of the tasks: Interleaved tasks are disjoint and have
different semantics, whereas blended tasks have regions
where semantics from different tasks overlap. While many methods work
well in interleaved tasks, blended tasks are difficult for methods with
strict, human-specified task divisions, such as Multitask Learning.
In such problems, task divisions should be discovered automatically.
To demonstrate the power of this approach, the MM-NEAT neuroevolution
framework is applied in this paper to
two variants of
the challenging video game of
Ms. Pac-Man.
In the simplified interleaved version of the game,
the results demonstrate when and why such machine-discovered task
divisions are useful. In the standard blended version of the game,
a surprising, highly effective machine-discovered task division
surpasses human-specified divisions, achieving the
best scores to date in this game. Modular neuroevolution is thus a
promising technique for discovering multimodal behavior for challenging
real-world tasks.
[ Winner of the GECCO-2015 Best Paper Award in Digital Entertainment and Arts ]
[ An
expanded version of this article appears in ECJ ]
