Multi-modal Behavior in NPCs (2009)
Author: Jacob Schrum

In this game the player controls the green agent, and evolved neural networks control the NPCs, which are yellow (Fight mode) or red (Flight mode). When the NPCs are yellow, they try to attack the player while the player tries to hit them with a bat. When the NPCs are red they must attack the player while keeping it from escaping (the player no longer has a bat). The challenge is to evolve multi-modal behavior for NPCs so that they can accomplish both tasks.

In the movies below, the player is controlled by a static computer strategy. In Fight mode, this bot constantly moves towards the nearest NPC in front of it while swinging its bat. In Flight mode the bot moves backwards away from the nearest NPC in front of it (away from what it can see).

a. ModeMutation Result

ModeMutation is mutation operator for neuroevolution that adds a new output mode to the output layer of a neural network in order to encourage multi-modal behavior. In the movie, the green bot plays the Fight or Flight game, and the NPCs try to kill the bot. The NPCs have evolved several distinct output modes, and each number displayed on an NPC indicates a different output mode. The NPCs use a baiting strategy against the bot. Notice that the output mode of the NPCs chasing the bot is 0, but that the output mode of the bait alternates between 1 and 2. When the NPCs become red, they are in the Flight task. Here the NPCs generalize the mode 0 behavior to work in the Flight task as well. They have to effectively corral the bot to keep it from escaping.

b. 1Mode Result 1

This shows the comparatively poor performance of the 1Mode method (only one network mode). Each network in the population tends to specialize in certain objectives, but none does particularly well in all of them. These NPCs develop a boring behavior for the Fight trial that avoids all contact with the bot. They do no damage to it, though they also take no damage and manage to live through the whole trial. However, the NPCs do well in the Flight trial that follows and exhibit corralling behavior. Doing damage in the Fight trial is sacrificed for the sake of the other objectives, thus putting this individual near the edge of the trade-off surface between objectives.

c. 1Mode Result 2

Another movie showing how the 1Mode method does not simultaneously optimize all objectives. This network meets the goal of killing the bot once (50 damage), but all NPCs are killed! Also, in the Flight trial following the Fight trial, the NPCs perform very poorly and let the bot escape very quickly. NPCs in this same 1Mode population that actually did well in the Flight task almost invariably did poorly in the Fight task. This is how the population as a whole could reach the required goals in terms of average performance, yet still fail to contain individuals that do well in all objectives.
Jacob Schrum Ph.D. Alumni schrum2 [at] southwestern edu
Risto Miikkulainen Faculty risto [at] cs utexas edu
IJCNN-2013 Tutorial on Evolution of Neural Networks 2013
Risto Miikkulainen, To Appear In unpublished. Tutorial slides..
Evolving Multi-modal Behavior in NPCs 2009
Jacob Schrum and Risto Miikkulainen, In IEEE Symposium on Computational Intelligence and Games (CIG 2009), pp. 325--332, Milan, Italy, September 2009. (Best Student Paper Award).
BREVE Monsters BREVE is a system for designing Artificial Life simulations available at http://spiderlan... 2010